by Joe Bauer
| September 25, 2014
Risk Analysis Methodology Overview – Method of Moments
In this second of three articles on risk analysis, we will discuss Method of Moments. Method of Moments (MoM) is an alternative to Monte Carlo simulation. Along with the methodology, we will present some pros and cons of using MoM over Monte Carlo.
What is a moment?
Before we discuss the methodology behind MoM, we first need to talk about moments. Caution: for all the master statisticians out there, this article is meant to boil down complex topics in an easy to understand manner. There are obviously more complex aspects to MoM and Monte Carlo risk analysis than I present here! Moments are nothing more than characteristics of a distribution. In this case, we refer to characteristics of an input distribution of our risk analysis. With the mean and variance of a distribution, we understand the location and variability of our data. Skewness and kurtosis provide an indication of the shape of our distribution. You have likely calculated, used, or at least heard of the mean, variance, skewness, and kurtosis of a distribution. You may have just not realized these were moments. As a quick refresher, we will discuss the 4 moments now. Refer to the triangle and description of the variables to orient yourself as you review the formulas.
a b c
Optimistic Most Likely Pessimistic
The first moment is the mean of the distribution. We all should recognize what the mean is. It is nothing more than a simple average of our data points, or (x1 + x2 + x3 + …. + xn) / n. Using our triangular distribution above, with the optimistic point represented by a, the most likely point represented by b and the pessimistic point represented by c, the calculation of the mean is very simple: (a + b + c) / 3.
The second moment is the variance, which of course provides an indication of the spread of our data points. Complex distributions may require slightly more complex calculations. However, again relying on our triangular distribution, the variance is calculated as: (a2 + b2 + c2 – ab – ac - bc) / 18.
The third moment is the skewness of the distribution. Skewness is a measure of the symmetry of the distribution. Perfectly symmetric distributions have a skewness of zero. Positively skewed distributions, with a long tail in the positive direction, have a skewness value that is positive. The opposite goes for negatively skewed distributions. Since we are limiting the inputs to a triangular distribution, the skewness is: ( 20.5 (a + b – 2c) (2a – b – c) (a – 2b + c) ) / ( 5 (a2 + b2 + c2 – ab – ac – bc)1.5 ).
The fourth moment is the kurtosis of the distribution. Kurtosis provides an indication how flat or peaked our distribution is. As luck would have it, we simply use a constant to reflect the kurtosis of triangular distributions: 12/5 or 2.387.
What is the “Method” in Method of Moments?
When we supply a range of pessimistic and optimistic values around our inputs and form those triangular distributions, we are also determining all of the necessary moments of the triangular distributions. Now, we can put that information to work and come up with an output, or resulting cost distribution, using the moments of all the inputs in our estimate.
In TruePlanning®, the MoM risk analysis methodology is used. Specifically, one particular MoM approach knows as FRISK is used. FRISK (Formal Risk Assessment of System Cost Estimates) was developed in 1990 by Philip Young of The Aerospace Corporation for the Air Force Space and Missiles Command. FRISK is simple, fast, and accurate. Furthermore, we only need to calculate two moments of our input distributions (mean and variance) to develop the resulting output cost distribution.
The means of all input distributions are summed to determine the mean of the output distribution. The calculation of the variance is a bit more complex, because we need to have some understanding of the correlation between the lower level WBS elements. Please stay tuned for an upcoming article on correlation, where we demystify the topic and show how correlation plays a key role in risk analysis, regardless if we use MoM or Monte Carlo simulation.
We then use the mean and variance (with correlation included) for the output distribution and calculate parameters (P, Q) of the lognormal distribution.
P = ½ ln (mean4 / (mean2 + variance))
Q = (ln(1 + (variance / mean2))0.5
Knowing P and Q allows us to know the shape of the lognormal cost risk distribution. P provides the probabilities of values in the distribution and Q is simply 1 – P. The values of P and Q, along with the mean and variance are all we need to come up with our typical cost risk analysis output. One outstanding resource on the exact formulas, descriptions, and methodology behind FRISK can be found at http://www.nasa.gov/pdf/741989main_Analytic%20Method%20for%20Risk%20Analysis%20-%20Final%20Report.pdf.
While some of these calculations may seem a bit complex, the truth is we can accomplish all we need to do in a simple spreadsheet tool. We can calculate the output cost risk distribution with a few formulas instantly and have our results. Thankfully, TruePlanning® makes these calculations for us!
The primary benefit of using MoM over Monte Carlo simulation is speed. Recall that Monte Carlo simulation requires a large number of trials to build the output cost distribution. MoM can be executed immediately because the results are based on few relatively simple mathematical operations easily handled by any statistical or spreadsheet software.
Simplicity is the second biggest advantage of MoM over Monte Carlo. Input distributions are simple. With optimistic, most likely, and pessimistic values, we are able to form triangular distributions for inputs, and use the resulting computations to develop a logarithmic output distribution. However, simplicity can be an advantage or a disadvantage, as we will discuss next.
Fast and simple is great, as long as we are dealing with simple inputs that are best modeled with a triangular distribution. Many inputs, or risk boundaries around inputs, do not behave in such a manner. For instance, software growth should not be modeled with a triangular distribution. Assuming a finite upper limit to software code size may result in understating the true risk in extreme circumstances. Improvements to the FRISK methodology allows for other types of inputs, as discussed in some of the references cited below. In particular, Covert, R. P. (2006). Correlations in Cost Risk Analysis. 2006 Annual SCEA Conference. Tysons Corner, VA. will prove to be a valuable resource.
Method of Moments is an alternative to Monte Carlo simulation when performing risk analysis. Using the characteristics, or moments, of triangular input distributions, we are able to develop a lognormal output distribution. The lognormal output distribution can be used to determine our confidence level in the point estimate, the mean, or any other confidence level for risk analysis or budgeting purposes.
Method of Moments is quick and simple. There are advantages to the simplicity and speed, but these advantages come at the expense of relying on potentially over simplified or inappropriate input distributions.
Correlation is a major component of any risk analysis. Throughout our discussion on MoM and Monte Carlo simulation, we have sidelined the topic. I will soon post an article to explain the need for correlation when performing risk analysis.