MT. LAUREL, NJ / March 31, 2005 / The 38th Annual DoD Cost Analysis Symposium, "Considering Risk," was hosted by the Office of the Secretary of Defense Cost Analysis Improvement Group (CAIG) on February 15-18, 2005 at the Williamsburg Marriott in Williamsburg, Virginia.
As risk management assumes a significant and growing role in the decision-making process, cost analysis has incorporated various techniques to account for uncertainty in a dynamic political and economic environment. This symposium provided a forum for the exchange of ideas on the subject, but also allowed participants to examine the role of risk analysis across the military, political, and commercial sectors. Presentation topics included:
The symposium was organized into plenary, training and workshop sessions, divided into Advanced, Intermediate and "Considering Risk" tracks. While the plenary sessions and luncheon speakers provided great insights into budgetary decision-making, of particular interest to PRICE were the cost research topics.
PRICE Systems was founded to deliver the findings of 20 years of fundamental cost research at RCA to the aerospace and defense market through application software, training and consulting. The RCA tradition continues with PRICE's cost research department headed by Chief Scientist Arlene Minkiewicz. With great pride in our "Powered by PRICE Research" moniker, we are always interested in third-party findings that can be leveraged for our clients as well as findings that support our ongoing research.
At the DoD Cost Analysis Symposium, Mr. Michael A. Gallo, Technomics Inc., presented "Simple is Not Necessarily Better: Why Software Productivity Factors Can Lead to Bad Estimates". Mr. Gallo's research was funded by a two-year SBIR contract with the objective to determine cost estimating relationships (CERs) for software project estimating. One of Mr. Gallo's findings was the behavior of cost as a function of the most common software sizing parameter, SLOC. Mr. Gallo relayed that the relationship to SLOC was exponential and that the exponent on SLOC was found to be greater than 1.0 (a diseconomy of scale). This productivity penalty for larger projects was surmised to be the result of increased complexity for integration. Mr. Gallo concluded that estimating errors would occur if simple linear productivity metrics measured on smaller projects are used on larger projects.
Our research findings concur with Mr. Gallo's. The PRICE True S model uses a hierarchy of Software Assemblies and Software Components. The model estimates the costs for each component and then looks at the assembly of components that must be integrated into a system. The more components and more SLOC in the system, the higher the complexity of systems engineering and integration activities - yielding exponentially higher costs. PRICE True S analysts will not fall victim to the use of simple productivity measures as described by Mr. Gallo.
The topic for one of the symposium's plenary sessions was "RAH-66 Comanche: A Case Study in Service Decision Making". One-by-one the panelists described a long, expensive program that was eventually cancelled. Repeatedly, problems could be traced back to an inaccurate initial estimate, often of the software systems. One of the greatest features of PRICE True S is the extensive built-in knowledge base of software components and systems. An estimator can quickly choose from a list software application types. PRICE True S then populates the object with input parameters to produce a credible "should-cost" estimate. This benchmark puts the initial estimate in the right range, avoiding the ill-fated assumptions that commonly plague programs from their outset. New knowledge bases are added with every release. Today, the list of knowledge bases available to PRICE True S analysts has grown to include:
Dr. Stephen A. Book, MCR Federal LLC, addressed the symposium on the topic of "Universal Risk Issues in Source Selection". Dr. Book discussed many practical rules of thumb and the hidden dangers of effective source selection techniques. His most prominent warning was on the hidden costs of commercial off-the-shelf (COTS) software. Dr. Book described the many unexpected activities and obsolescence issues experienced with COTS.
Our findings are the same. In 2004, PRICE's Arlene Minkiewicz won many awards for her research and development of the PRICE True S COTS cost-estimating model. PRICE True S COTS discloses the many hidden activities, resources and costs typical of COTS implementations, such as Evaluate and Select, Tailoring and extensive Qualification Testing. It is common for contractor bids evaluated with PRICE True S to be sent back to the contractor for revision to include activities that they did not consider. An analyst preparing an estimate with PRICE True S is well armed with a credible solution that management and decision-makers can trust to ensure that a budget is adequate for project completion.
Mr. Raymond Covert of The Aerospace Corporation presented a paper on "Cost Risk Methods". Several methods and their pros and cons were discussed-one of the most useful and accurate being the FRISK method. The FRISK Methodology was originally developed in 1990 at the request of James R. ("Pete") Barnum, Technical Director of the Cost Division at the USAF Space and Missiles Center at Los Angeles AFB. The principal developer of the method was Philip H. Young, Senior Project Engineer in the Resource Analysis Department of The Aerospace Corporation. FRISK circumvents the correlation issues posed by deficiencies of the "Completely Correlated" and "Completely Uncorrelated" options and is more responsive than Monte Carlo methods.
In 2004, FRISK risk methodology was incorporated into the PRICE TruePlanning estimating suite. Our findings indicated that it was the most robust and responsive method. Now, TruePlanning analysts can rapidly generate mathematically correct probabilistic cost estimates for software and information technology systems using this proven, accepted risk methodology. Decision-makers can quickly identify from a FRISK report the confidence level corresponding to various budget numbers.
The research findings presented by industry sources external to PRICE at the 38th Annual DoD Cost Analysis Symposium supported and added to the findings of our cost research team. PRICE clients can be assured that they are using and will continue to be kept up to date with the latest research and methodologies, continuing a proud tradition we call "Powered by PRICE Research."
"Powered by PRICE Research" is not just a catch-phrase; it's our company philosophy. Powered by PRICE Research represents the industry's most effective model for conducting and capitalizing on independent cost research and analysis.