2005 PRICE INSight
Q&A Special with PRICE Systems' Anthony DeMarco: PRICE President & CEO Provides Valuable Insights for Complying with OMB Independent IT Validation Mandate
A recent mandate from OMB's Office of Electronic Government and Information Technology (OEGIT) requires Government agencies to perform an independent validation of cost, schedule, and performance goals for all major IT projects, including new, ongoing development, and high risk projects. Objectives are to better ensure improved execution and performance, as well as promote more effective oversight. Agencies that do not comply with the requirement by March 2006 cannot obligate funds for such projects.
PRICE President and CEO Anthony DeMarco, a pioneer of cost forecasting and analysis technology, occupies a unique vantage point from which to comment on the OMB mandate. In 2001, he was appointed by then NASA Administrator Daniel S. Goldin to serve on the International Space Station Management and Cost Evaluation Task Force (IMCE). He helped NASA to address cost growth on the program by assessing the quality of the ISS cost estimates as well as program assumptions and requirements, and by identifying high-risk budget areas and potential risk mitigation strategies. The following are Mr. DeMarco's responses to our questions on the OMB's independent IT project validation requirement.
1) What are the necessary elements in implementing OMB's recent mandate on new and current IT programs?
The three most critical elements of independent validation are speed, credibility, and transparency.
Speed
Programs are dynamic. They are constantly morphing to meet changing requirements, resource turnover, and new technology. There is nothing worse than validating an outdated baseline, therefore the basis of the independent validation needs to be as agile as the program team to keep up with the dynamics. That is why "models" are so important. Capturing the relationships among the program's performance, products, activities and resources in an agile cost and schedule model empowers the program to perform the right estimate at the right time.
Credibility and Transparency
Independent estimates will be challenged by program offices, contractors, agency management and Congress. To be believed, the basis of the independent estimate must be easily understood, logical, and based on fact. Therefore, transparent, open models based on tangible benchmarks are necessary to withstand the scrutiny.
2) Can you clarify what an independent assessment of a program means?
An independent validation means that a non-advocate party reviews, challenges, and renders an independent opinion of the program's bases of estimates for performance, cost, schedule and risks. Non-advocacy is extremely important. For the validation to be credible, the performing party's future must not depend in any way on the successful continuance or cancellation of the program. Also, the validation party must challenge the fundamental size and complexity assumptions of the program. I have seen too many so-called independent validations based on the program office or contractor estimates of size and complexity. This is estimating malpractice. Any good estimator will challenge and render their own opinion of these critical cost and schedule drivers.
3) What should a CIO look for when acquiring the services of an Independent organization to perform a validation of cost, schedule and performance of a program?
As discussed previously, they should first look for true non-advocacy. Then they should look for experience: proven models, proven methods, subject matter expertise, and a knowledge-base of relevant benchmarks.
4) How does a validation of cost, schedule and performance help manage the affordability of a program thus ensuring the program's success?
Because optimism reigns supreme among program offices and contractors, independent estimates are typically higher than program office estimates and are typically more accurate. Therefore, agency management must decide how to best use the independent estimate based on their risk tolerance. Some may re-baseline the program budget to the independent estimate and make hard choices about other projects in the program portfolio. Others may use the independent estimate to identify risks that must be mitigated and keep a budget reserve in case mitigation plans fail. Other may pass the risk on to the contractor through performance-based contractor incentives. In any case, the independent estimate helps the agency make educated choices.
5) PRICE Systems has worked with organizations like the Department of Defense's Standard Procurement System and the IRS to provide Independent Government Cost Estimate (IGCEs). How were PRICE Systems' Program Affordability Management solutions used to ensure the success of their programs?
The number one reason that programs fail is initial and continuous underestimates. It is important that estimates are right the first time and every time. The solutions we deliver are structured around the disciplines discussed above. First, we assemble a right-size, non-advocate team of subject matter experts and practitioners for the independent estimate. Then we arm them with agile models and relevant analogous program benchmarks. Our consultants are skilled interviewers and data miners. They immediately start probing and challenging the program's size and complexity assumptions, compare them to the benchmarks and form their independent opinion. The next step Is to convert the program's parameters into model cost and schedule drivers, bounding the most uncertain drivers to characterize the program's most risky deliverables. The independent opinion is then rendered with useful charts and reports comparing the baseline estimate with the independent estimate. This new knowledge helps agency management and program offices make more informed decisions to keep programs healthy and adequately funded, leading to program success.
For more information on PRICE Systems' IT Project Validation contact:
1.856.608.7219 or email webmaster@pricesystems.com
