• Predictive Analytics for Improved Cost Management



Blog



I just recently attended the 28th Annual IEEE Software Technology Conference (STC) sponsored by IEEE and hosted and the National Institute of Standards (NIST).    The conference provided attendees with incredible quality content – 8 wonderful keynote sessions and 51 great presentations (OK – I didn’t attend all of them obviously but the ones I did were insightful, useful and informative.  STC was founded in 1989  by the Assistant Secretary of the Air Force for Acquisition (Communications, Computers, and Support Systems), Mr. Lloyd Mosemann, through the Software Technology Support Center (STSC) at Hill Air Force Base.  The purpose of the ...
Here’s a great article written by Charles Symon’s discussing current challenges in software project estimation and how different organizational cultures deal with them differently.  http://nesma.org/2015/10/chaotic-and-controlled-software-project-estimating/.  According to the author, in a study of 105 software projects completed between 1997 and 2007 in the UK public sector – 30% were terminated, 57% experienced cost overruns averaging 30.5% and 33% suffered major delays. (“Cost over-runs, delays and terminations: 105 outsourced public escort ICT projects – D. Whitfield).  And this is only one study; any of us who live in the software world have heard more than our share of tales of ...
We’ve kicked off a study of the cost impacts of various development standards, and this post discusses a customer request on the cost impacts of IEEE/EIA 12207. IEEE 12207 establishes a common framework for software life cycle processes, with well-defined terminology that can be referenced by the software industry [1].  Adherence to this standard helps to eliminate misunderstandings between contractors and procurers and significantly improves chances of mission success, a major part of which is preventing cost and schedule overruns [2, 3]. IEEE 12207 contains a set of management, engineering, and data requirements for all parties involved (acquirers, suppliers, developers, operators, ...
Being that software measurement is a topic of great interest, I will occasionally look to Google for inspiration and trends.  Often these searches will return several of my own papers and it’s fun to revisit them and think about how much as things change, they remain the same. Recently, I stumbled across a paper I had written many years ago – it was in fact the second or third article that I had published in a magazine (see the article here).  I read through it briefly and was really struck by how relevant it still is today.  The paper ...
Original Post Date: Wednesday, April 14, 2010 The first time I remember hearing the term, “data driven estimating,” was at an ISPA/SCEA conference in Phoenix about 6 years ago. I thought I had a pretty good concept of what that meant. Then, I started hearing it more frequently about 6 to 9 months ago, and I wasn’t as sure of my understanding. So, I went to the Internet and Googled, “Data Driven Estimating.”  The first 3 pages yield links to statistical, engineering, and technical estimating founded on data; e.g. packet routing, data delivery rate, radar and rain gage data, ...
Original Post Date: Tuesday, June 15, 2010 The GAO just published a report saying that strong leadership is key to planning and executing stable weapons programs. And, evidence is presented to back the claim, including result from study of a subset of the 21% of the 2008 defense programs that were deemed stable - on track with original estimates of cost and schedule. What kind of strong leadership made these programs stable? Things like experience, continuity, and open and honest communication, knowledge-based planning, disciplined execution of plans, and establishment of realistic cost and schedule estimates that account for risk are all cited.  Here’s ...
Original Post Date: Friday, July 9, 2010  While sitting in the operatory chair yesterday, my dentist said something that made me stop. He was complaining about an increasing rate of incompetence and apathy he observes in those delivering services to him. And while I do agree with him in principal, he and I are of the age where some folks label us as grumpy old men. So, it may not be as bad as we think. Regardless, the statement he said he made to the an unfortunate poor-quality service provider was, “If you don’t have the time to do it ...
Original Post Date: Wednesday, September 29, 2010 About 75 aerospace & defense (A&D) sector leaders met at an executive summit in Annapolis, MD several weeks ago. Among the topics was a warning about growth in the amount of overruns in the A&D sector. Deloitte produced an analysis of the overrun of major defense acquisition programs relative to their baseline price which showed an unappealing straight-line growth – 26% (growth over baseline) in 2008, projected to be 46% by 2018. A conclusion being tossed about was that the country cannot afford a future like this. Didn’t Norm Augustine come to a ...
Original Post Date: Monday, October 18, 2010 Some of us remember taking the Iowa tests during our early school days. The Iowa Tests of Basic Skills (ITBS) are standardized tests provided as a service to schools by the College of Education of The University of Iowa. The tests, administered to students in grades K-8, became a national standard for measuring scholastic aptitude – I was educated in Pennsylvania. Now out of Iowa comes another test of sorts, something called an Integrity Index Score based upon a proprietary algorithm of an organization called Iowa Live. Iowa Live calls itself, “a ...
Original Post Date: Wednesday, November 3, 2010 The midterm elections are finally over. The themes of reduced spending and lower taxes showed up in force at the ballot box. But what does that mean for the defense industry? The U.S. Secretary of Defense, Robert Gates, caused quite a stir when he announced his proposals for reigning in defense spending. There are the expected assortment of eliminations (U.S. Joint Forces Command and Business transformation Agency to name two), reductions (in service support contracts, number of senior civilian executive and general/admiral military officers, and funding for intelligence community advisory contracts), freezes (of ...
Original Post Date: Wednesday, May 12, 2010 From my perspective as a cost researcher, the calibration tool is one of the most powerful analysis capabilities built into the TruePlanning cost management software . One way I can use this tool is to go back to an old estimate for a project that is now completed, and analyze the correctness of the previously entered input values. With this analysis, I can find ways to improve our methods of soliciting input values from the user to ensure the best values are entered the next time. This way, the TruePlanning models keep getting “smarter” as new information ...
Original Post Date: Friday, June 4, 2010 One of the great features of the TruePlanning cost management software is the fact that it makes it easy to handle complications of inflation and estimating projects performed in different countries and currencies. The costs associated with doing work in different countries, and the relative value of different currencies is constantly changing. To address this, the cost research team at PRICE does an annual economic update performed by the cost research team, and this blog will introduce some of basic concepts and research that goes into maintaining this feature every year. The price of goods and ...
Original Post Date: Thursday, June 24, 2010 "And me boss. And me boss. And me boss!" Just like Bugs Bunny tricking the mob boss into an unfair share of the loot, who doesn’t want a piece of the action. In this case the “action” is the estimate you have just finished in TruePlanning and would like to share with your coworkers. No problem, just share your project. Project sharing is a feature that is available to users who are using the Client/Server version of TruePlanning and it allows users to access projects that have been created on the centralized database by other users. If ...
Original Post Date: Monday, April 26, 2010  With so many acquisition programs over budget and behind schedule, the term “Cost Realism” is suddenly very popular. In my experience as an estimator on many major acquisition programs, two things have remained certain over years (besides death and taxes). First, the probability of the program ever achieving the original cost estimate is exactly zero and second, the more information that is known about a program, the more it will exceed its original cost estimate.    With that said, the move to Cost Realism is so important because it recognizes these two fundamental ...
Original Post Date: Friday, May 21, 2010 Last month I blogged about the importance of cost realism, its roots and how as estimators we must always reflect the truth, no matter how unpopular. This month I want to share with you a recent experience on a Source Selection. As part of the Source Selection team, my role was to conduct a Cost Realism estimate on each of the performers submitting bids. I want to share with you a few insights from that experience. One of the first rules I always follow is to never ask engineers to provide data that ...
Original Post Date: Wednesday, June 30, 2010  I recently had the opportunity to work directly for one of our clients on a high visibility, must-win proposal. The contractor was just about ready to commit to the bid number, but wanted to know the likely bids of the other two performers. We were asked to do a “Ghosting the Competition” study where we ethically collect open source data on two competing designs and combined with engineering technical data to develop a best cost estimate of the competitor’s bid positions.   Unfortunately, not much intelligence was known about the competing configurations, but the ...
Original Post Date: Tuesday, July 20, 2010 Next month (8/4 @ 12pm EST) I am presenting a webinar to discuss using TruePlanning on Source Selections. What prompted me to develop this webinar were the many recent success stories I’ve had using TruePlanning during the Source Selection process. Going a bit further, I am going to show an actual case study where TruePlanning was used to conduct an Analysis of Alternatives (AoA) exercise – along with cost/effectiveness results. We will explore a bit about the technical side of the proposed designs, develop the modeling in TruePlanning and discuss the results. In addition, we will explore ...
Original Post Date: Wednesday, September 1, 2010 I had expected to present my webinar,  “Best Practices for Cost Effectiveness Studies using TruePlanning” in early August. As you might know, I was planning to show a real world example from a recent engagement with a government customer. Unfortunately, since the Source Selection has not concluded with a downselect, I was not able to obtain the public release in time. However, for this month’s blog I will continue share some of the highlights of the webinar.   In last month’s blog we explored the uses of TruePlanning during Source Selection from the Supplier’s (or ...
Original Post Date: Tuesday, October 12, 2010 When I glanced at the Washington Post on Sunday, the following headline screamed out: Defense cuts could slow D.C. economy for years The article basically covers how Defense Secretary Robert M. Gates is calling for reducing spending on "support contractors" by 10 percent each of the next three years as the Defense budget shrinks. As Washington DC is a hub for these types of companies, the impact is expected to be significant. According to the article, more than a quarter of national defense spending contains of outlays for service contracts. Among the largest companies ...
Original Post Date: Wednesday, November 10, 2010 I was recently struck by Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). Within this broad sweeping memo, Ash Carter outlines 23 principal actions in five major areas aimed at increasing efficiency in Defense acquisition.  The first major area covered is “Target Affordability and Control Cost Growth”. Within this major area, program managers must treat affordability as a requirement before milestone authority is granted to proceed (starting with Milestone A). This ...
Original Post Date: Friday, December 17, 2010 In last month’s blog I wrote about Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). I concluded the TruePlanning unified framework and comprehensive cost models, is a tool very well suited to provide the types of analysis outlined in the memorandum. In terms of Should Cost and Independent Cost Estimates (ICE), TruePlanning estimation software provides the industry standard capability to conduct Should Cost and calibration (actual program history) for ICE. Most ...
Original Post Date: Wednesday, June 23, 2010 Parametric modeling is excellent for all aspects of early-concept cost estimation, including go/no-go decisions downstream. So, in the spirit of bringing a transparency to (ethical) financial engineering… why not apply our craft to pricing “real-options”? The latter are essentially strategic opportunities for engaging resources (cost/schedule) into projects, ventures, investments, or even abandonments. The opportunity choice has value itself!  Unlike static project Net Present Value (often, but not exclusively, approximated with Discounted Cash Flow) assuming pre-defined decisions, real-options reflect the merit of flexibility. If an R&D or proof-of-concept presents viability/ marketability learning, the option has positive value, above ...
Original Post Date: Friday, June 25, 2010  Like titanium and other exotic metal-materials, “composites” (by definition, combinations of materials) offer significant weight-savings and reduced part counts, but at a price of high production cost. Sound contrarian to our parametric cost estimating view?   Not really. Complexity of manufacture is quite higher. Likewise process index and structural tooling values grow. Plus, design lead times drive developmental cycles. That said, understand that composites represent more than a material type. They can involve a highly labor-intensive approach to preparing, braiding/ winding, molding, bonding and modular assemblage. Yes, some aspects of braiding and molding lend themselves to automation—which then drives tooling ...
Original Post Date: Thursday, October 7, 2010 Ahhhh, the 80s… a challenging (but often confusing) time in an evolving computing world.  Working in 1985 as a software estimator as well as SQA engineer in a quality assurance department that “audited” real-time projects using new concepts like OOD & OOP… well, you get the picture.  It was a great time to get immersed into great work.  And the good news:  that company’s process as well as its developers were bullish on a young estimation/ quality types asking plenty of questions… as long as they were of the Yes-No variety.  And ...
Original Post Date: Monday, December 6, 2010  In his August blog-entry here, Zach Jasnoff outlined typical client perspectives for the different types of analyses that TruePlanning can accommodate. Working on a large project, we’ve experienced situations that, realistically, can happen where the initial intent and model structuring later have the boundaries of model appropriateness stretched. An Analysis of Alternatives (AoA), for example, is meant to measure deltas between baseline and its alternatives. If common costs “wash” then they can be excluded… which becomes an issue when treated as a Rough Order Magnitude for customer budgeting.  Likewise, if a ROM or Independent Cost Estimate ...
Original Post Date: Thursday, August 12, 2010 The late Norm Crosby’s “Quality is Free” taught us that an investment into quality is more than offset by prevention of defects based upon understanding of requirements. Only with the latter can lack of conformance (and subsequent costs) be captured and hence quality quantified. So how then is Parametrics relevant?  Parametric estimating is more than cost modeling. Our craft represents an initial consulting function into the accuracy and completeness of program planning concepts. Our customers trust us to know when to ask and when to supplement. Yes, we are mathematical and financial modelers too. But I’d suggest that “Parametrics is ...
Original Post Date: Tuesday, August 24, 2010 Over the past several weeks several users have inquired about the best way to estimate costs associated with porting existing software to a new hardware environment. Normally for this situation some of the existing software will require some amount of adaptation to operate on a new server. However, a large portion of the existing software will only require integration into the new environment.   Estimating software costs associated with the above will require the use of several cost objects: - Systems cost object if program management, Quality Assurance, configuration, and    documentation costs are to be included in ...
Original Post Date: Tuesday, April 20, 2010 In September of 2009 the United States Government Accountability Office (GAO) submitted a report[1] discussing the lack of robust Analysis of Alternatives for weapons systems. The report indicated that … “Cost, schedule, and performance problems in the Department of Defense’s (DOD) weapon system programs are serious. Why is it that DoD weapons programs experience a simultaneous cost growth and performance degradation? I believe the answer is found in unrealistic cost estimates and schedule estimates mostly driven by pressure to win a program within a certain budget constraint. Excessive requirements change either through poor ...
Original Post Date: Monday, June 7, 2010 Currently we are exploring the best approach to including a more comprehensive cost estimate for Total Ownership Costs (TOC) into TruePlanning. The current version of the software has focused on development and production costs with some life cycle costing including. The life cycle costs included are focused on the system specific O&S costs such as initial spares for priming the supply pipeline, maintenance, replenishment spares, etc. It is a system view as opposed to a program view of TOC. As we better understand the need to conduct affordability studies it has become clear that design decisions ...
Original Post Date: Thursday, July 15, 2010  “I know that you believe you understand what you think I said, but I'm not sure you realize that what you heard is not what I meant.” This is a famous quote by author and US. State Department spokesman Robert McCloskey (September 15, 1914 – June 30 2003) and was attributed to him by Marvin Kalb, a CBS reporter, in TV Guide 31 March 1984. This quote was in reference to a press briefing during the Vietnam War. What it really addresses is the ease to which the communication of the meaning / understanding ...
Original Post Date: Monday, September 20, 2010 I have been fortunate in my career to have been associated with some great mentors. Each individual has provided me a bit of a golden nugget to carry with me as I tried to navigate my way through the professional waters. My first “civilian” manager, after I left the service and joined industry, provided me a list of the Laws of Analysis (I had just started a position as an operations research analyst). He explained that this list was a mix of serious and tongue in cheek snippets of wisdom. I looked at ...
Original Post Date: Wednesday, September 29, 2010 In May of this year the Washington Post published an editorial article on the need to reduce waste in the Defense Department. The byline of the article was “Defense Secretary Gates’s war of necessity against wasteful spending.” In this article the writer points out that the secretary is taking on the challenge of maintaining our military force [at reasonable level of effectiveness] during a time in which the President and Congress are seeking cost savings / reductions based on the decrease in our presence in Iraq.  Mr. Gates goal is to look for efficiencies ...
Original Post Date: Monday, November 8, 2010  Margaret Wolfe Hungerford in 1878 in her book Molly Bawn coined the phrase …”Beauty is in the eye of the beholder”. This concept of the “value of beauty” has been expressed by others such as:    Benjamin Frankin in Poor Richards Almanack 1741 when he wrote;                 “Beauty, like supreme dominion                 Is but supported by opinion” David Hume in Moral and Political 1742 “Beauty in things exists merely in the mind which contemplates them.”   So what does this have to do with Cost Benefit? Well Merriam-Webster dictionary defines benefit as something that provides useful aid. Inherent in the term “useful” is ...
Original Post Date: Wednesday, August 25, 2010 To me the greatest strength of TruePlannning®  is its flexibility and wide range of parameters. AND, to me the greatest weakness of TruePlanning®  is its’ flexibility and wide range of parameters. The nature of the TruePlanning® framework, the idea of cost catalogs with cost objects, and the implementation of activity based costing allows or provides for a wide range of solutions to our cost estimating requirements. TruePlanning’s ® ability to address a wide range of cost problems lies in its flexibility and extensibility. The clever use of worksheets, multipliers, and the product breakdown ...
The CRASH (CAST Research on Application Software Health) report for 2104/2015 is out and an Executive Summary can be downloaded for free from this link.  This is the third biennial report produced by CAST based on an analysis of the data collected by their  AppMarq static code analysis tool to develop a report on the health of software projects based on their structural quality.  Structural quality speaks to the engineering goodness of the architecture and code for an application, rather than the functional quality that results by delivering software that solves users’ problems.  CAST determines structural quality of code ...
Introduction: The goal of this blog is to show how data can flow between TruePlanning and ProPricer. This walkthrough is based on estimating a software application that will provide users the ability to track objects orbiting the Earth using a feed from some fictitious data stream. The benefit is the ability to get the labor requirement (effort in hours) estimated by TruePlanning into ProPricer in a seamless, easily repeatable process.   1. Create ProPricer Proposal for the Orbiting Body Tracking application The first step is to create a proposal in ProPricer with a WBS. Each task in ProPricer will have a set of ...
Original Post Date: Monday, August 18, 2014 I had the distinct pleasure last week of attending the 2014 NASA Cost Symposium.  While to the uninitiated this might sound like a bit of a snoozer – it was actually quite interesting and proved to be the source of a ton of valuable information.  The event took place at Langley Research Center in Hampton, VA – near Williamsburg, Newport News, and not too far from Virginia Beach.  My participation was somewhat self-serving in that I was there to talk about PRICE’s new Space Missions Cost Models for TruePlanning®.  This model – discussed ...
Original Post Date: Friday, July 25, 2014 July 2014 marked the 25th anniversary of Neil Armstrong’s historic stroll on the moon.  If you go to the NASA website and select Missions you’ll probably be amazed at the number of missions in NASA's past, present, and future.  Unless you’re living under a rock, you know about the International Space Station, and the Hubble telescope but I’m guessing there’s a lot about space missions that many of us are unaware of.  The Dawn spacecraft, which was launched in 2007 from Cape Canaveral, was sent into Space to help NASA scientists learn about the history ...
Original Post Date: Thursday, July 17, 2014 Introduction Parametric cost estimates provide high quality, defendable estimates early in a project’s life cycle. This makes them ideal when producing bid and proposals. The nature of parametric cost estimates, however, requires the results of the estimate to be framed in terms of specific CERs and Activities and Resources. It is common for an organization to have a more granular set of Resources than the ones used to support the CERs. One approach to resolving this issue would be to use the TrueMapper application from PRICE Systems to map TruePlanning Resources to a more ...
Original Post Date: Thursday, July 10, 2014 Introduction TruePlanning provides a powerful and highly customizable reporting environment. Project data can be viewed from many different perspectives and those perspectives can be saved and reused across all projects. Results can be exported to Excel and Word as custom reports. There are, however, some instances where there is a need to get beyond the two axes used in TruePlanning’s reporting engine. This need is sometimes expressed by users preparing TruePlanning cost estimating project data for use in a bid or proposal. Perhaps the data needs to be split by phase and labor/non-labor over ...
Original Post Date: Tuesday, July 1, 2014 Whether you’re doing a software cost estimate to support a Bid and Proposal effort, a software valuation, should cost analysis, or to develop a detailed project plan, it is vitally important to understand the ‘size’ of the software you are estimating.  The problem with software size is that it tends to fall into the intangible realm of reality.  If you tell me you are building a widget that weighs 13 pounds, I can really start to get my head around the task at hand.  If I’m chatting about this with my European colleagues, ...
Original Post Date: Friday, June 20, 2014 Proposal estimates based on grassroots engineering judgment are necessary to achieve company buy-in, but often are not convincing or not in sync with the price-to-win.  This contention can be resolved through by comparing the grassroots estimate to an estimate developed using data driven parametric techniques.  Parametric estimates apply statistical relationships to project data to determine likely costs for a project.  Of course, for a parametric model to properly support this cross check of the grassroots estimate, the proper data must be fed into the model.  This most likely requires the estimator to reach ...
Original Post Date: Wednesday, April 2, 2014 Introduction Parametric estimates provide reliable, reproducible, and flexible views into cost and effort so it’s only natural to want to include this data in a bid and proposal workflow. With TruePlanning 2014 big steps have been taken to make such integration seamless and easily reproducible.  New tools in the TruePlanning suite of products, as well as, integrations with some of the major bid and proposal software applications are at the heart of this new feature set. You can learn more about TruePlanning 2014 and the PRICE cost estimation models at our website, but let's ...
Original Post Date: Thursday, March 20, 2014 Here’s a conundrum.  You are a software estimator responsible for helping the decision makers in your company determine what business to pursue and what business to steer clear of.  You know, that to win profitable business, your company first needs to decide which opportunities are golden and which should be avoided.  You also know, that at the point at which this decision needs to be made, there is very little information available to support a quality estimate.  Add to this the fact that software estimation is hard   at almost any stage.  What’s ...
Original Post Date: Thursday, March 20, 2014 One of the complications in generating Bids and Proposals for Modules and Microcircuits is determining the “Should Cost” for better cost realism. Most of the electronic modules and their components in the proposals are not actually manufactured by the Proposer, but rather by a subcontractor, thus becoming a Purchased item. It is difficult to determine the cost of making the Module, and determining a fair cost. Costs for the modules include Assembly and Test costs together with the component costs. Components such as ASIC’s (Application Specific Integrated Circuits), have both the cost of developing the devices and ...
Original Post Date: Friday, October 4, 2013 True Planning results have many options, including viewing Costs by Activity. While simple, this view can be quite powerful, especially when exported for re-organization manipulation. In a recent exercise, the WBS mapping of common objects, estimated by separate multiple scenarios, presented a non-trivial chore in Excel. “Transposition” features work fine for matrices, as do pivot tables. But how does one map object by activity grids into activity lists, similar to MIL-STD 881a, with singular “roll up” instances of all nonzero object costs? The secret is in how True Planning appends each activity output with the ...
Original Post Date: Friday, October 4, 2013 The late Norm Crosby’s “Quality is Free” taught us that an investment into quality is more than offset by prevention of defects based upon understanding of requirements. Only with the latter can lack of conformance (and subsequent costs) be captured and hence quantified towards quality. So how then is Parametrics relevant? Parametric estimating is more than cost modeling. Our craft represents an initial consulting function into the accuracy and completeness of program planning concepts. Our customers trust us to know when to ask and when to supplement. Yes, we are mathematical and financial modelers ...
Original Post Date: Friday, October 4, 2013 In his August 2010 blog-entry, Zac Jasnoff outlined typical client perspectives for the different types of analyses that True Planning can accommodate.  Working on a large project, we’ve experienced situations that, realistically, can happen where the initial intent... and model structuring… later has the boundaries of model appropriateness stretched.  An AoA, for example, is meant to measure deltas between baseline and its alternatives.  If common costs “wash” then they can be excluded… which becomes an issue when treated as a Rough Order Magnitude for customer budgeting. Likewise, if a ROM or ICE of ...
Original Post Date: Friday, October 4, 2013 I’m not a golfer. But we’ve all heard one say “that’s why I play” after hitting a shot and feeling like it all came together. What “it” is, in terms of mechanics and timing, I’m not really sure. In our own world of parametrics, it’s the feeling of adding value in that golden moment of facilitating decisions and forward momentum. We wear many hats: estimating, consulting, systems engineering...even cost accounting. Building an AoA, ICE or ROM is where rubber-meets-the-road in regards to configurations and assumptions. Not too long ago I was in a discussion with ...
Original Post Date: Friday, October 4, 2013 In Parametrics is Free, I acknowledged receiving (too late) “you should’ve known to ask that” over the years. Quality control after-the-fact is fine; but it’s better and cheaper to take a systematic approach to quality assurance as part of your estimating process. The sheer volume of what we model can often keep us so close to the details that we are unable to step back and put on our QA hat on for a sanity check. Enter Quality! On a very large project, our team has introduced a few regular cross-checks, notwithstanding typical ...