• Predictive Analytics for Improved Cost Management



Blog



In 2015, the Systems Engineering Research Center (SERC) published the Software Cost Estimation Metrics Manual for Defense Systems. This manual presents the results of research conducted by SERC Researchers Ray Madachy and Brad Clark, who with the help of Barry Boehm, have documented the core metric definitions along with relevant and useful cost estimating relationships for software intensive defense systems. The e-book is available for download at the following link… http://cdn.sercuarc.org/wp-content/uploads/2014/05/Software-Cost-Estimation-Metrics-Manual-for-Defense-Systems.pdf What’s great about this manual is that it covers best practices specific to software systems in the areas of measurement, data collection, normalization, analysis, benchmarking, and CER development.  The ...
Last week I had the pleasure of attending the 30th annual COCOMO Forum in Arlington, VA.  As always it was an informative and enjoyable exchange of software and system estimating topics and challenges with some of the brightest practitioners in the industry.  Topics ranged from technical debt and software maintenance, early phase effort and schedule estimation, COSMIC function points and various data analytics topics.  In addition to the presentations there were several lively workshop panels discussing and debating future directions for the COCOMO and CoSysmo models (contemplating the 3.0 versions of each of these models. While all the talks were ...
Being that software measurement is a topic of great interest, I will occasionally look to Google for inspiration and trends.  Often these searches will return several of my own papers and it’s fun to revisit them and think about how much as things change, they remain the same. Recently, I stumbled across a paper I had written many years ago – it was in fact the second or third article that I had published in a magazine (see the article here).  I read through it briefly and was really struck by how relevant it still is today.  The paper ...
Recently I was asked for more detail about the implementation of COSMIC Function Points as a functional size measure for the software estimation model in TruePlanning® (True S).  Common Software Measurement International Consortium (COSMIC) is a group of software measurement experts who in 1998 recognized the need to improve on the traditional function point methods.  Because the origin of International Function Point User Group (IFPUG) Function Points with IBM, the method tends to favor business systems (though advances have been made to extend IFPUG Function Points with “Software Non-Functional Assessment Process (SNAP) – more on this to come in ...
Original Post Date: Friday, July 9, 2010  While sitting in the operatory chair yesterday, my dentist said something that made me stop. He was complaining about an increasing rate of incompetence and apathy he observes in those delivering services to him. And while I do agree with him in principal, he and I are of the age where some folks label us as grumpy old men. So, it may not be as bad as we think. Regardless, the statement he said he made to the an unfortunate poor-quality service provider was, “If you don’t have the time to do it ...
Original Post Date: Wednesday, May 12, 2010 From my perspective as a cost researcher, the calibration tool is one of the most powerful analysis capabilities built into the TruePlanning cost management software . One way I can use this tool is to go back to an old estimate for a project that is now completed, and analyze the correctness of the previously entered input values. With this analysis, I can find ways to improve our methods of soliciting input values from the user to ensure the best values are entered the next time. This way, the TruePlanning models keep getting “smarter” as new information ...
Original Post Date: Friday, June 4, 2010 One of the great features of the TruePlanning cost management software is the fact that it makes it easy to handle complications of inflation and estimating projects performed in different countries and currencies. The costs associated with doing work in different countries, and the relative value of different currencies is constantly changing. To address this, the cost research team at PRICE does an annual economic update performed by the cost research team, and this blog will introduce some of basic concepts and research that goes into maintaining this feature every year. The price of goods and ...
Original Post Date: Monday, April 26, 2010  With so many acquisition programs over budget and behind schedule, the term “Cost Realism” is suddenly very popular. In my experience as an estimator on many major acquisition programs, two things have remained certain over years (besides death and taxes). First, the probability of the program ever achieving the original cost estimate is exactly zero and second, the more information that is known about a program, the more it will exceed its original cost estimate.    With that said, the move to Cost Realism is so important because it recognizes these two fundamental ...
Original Post Date: Friday, May 21, 2010 Last month I blogged about the importance of cost realism, its roots and how as estimators we must always reflect the truth, no matter how unpopular. This month I want to share with you a recent experience on a Source Selection. As part of the Source Selection team, my role was to conduct a Cost Realism estimate on each of the performers submitting bids. I want to share with you a few insights from that experience. One of the first rules I always follow is to never ask engineers to provide data that ...
Original Post Date: Wednesday, June 30, 2010  I recently had the opportunity to work directly for one of our clients on a high visibility, must-win proposal. The contractor was just about ready to commit to the bid number, but wanted to know the likely bids of the other two performers. We were asked to do a “Ghosting the Competition” study where we ethically collect open source data on two competing designs and combined with engineering technical data to develop a best cost estimate of the competitor’s bid positions.   Unfortunately, not much intelligence was known about the competing configurations, but the ...
Original Post Date: Wednesday, September 1, 2010 I had expected to present my webinar,  “Best Practices for Cost Effectiveness Studies using TruePlanning” in early August. As you might know, I was planning to show a real world example from a recent engagement with a government customer. Unfortunately, since the Source Selection has not concluded with a downselect, I was not able to obtain the public release in time. However, for this month’s blog I will continue share some of the highlights of the webinar.   In last month’s blog we explored the uses of TruePlanning during Source Selection from the Supplier’s (or ...
Original Post Date: Wednesday, November 10, 2010 I was recently struck by Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). Within this broad sweeping memo, Ash Carter outlines 23 principal actions in five major areas aimed at increasing efficiency in Defense acquisition.  The first major area covered is “Target Affordability and Control Cost Growth”. Within this major area, program managers must treat affordability as a requirement before milestone authority is granted to proceed (starting with Milestone A). This ...
Original Post Date: Friday, December 17, 2010 In last month’s blog I wrote about Ash Carter’s (Under Secretary of Defense for Acquisition, Technology & Logistics) Memorandum for Acquisition Professionals, Better Buying Power: Guidance for Obtaining Greater Efficiency and Productivity in Defense Spending (14 September 2010). I concluded the TruePlanning unified framework and comprehensive cost models, is a tool very well suited to provide the types of analysis outlined in the memorandum. In terms of Should Cost and Independent Cost Estimates (ICE), TruePlanning estimation software provides the industry standard capability to conduct Should Cost and calibration (actual program history) for ICE. Most ...
Original Post Date: Wednesday, June 23, 2010 Parametric modeling is excellent for all aspects of early-concept cost estimation, including go/no-go decisions downstream. So, in the spirit of bringing a transparency to (ethical) financial engineering… why not apply our craft to pricing “real-options”? The latter are essentially strategic opportunities for engaging resources (cost/schedule) into projects, ventures, investments, or even abandonments. The opportunity choice has value itself!  Unlike static project Net Present Value (often, but not exclusively, approximated with Discounted Cash Flow) assuming pre-defined decisions, real-options reflect the merit of flexibility. If an R&D or proof-of-concept presents viability/ marketability learning, the option has positive value, above ...
Original Post Date: Friday, June 25, 2010  Like titanium and other exotic metal-materials, “composites” (by definition, combinations of materials) offer significant weight-savings and reduced part counts, but at a price of high production cost. Sound contrarian to our parametric cost estimating view?   Not really. Complexity of manufacture is quite higher. Likewise process index and structural tooling values grow. Plus, design lead times drive developmental cycles. That said, understand that composites represent more than a material type. They can involve a highly labor-intensive approach to preparing, braiding/ winding, molding, bonding and modular assemblage. Yes, some aspects of braiding and molding lend themselves to automation—which then drives tooling ...
Original Post Date: Thursday, October 7, 2010 Ahhhh, the 80s… a challenging (but often confusing) time in an evolving computing world.  Working in 1985 as a software estimator as well as SQA engineer in a quality assurance department that “audited” real-time projects using new concepts like OOD & OOP… well, you get the picture.  It was a great time to get immersed into great work.  And the good news:  that company’s process as well as its developers were bullish on a young estimation/ quality types asking plenty of questions… as long as they were of the Yes-No variety.  And ...
Original Post Date: Tuesday, August 24, 2010 Over the past several weeks several users have inquired about the best way to estimate costs associated with porting existing software to a new hardware environment. Normally for this situation some of the existing software will require some amount of adaptation to operate on a new server. However, a large portion of the existing software will only require integration into the new environment.   Estimating software costs associated with the above will require the use of several cost objects: - Systems cost object if program management, Quality Assurance, configuration, and    documentation costs are to be included in ...
Original Post Date: Monday, June 7, 2010 Currently we are exploring the best approach to including a more comprehensive cost estimate for Total Ownership Costs (TOC) into TruePlanning. The current version of the software has focused on development and production costs with some life cycle costing including. The life cycle costs included are focused on the system specific O&S costs such as initial spares for priming the supply pipeline, maintenance, replenishment spares, etc. It is a system view as opposed to a program view of TOC. As we better understand the need to conduct affordability studies it has become clear that design decisions ...
Original Post Date: Monday, September 20, 2010 I have been fortunate in my career to have been associated with some great mentors. Each individual has provided me a bit of a golden nugget to carry with me as I tried to navigate my way through the professional waters. My first “civilian” manager, after I left the service and joined industry, provided me a list of the Laws of Analysis (I had just started a position as an operations research analyst). He explained that this list was a mix of serious and tongue in cheek snippets of wisdom. I looked at ...
Original Post Date: Monday, August 18, 2014 I had the distinct pleasure last week of attending the 2014 NASA Cost Symposium.  While to the uninitiated this might sound like a bit of a snoozer – it was actually quite interesting and proved to be the source of a ton of valuable information.  The event took place at Langley Research Center in Hampton, VA – near Williamsburg, Newport News, and not too far from Virginia Beach.  My participation was somewhat self-serving in that I was there to talk about PRICE’s new Space Missions Cost Models for TruePlanning®.  This model – discussed ...
Original Post Date: Friday, July 25, 2014 July 2014 marked the 25th anniversary of Neil Armstrong’s historic stroll on the moon.  If you go to the NASA website and select Missions you’ll probably be amazed at the number of missions in NASA's past, present, and future.  Unless you’re living under a rock, you know about the International Space Station, and the Hubble telescope but I’m guessing there’s a lot about space missions that many of us are unaware of.  The Dawn spacecraft, which was launched in 2007 from Cape Canaveral, was sent into Space to help NASA scientists learn about the history ...
Original Post Date: Thursday, July 17, 2014 Introduction Parametric cost estimates provide high quality, defendable estimates early in a project’s life cycle. This makes them ideal when producing bid and proposals. The nature of parametric cost estimates, however, requires the results of the estimate to be framed in terms of specific CERs and Activities and Resources. It is common for an organization to have a more granular set of Resources than the ones used to support the CERs. One approach to resolving this issue would be to use the TrueMapper application from PRICE Systems to map TruePlanning Resources to a more ...
Original Post Date: Thursday, July 10, 2014 Introduction TruePlanning provides a powerful and highly customizable reporting environment. Project data can be viewed from many different perspectives and those perspectives can be saved and reused across all projects. Results can be exported to Excel and Word as custom reports. There are, however, some instances where there is a need to get beyond the two axes used in TruePlanning’s reporting engine. This need is sometimes expressed by users preparing TruePlanning cost estimating project data for use in a bid or proposal. Perhaps the data needs to be split by phase and labor/non-labor over ...
Original Post Date: Tuesday, July 1, 2014 Whether you’re doing a software cost estimate to support a Bid and Proposal effort, a software valuation, should cost analysis, or to develop a detailed project plan, it is vitally important to understand the ‘size’ of the software you are estimating.  The problem with software size is that it tends to fall into the intangible realm of reality.  If you tell me you are building a widget that weighs 13 pounds, I can really start to get my head around the task at hand.  If I’m chatting about this with my European colleagues, ...
Original Post Date: Friday, June 20, 2014 Proposal estimates based on grassroots engineering judgment are necessary to achieve company buy-in, but often are not convincing or not in sync with the price-to-win.  This contention can be resolved through by comparing the grassroots estimate to an estimate developed using data driven parametric techniques.  Parametric estimates apply statistical relationships to project data to determine likely costs for a project.  Of course, for a parametric model to properly support this cross check of the grassroots estimate, the proper data must be fed into the model.  This most likely requires the estimator to reach ...
Original Post Date: Wednesday, April 2, 2014 Introduction Parametric estimates provide reliable, reproducible, and flexible views into cost and effort so it’s only natural to want to include this data in a bid and proposal workflow. With TruePlanning 2014 big steps have been taken to make such integration seamless and easily reproducible.  New tools in the TruePlanning suite of products, as well as, integrations with some of the major bid and proposal software applications are at the heart of this new feature set. You can learn more about TruePlanning 2014 and the PRICE cost estimation models at our website, but let's ...
Original Post Date: Friday, October 4, 2013 In Parametrics is Free, I acknowledged receiving (too late) “you should’ve known to ask that” over the years. Quality control after-the-fact is fine; but it’s better and cheaper to take a systematic approach to quality assurance as part of your estimating process. The sheer volume of what we model can often keep us so close to the details that we are unable to step back and put on our QA hat on for a sanity check. Enter Quality! On a very large project, our team has introduced a few regular cross-checks, notwithstanding typical ...
Original Post Date: Friday, October 4, 2013 Like titanium and other exotic metal-materials, “composites” (by definition, combinations of materials) offer significant weight-savings and reduced part counts, but at a price of high production cost. Sound contrarian to our parametric cost estimating view? Not really. Complexity of manufacture is quite higher. Likewise process index and structural tooling values grow. Plus, design lead times drive developmental cycles. That said, understand that composites represent more than a material type. They can involve a highly labor-intensive approach to preparing, braiding/ winding, molding, bonding and modular assemblage. Yes, some aspects of braiding and molding lend themselves ...
Original Post Date: Friday, October 4, 2013 Parametric modeling is excellent for all aspects of early-concept cost estimation, including go/no-go decisions downstream. So, in the spirit of bringing a transparency to (ethical) financial engineering… why not apply our craft to pricing “real-options”? The latter are essentially strategic opportunities for engaging resources (cost/schedule) into projects, ventures, investments, or even abandonments. The opportunity choice has value itself! Unlike static project Net Present Value (often, but not exclusively, approximated with Discounted Cash Flow) assuming pre-defined decisions, real-options reflect the merit of flexibility. If an R&D or proof-of-concept presents viability/marketability learning, the option has positive ...
Original Post Date: Friday, October 4, 2013 Ahhhh, the 80s… a challenging (but often confusing) time in an evolving computing world. Working in 1985 as a software estimator as well as SQA engineer in a quality assurance department that “audited” real-time projects using new concepts like OOD & OOP… well, you get the picture. It was a great time to get immersed into great work. And the good news: that company’s process as well as its developers were bullish on a young estimation/ quality type asking plenty of questions… as long as they were of the Yes-No variety. And ask ...
Original Post Date: Friday, October 4, 2013 ...wear the worst shoes. The cobbler was a master at his craft; he was just too tired to practice it when he got home from the shop. Sound familiar? A disciplined approach to understanding (functional) requirements as well as analogous projects (with actuals) is our not-so-secret sauce. Why run the risk of creeping back up our career learning curve? There’s already enough scope creep to keep us busy. Plus, for you management types charged with prospecting, a consistent approach towards estimation is a great way to connect with people who've felt the pain of ...
Original Post Date: Friday, October 4, 201 My "Real Options Valuation" blog suggested the use of parametrics in real options valuation. I’d like to offer the generalized use of our type of modeling in valuing tangible assets. Typically, fundamental analysis evaluates the intrinsic value of securities. I won’t attempt to compete with Warren Buffet here. But it is certainly the case that a company, or portfolio of securities reflecting many companies, is based in part on the market value of its product assets and their potential for future earnings, as well as other objective and subjective considerations. In parametric estimation, ...
Original Post Date: Wednesday, September 25, 2013 The “Systems Folder” cost object which is found at the start of every TruePlanning Project is most often confused with the “Folder” icon. These two however should not be confused. The “Folder” icon does not have an input sheet at all. It is not a cost object and contains no cost estimating logic or relationships.  It is provided as a collection point so that cost objects can be grouped in ways for clarity like to separate out phases of the acquisition lifecycle or to divide costs between subcontractors, etc.  Whereas, the “System Folder” contains all ...
Original Post Date: Wednesday, September 25, 2013 We may all agree that risk analysis is a necessary, vital part of any valid/defensible cost estimate.  We may not agree as much on the best approach to take to quantify risk in an estimate.  All estimates contain risk.  In the words of a wise cost estimator I know, “That’s why they’re called estimates, and not exactimates!”  We must quantify and manage levels of risk.  Why?  One vital part of a successful program is the ability to build a budget based on reliable cost projections.  Reliability increases when we can analyze inherent risk, ...
Original Post Date: Wednesday, September 25, 2013 A lot of clients have been expressing interest in modeling ASICs, FPGAs, and various other electronic modules inside TruePlanning® (TP). In the release of TruePlanning® 2014 there will now be the capability to model all these products inside our framework. Not only will you be able to model these products but you will of course be able to model the integration cost of these electronic components with Hardware and Software components. In addition you would be able to add and estimate the program management of your total project through our integrated framework. TruePlanning Microcircuits ...
Original Post Date: Wednesday, September 25, 2013 “Integration Factors – What Makes TruePlanning™ Stand Out in the Crowd” In today’s world, system integration is becoming more and more important. The government has started asking for designs that have additional capabilities, which allow connectivity both with systems under construction and systems already in use and deployed. The reason systems integration is important is because it adds value to the system by adding abilities that are now possible because of new interactions between subsystems. In a recently posted article on “The True Costs of Integration” the writer defined the costs of a typical integration ...
Original Post Date: Wednesday, September 25, 2013 Introduction/Problem Statement A current client has expressed interest in the default values on the simple vs. the detailed input sheet. More specifically the question arose because this particular customer as well as others had a misconception about the simple vs. the detailed input sheet default values. Most users did not realize that if they were only inputting values on the simple input sheet that the detailed input sheet default values were still be used in the calculation for their cost estimate. So the question became how much are each of these default value inputs ...
Original Post Date: Wednesday, September 25, 2013 These days bidding can be a game, and contractor leadership is constantly making decisions on whether to take on risk in order to stay competitive or to bid conservatively for the safety of not overrunning.  You may complete a cost model for a program, and spend time analyzing the uncertainties behind each input and in the end find that your estimate lands at the 30% confidence level.  After some strategic analysis, the bid leadership team decides, we would like to bid at the 80% Confidence level, “please present your estimate to support that total”.  ...
Original Post Date: Wednesday, September 25, 2013 Background: FPGA design typically uses a library of Tiles (Sets of gates and transistors) and a CAD system to physically lay out the actual active devices on an empty portion of substrate. These devices are interconnected by physical (usually copper) traces to route the signals and perform the desired tasks. The design may be totally customized for a single set of functions and may not need any form of programming. Other designs may allow some parts of the device to be electronically re-programmed to allow the device to be calibrated or adjusted for ...
Original Post Date: Wednesday, September 25, 2013 During a recent Analysis of Alternatives (“AoA”) consulting project, our customer asked that we provide more insight into TruePlanning’s System and Assembly objects, which in our AoA context we termed Systems Engineering/ Program Management (SE/PM) and Alternative Integration/ Test, respectively. The customer’s challenge was understanding our parametric model’s treatment of principally hardware-COTS objects, combined with other cost, purchased service and training objects.Our Chief Scientist, Arlene Minkiewicz, provided us with insights that I’d like to share with you, as well as my views on how we at PRICE systems have consistently used these parent ...
Original Post Date: Wednesday, September 25, 2013 It is impossible to find a news website or magazine that is not full of articles on the effects of Sequestration.  As a cost estimator, I find the topic very interesting (and troublesome).  The immediate effects of Sequestration are widely discussed.  However, I do not see quite as much news coverage on the second and third order effects of this extremely complex policy decision. The Department of Defense (DoD) has a specific target that must be removed from the budget over the next 10 years.  Some analysts claim a doomsday scenario.  Others claim it ...
Original Post Date: Wednesday, September 25, 2013 In a recent National Public Radio (NPR) interview, Winslow Wheeler (Director of the Straus Military Reform Project of the Project on Government Oversight in Washington, D.C.), spoke on the recent problems with the Joint Strike Fighter acquisition process.  “Wheeler explained that Lockheed Martin, the manufacturer of the jet, uses a pricing vocabulary that masks costs. ‘Flyaway costs, non-recurring and recurring costs, and lots of gobbledygook, and they’ll say that comes to a number like $60-$70 million dollars. And, it’s complete baloney,’ said Wheeler.” (pogo.org)    The F-35 has the distinction of being the most ...
Original Post Date: Wednesday, September 25, 2013 What follows is PRICE's interpretation of how to model FPGA’s and ASICs inside our current version of TruePlanning® 2012 SR2 and older versions. ASICs are an application-specific integrated circuit, customized for a particular use. FPGAs are Field-programmable gate arrays and are considered the modern-day technology that can be used in many different applications because of the programmable logical blocks. In industry ASICs has grown from 5,000 gates to over 100 million. Designers of digital ASICs use a VHDL language for the functionality.   Estimating Best Practice for TruePlanning: Model your electronics at the board level ...
Original Post Date: Wednesday, September 25, 2013 Last November I hosted a webinar that discussed the use of the companion applications Live! This session helped to further explain how to use them in congruence with TP, the history, & why we created them, etc. During the presentation I showcased the success’s I have encountered with using them both in the recent AOA I described in part 2 and this blog part 3. You can find the recorded webinar on our site. In addition I described as I am going to do here the differences between the large project engine and the excel ...
Original Post Date: Tuesday, September 24, 201 Risk Analysis Methodology Overview – Method of Moments In this second of three articles on risk analysis, we will discuss Method of Moments.  Method of Moments (MoM) is an alternative to Monte Carlo simulation.  Along with the methodology, we will present some pros and cons of using MoM over Monte Carlo.  What is a moment? Before we discuss the methodology behind MoM, we first need to talk about moments.  Caution:  for all the master statisticians out there, this article is meant to boil down complex topics in an easy to understand manner.  There are obviously ...
Original Post Date: Wednesday, January 9, 2013 While TruePlanning by PRICE Systems is the world leading parametric cost estimation tool, the reality is that not all members in an organization are going to have TruePlanning installed on their desktops, but that doesn’t mean those people can’t get a view into the power of TruePlanning. The TruePlanning Viewer is a tool under development that allows users to view the PBS and data contained in a TruePlanning project without having TruePlanning on their desktops. Figure 1 TruePlanning Viewer TruePlanning users will be able to create special “tpview” files that non-TruePlanning users can use with ...
Original Post Date: Wednesday, October 17, 2012 A frequent question from students and consulting clients is how to estimate software size when either: detailed functional requirements descriptions are not yet documented or, even if the latter do exist, the resources necessary (in cost and time) for detailed function point (“FP”) counting are prohibitive. If appropriate analogies or detailed use cases are not available, fast function point counting can be a non-starter, without nominal understanding of pre-design software transactions and data functions.  Hence, the challenge is to find an estimating basis for functional measure (i.e., ...
Original Post Date: Thursday, October 11, 2012  Over the past year and half of customer mentoring, I have been responding to more and more requests regarding how to represent the DoD Acquisition Phases in the development of TruePlanning® cost estimates. With the renewed interest in Total Ownership Costs, there appears to be a desire to have greater visibility into costs by appropriation over a well-defined / understood schedule. This need to estimate and report out on cost by appropriation and schedule has been a driver behind the need to represent the acquisition phases more explicitly within TruePlanning® than is ...
Original Post date: Wednesday, October 10, 2012 Deciding whether Excel® is a friend or foe is a hefty topic, so I decided to dedicate several blog posts to the issue!  This first posts addresses all of PRICE’s new boundless Companion Applications.  The second will address my experience using the applications with our customers (Do’s and Don’ts); and lastly, the third and final blog will wrap it up and explain a more in-depth large project engine that PRICE is currently testing.  As we all know Microsoft Excel® is a powerhouse tool.  It allows you to house data, format it in many ...
Original Post Date: Monday, October 8, 2012 The answer:  ~ 10.226.  At least that’s the value on our complexity scale we found via calibration after modeling it in TruePlanning(R).   Check out this Teardown of the new iPhone 5 , which breaks it down into a bill of materials, each with an estimated cost.  A colleague had the cool idea to model the iPhone 5 with TruePlanning, using information we could find on the internet.  This was a really thought-provoking exercise to help me as I’m updating our electronics complexity guidance, because the electronics in the iPhone 5 are state-of-the-art. Around ...