Here’s a great article written by Charles Symon’s discussing current challenges in software project estimation and how different organizational cultures deal with them differently.  According to the author, in a study of 105 software projects completed between 1997 and 2007 in the UK public sector – 30% were terminated, 57% experienced cost overruns averaging 30.5% and 33% suffered major delays. (“Cost over-runs, delays and terminations: 105 outsourced public escort ICT projects – D. Whitfield).  And this is only one study; any of us who live in the software world have heard more than our share of tales of woe related to software projects gone bad.

According the Symons, the reasons for these failures, going back at least 30 years generally fall into two categories:

  • Lack of commitment on the part of senior management and lack of involvement of the end users
  • Inadequate project management, team with high turnover and/or inexperienced staff

The author splits the software world into two broad categorizations:

  • Chaotic: these are the ones for whom the above types of statistics generally apply.  Organizations in the chaotic world are characterized by software projects with frequent overruns of cost, schedule or both to whom completely failed projects are no strangers
  • Controlled: Organizations in the Controlled world are characterized by repeated software projects successes, regularly delivering projects that delight users within time and cost constraints.

The focus of the paper is how the chaotic organizations can move into more controlled environments.  All readers involved in delivering software projects already knows the answer… teach the chaotic organizations to follow the data.  We all also know that is often easier said than done.

The author presents an example of Renault, an organization firmly in the controlled world (for more on Renault follow this link…  The example revolves around their purchase of Electronic Control Units for the vehicles they produce.  Not surprisingly, the average family car has approximately 50 of these units that control all of the electronics in the vehicle.  Even less surprising, an important part of these units is the software that does much of the heavy lifting.  A simplification of the process that Renault uses to successfully negotiate with the suppliers who develop and deliver the software for these units follows:

  • An automated COSMIC function point counter is used to determine the functional size of software required based on the requirements specification for each individual unit.
  • Historical data which has been collected and statistically analyzed is used as a basis for the prediction of effort associated with delivering that functionality.

Clearly it’s a bit more complicated than that but the bottom line is that the first step on the path from Chaos to Control is to start collecting data.

To be fair, the author does acknowledge that an organization such as Renault is able to operate in the Controlled World because they are estimating projects where specifications are complete and requirements are well hashed out.  Organizations in the Chaotic world tend to meet demise due to estimates that are performed very early on in the process, before requirements are firm when there is much uncertainty.  He is also pretty quick to point out that maybe an estimate performed at this point in the project shouldn’t be the one to hang one’s hat on for all perpetuity.

Having said this, data collection can be an important first step on the path to control and in fact may facilitate a fairly impressive journey down that path.  Symon’s cites a process that was developed 15 years ago by the Government of the State of Victoria in Australia.  Though this process has seen widespread success locally (proponents claim cost overruns can be reduced to within 10%), it’s adoption has not been impressive.  In a nut shell the process works as follows:

  • Given the requirements for a software project, suppliers estimate the functional size and bid a fixed price per functional size unit
  • The contract award is for the product of functional size * price per functional size unit
  • Actual size is monitored by an independent ‘Scope Manager’ and the total price varies as the functional size varies

In other words the customer assumes the risk of adding or changing requirements while the supplier assumes the risk of bidding the ‘right’ unit price.

Bottom line – if you feel your organization’s practices are closer to chaotic than controlled – check out this article and see how well reasoned data collection can start you down the path to successful project estimation.