The 2017 Crash Report is now available.   This is a report of CAST’s Research on application software health.  The results are based on a study of the structural quality of 1850 applications totaling more than one billion lines of code from around the world based on five health factors that measure:

  • Robustness – measuring the likelihood of outages and time to repair based on poor implementation practices
  • Security – measuring violations of secure coding practices which can lead to security breaches and data theft
  • Performance Efficiency – measuring potential performance  issues to inefficient use of resources such as memory or processor time
  • Changeability  - measuring potential difficulties in modifying applications, fixing bugs or adding new features
  • Transferability – Measuring the potential difficulty of understanding the application as new teams or team members assume responsibility

The score for each factor ranges from 1 to 5 with 1 being a really bad score and 5 indicating exceptional structural quality.  If you’re interested in exactly how they do this – check out the report – it goes into a great deal of detail on the demographics of the applications studied followed by a thorough analysis of how various factors affect application health and structural quality.

The report has some interesting findings.  Not surprisingly - an organizations Capability Maturity level (as determined by the CMMI) is an indicator of the health factors of the applications it develops.  It’s also true that for the applications studied development method is a significant indicator.  The study found that organizations that use a hybrid process - starting with the up-front design and architecture activities of a waterfall process followed by short iterations of capability delivery and customer feedback of an agile process - develop code with higher health scores.

While capability maturity and development methods had the most significant results, the report also suggests that factors such as development team size, industry segment and number of end users also see to be good indicators of application health.  Application size does not appear to be a relevant factor.

The report also focused on the technologies used in the applications with the caveat that because there are different sets of rules for evaluating different technologies so it’s not a completely fair comparison. They found significant variations between technologies for the health factors of Robustness, Performance and Changeability.   The languages they included in this study were Java-EE, Cobol, .Net, ABAP, Oracle Server, C and C++ which comprised of 86% of the applications they studied.  Much of the variance seen for languages had to do with the nature of the languages - C and C++ provide better access to the machine than Cobol making it easier to improve performance and robustness – and the age of the language – more modern languages provide and encourage more modular and object oriented code – impacting the scores for changeability.

All in all, the report supplies a great deal of information and food for thought.  Poor structural quality is often illusive as many violations of the rules for good health are not immediately recognized as bugs – since they aren’t making the app crash and burn.  But it’s a good idea to be aware of potential issues and understand some things that impact structural quality in your code.