The CRASH (CAST Research on Application Software Health) report for 2104/2015 is out and an Executive Summary can be downloaded for free from this link.  This is the third biennial report produced by CAST based on an analysis of the data collected by their  AppMarq static code analysis tool to develop a report on the health of software projects based on their structural quality.  Structural quality speaks to the engineering goodness of the architecture and code for an application, rather than the functional quality that results by delivering software that solves users’ problems.  CAST determines structural quality of code based on an analysis of the following things:

  • Robustness – stability and resiliency and likelihood of code changes to introduce new defects
  • Performance – efficiency with respect to process time and resource usage
  • Security – application of good coding practices to prevent unauthorized intrusions
  • Changeability – how easy it would be to change or modify the application
  • Transferability – how easy would it be for a new team to understand an application well enough to be productive maintaining or altering it.

The 2014/2015 report has some pretty interesting findings.   And while many of these are certainly no-brainers to those familiar with good software engineering practices, they’re still worth thinking about.  The finding in the executive summary are based on the analysis of projects across many industries with a primary language of Java-EE (because this was the largest language data set in the repository).  A summary of the findings follows:

  • From a structural quality perspective – very little difference was observed for applications that were developed in house or outsourced, nor was there much noticeable difference between those developed on-shore vs. off-shore.
  • The number of end users was found to be a significant driver in structural quality with applications serving 5000 or more users being noticeably healthier than those serving fewer than 5000. 
  • This research also determined, not surprisingly, that the investment of going from a CMMI level of 1 to CMMMI level 2 or 3 definitely pays off with respect to structural quality.
  • Probably the most interesting finding had to do with the influence that development method had on structural quality.  While there was little significance in the difference on projects developed using agile methodologies vs. waterfall, it was clear from the results that organizations that combined agile and waterfall were building better software – reinforcing the notion that there are some parts of the software development life cycle that benefit from more creative and relaxed practices.

Mostly this report confirms what we already know about good software engineering practices and how their use leads us to write better software.  You should check it out – it’s definitely worth the read.