Recently I have been playing around with the International Software Benchmark Standards (ISBSG) database for Development and Enhancement projects. And in the interest of full disclosure I admit that I am more than a little excited to have close to 6000 data points at my fingertips. I will further admit that there’s something quite daunting about having this much data; where to start, what should I be looking for, how can I best use this data to offer some useful guidance to inform software cost estimation. For those of you not familiar with this data source, the ISBSG database contains data on software projects submitted from software development organizations all around the world.
Naturally, the first thing I wanted to do was look at Project Delivery Rates (PDR) and try to find some interesting information about what might drive PDR. Starting small (ish) I filtered the data to eliminate all those data items which the ISBSG has rated as of low or questionable quality and filtered all those projects which had included in their labor totals hours for resources not strictly part of the development team. I started by trying to trend Functional Size with PDR. The data was all over the map. In an effort to get some context as to where the good data might be lurking, I began to look at average PDR rates for each Organization Type. I selected Organization Type because this seemed to be the most granular category that had some structure to it. Organization Type is intended to indicate the type of organization that the software application is intended for. Although an Application Type is provided, this value is free form with each submitter choosing their own terminology. The same is actually true for Organization Type but because of the submission process there was a finite set of responses which could be used as a basis of stratification. Although the ISBSG data is measured using functional size measures, the list of ISBSG acceptable Functional Size Measures is long and includes IFPUG, NESMA, COSMIC, Mark II, FiSMA, etc. In order to compare apples to apples my analysis needed to focus on each size unit individually. I started with IFPUG because this group contains significantly more data points than any other functional measurement category. I thought I would share some initial findings. The following table shows the productivity rates for various Organization Types.
So what does this table tell you? It certainly needs to be interpreted with care as you will note that for many of these productivity rates the distribution is all over the map. For industries where the sample size is significant it gives some pretty interesting comparative information. It also provides some useful information into the types of industries for which you can find data in the ISBSG database (not inclusive because this did not cover all the data points – only the IFPUG ones) There is also the caveat that for different submitters have different ideas about definitions of things like industry type. Despite that I think there’s something to learn here. If you’re interested in data, and who isn’t you should check out the ISBSG’s offerings – they have a pretty cool arrangement using an OLAP interface to allow you to find and pay for only the data you can use. Check it out at www.isbsg.org !!
And I’m just getting started – check back for more observations on software productivity!