Technical Debt And Static Analysis

CAST Software, a maker of software analysis and measurement solutions published some interesting findings on the quality and security of software applications across a number of different industries and applications.  (75 organizations, 288 custom applications).

While clearly, vendor funded need to be taken with a grain of salt (for instance the calculation was based on their tool's defect density and a fix rate presumably from their experience), the approach and findings do cast an interesting light on the quantification of quality initiatives.  To start with, they define "technical debt" as "cost of the effort required to fix problems that remain in the code when an application is released to operation.  Like financial debt, Technical Debt incurs interest in the form of the extra effort it takes to maintain and enhance an application due to the structural quality flaws left in that code."

In summary, some of the findings are:

• The average technical debt was found to be about $1M with the average code size being 374,000 lines of code. 
• The average maintenance cost per line of code was $2.82 assuming a $75/hour average, fully loaded wage for a developer and that a bug takes about an hour to fix.
• C/C++ applications had wider variability but on average had lower technical debt that some of the Web technologies such as Java and .NET.

My assumption though is that this number was calculated from the defects that came from the Cast tool that would presume to be fixed and therefore is a subset of the actual technical debt for an application (presumably there are many other bugs that would need to be fixed, not found by a static tool).  If true, I would conclude that they are saying that if you use the Cast tool to the level they suggest, that it would save $1M in technical debt, an amount that over time would grow larger due to interest.

One could argue either way whether the number is bigger or smaller.  On the one hand, there is a cost to fixing, starting from the license cost all the way to the extra hardware, training and usage cost.  On the other hand, the calculation of technical debt was only done based on developer time and not on the effect of a defect, which can be many multiple times more costly than simply the time it takes to fix a bug.  We sometimes call these the "million dollar bug" and it varies significantly among industries and application types.  While this cost may not technically be a part of the technical debt calculation, you may wish to figure the cost of a defect in deciding whether to use static analysis or not.  Sure, these defects are few and far between but if you can find just one...

Either way, static analysis does appear to save money and help developers produce better quality software - the hard part has been in measuring it.  Technical debt is one interesting way to measure it.