What Motivates People to Buy Static Analysis

We’ve visited a lot of companies who are using static analysis to improve their software development. What drives companies to use static analysis varies considerably from company to company and from industry to industry. We attempt to categorize the most common drivers for using static analysis into various buckets. We’d love to hear about other drivers as well so please let us know if you have others to contribute.

There are a lot of factors to consider when trying to understand what drives a company to use static analysis. You have to consider not only what are the business drivers and benefits but also who are the people who are driving and influencing the decision.

We have bucketed them accordingly
• The opportunists
• The code reviewers
• The engineering management
• The quality portfolio manager
• The compliers
• The automators

No categorization is discrete. There’s quite a bit of overlap and different roles can exist within the same company. Benefits can be felt across a wide range of people.

The Opportunists
The Opportunists use static analysis tactically, to capture the low hanging fruit in the codebase. They recognize that one of the biggest benefits of static analysis is finding and fixing bugs are much cheaper than most any other means. But, Opportunists only want to invest as little time in the tool as possible. They try to find and fix as many of the easy problems as they can find and focus only on the areas that they feel are most important to them for example like focusing on crashes or memory leaks.

The downside of this approach is that incrementally, Opportunists get a really high ROI because their "Investment" is low and their "Return" is good. However, they must take into consideration the license and support costs plus any other costs associated with the tool such as training, set up and configuration. With all parts of the equation adequately modeled, the ROI can suddenly become much less attractive. In order to keep the costs low, many Opportunists just run the tool out of the box with the default settings, not even taking the time to tune the analysis to find more of the low hanging fruit that they want to fix. Clearly there is more benefit opportunity for Opportunists, to better increase the marginal utility from the marginal cost they put into it.

The opportunist is usually a single developer or a few developers who are highly proactive in improving the quality and security of their code. They are also usually extremely busy with little time to devote to static analysis.

The Code Reviewers
Code Reviewers can live within the software development organization (such as a code review team) or outside, such as in the case of a security review team. Manual code review is a tedious, error-prone and non-scalable process. While static analysis doesn’t replace manual code review, it helps make the manual code review process much more focused on higher value activities. At the same time, static analysis helps reviewers gain significantly more coverage, reduce human error and provide more consistency in the review process.

Code reviewers like to have a well-tuned analysis, specific to the codebase with all the right settings to find the class of problems that are most relevant to them, for example, security reviewers like sources and sinks defined as part of their data flow analyses (without it, the data flow analysis doesn't have much to work with). The analysis can also be tuned to find a wider range of problems at the cost of a little less accuracy. Code Reviewers can also write analysis rules to find specific classes of problems or enforce coding rules. When put into a regression test, these rules can make sure the problem or coding violation doesn’t arise again either in the original place it was found in or in any other place in the codebase or even in other codebases that are being analyzed.

The main challenge in these environments is the natural friction caused by reviewers versus developers. When handled improperly, developers can become defensive and push back on static analysis reports as “not bugs.” Code Reviewers need to be sensitive in how they assign and discuss potential problems. Developers take a lot of pride in their work but should also be open minded of efficient ways to improve the quality of their output. When working well, this model has great checks and balances in place to ensure proper usage of the tool.

The Engineering Management
We’ve all heard the analogy of how software development should borrow more from the manufacturing industry. Many software development managers recognize that static analysis actually saves overall development time and can improve cycles times. Finding defects earlier in the process can most definitely save on overall time and costs.

Management’s big challenge is to ensure broad, consistent usage so that the benefits can be realized. Not every developer recognizes the benefits amidst the negatives. Static analysis is not perfect and requires some learning, to understand defect reports, process changes and most importantly dealing with false positives and “don’t care” reports which, if not handled properly, can decimate the morale of the developers.

The biggest challenges to these environments is lowering the cost of usage through deep integration with the current software development process and automation to minimize any manual work. In addition, tuning of the analysis to lower the rate of false positives and “don’t care” reports will lower the cost significantly and provide a big psychological boost. Using outsourced teams to review the defects first help maximize your most precious resource – your developer’s time.

Administrating the tool requires effort and most tools engineers are already too busy to not only learn the tool but manage it ongoing. Contractors and consultants can help either to jump start the process or to take ownership of the management and administration ongoing.

The Quality Portfolio Manager
There’s no silver bullet to quality and security. In order to get acceptable quality and security, you have to take a multi-pronged approach. Unit tests, code review and now, static analysis are parts of the overall strategy to lower the risk of field defects.

Holistic approaches to quality are often driven by a forward thinking engineering or QA manager, or as a response to a serious bug or serious class of bugs that has now driven a top-down quality initiative. Other companies may realize that bugs are causing a high recall rate or high support costs which is driving down profitability. This realization create the necessary organizational capital to move static analysis forward.

The challenges that arise from these types of initiatives stem higher than those faced by the Engineering Management example. Specific ownership and clear goals are too often not considered when instituting a static analysis solution from the top down. When static analysis becomes just a checkbox among a range of other strategies, it often doesn't get the necessary attention to make it succeed. Ensure that the ownership of the tool has teeth, whether it be owned by someone internally or externally.

The Complier
In some industries, you just don’t get much choice. If you are developing software that will be implanted into someone, or go up into space, or guide a missile, then you have to be concerned about regulations and industry guidelines such as FDA, DO-178B and MISRA. In all of these industries the government either requires or recommends the use of static code analysis to improve the quality and security of the delivered product.

The challenges faced by the Complier are similar to the Quality Portfolio Manager. The further away the driver for a solution is from the implementer and maintainer of the solution, the more that will be lost in translation and commitment. Developing the expertise in-house is not insignificant. Creating ownership with real incentives and consequences is critical.

The Automator
Automation is the fashion, particularly with the movement towards Agile methodology where cycle times are much more rapid. Continuous integration builds are becoming more common and creates better software earlier in the process. Automators recognize that static analysis is perfect for this. Static analysis doesn’t require test cases and can be automated as part of the build process (as most modern static analysis is integrated with the build process). Results can be churned out in minutes or hours. Developers can even analyze code as they are developing it allowing organizations to institute rules that code must be free from any static analysis errors before check-in.

The largest challenges here is that automation takes some work. It’s not hard but it’s also not necessarily turnkey. Being able to have the right deployment model so that the analysis is fast for each developer can be tricky but is most certainly possible. Automation usually is two pronged – where developers can run a fast analysis locally on their changed code while a central analysis handles a full integration analysis. Automation is well worth the effort even in the short term.

Many Reasons, Many Options
Clearly there are many drivers for static analysis. In conert, the optimum deployment model varies considerably. For example, tuning an analysis to find everything under the sun may not be a good model for the Opportunists. Likewise, a full developer deployment model may not be a good choice for the Reviewer's. Matching the deployment model and the analysis tuning to the business need will increase by orders of magnitude the ROI you get from a static tool. Companies have woken up to the fact that developing this expertise in-house is an investment that is appealing to some and unappealing to others. Some prefer a jump start mentoring program to help them get pointed in the right direction and others prefer to outsource to get best practices static analysis expertise. Regardless, static analysis can provide a multitude of benefits for most software development organizations. By recognizing the drivers, you can create a great custom solution that makes almost everybody who involved get what they need out of it.