Thursday, April 22, 2010

Lowering Developer Resistance to Static Analysis Tools

Imagine a team of software engineers who embraces the software development process.  They  dutifully create their unit tests before writing a line of code, run and fix all necessary static analysis and dynamic tests prior to check-in, fix all compiler warnings, participate actively in code review. They write code that is brilliant, yet easy to understand and maintain. They write unbrittle code that can withstand the inevitable changes to the assumptions that surround the code. Everything of course is well documented.

They realize that their code is bigger than themselves -- that it has to work as part of a larger ecosystem of code created by engineers from around the world. They realize also that their code may last for decades and may change ownership several times. They realize that taking proactive efforts to make good code mean less reaction down the road.  They welcome new process that will help them write better code overall even if it means a little more work upfront.

Most developers won't disagree with these ideals, however, when instituting process that is "good for us to do", they bristle at, argue with and/or ignore the change. It's not their fault. Developers are often set up to fail at following the process right from the beginning.

With every new change comes an investment in time and energy to make it happen. If you were asked to add a new step in your development process, you have to make an investment (sometimes sizable) at the beginning to understanding and instituting the change and subsequent investments every time you perform that new added step.

We all know that developers are under immense pressure to deliver more for less. They simply aren't given enough time to make the upfront and ongoing investments. Routinely asking engineers to do feature X in 2 days instead of 3 days will result in cries of "foul" every time something might get in the way of their already oversubscribed status quo. Clearly the managers and project managers must understand and manage properly the time pressures.

In addition, the upfront and ongoing investments in time and energy can be significantly reduced.  When instituting static analysis tools at various companies, we've found the following to be helpful:
  • Training can be a huge time saver.  But not all people learn the same way.  Some like it fed to them and some like to go and get it.  We've found that taking a multi-pronged approach, by offering training, group and individual mentoring, a centralized resource with good, simple step-by-step instructions, a helpdesk, a forum/wiki and more get just about everyone onboard.
  • Package it well.  Developers can figure out just about anything technical.  But, it doesn't mean they have to.  Having to fiddle with getting a tool installed may provide some individual technical satisfaction but doesn't mean they've done anything useful towards meeting a business goal.  Multiply that by how many developers you have and you've created a large time sink.  You'll also lose support by developers in this way by making it hard to use.
  • Keep the standard workflow simple.  We worked with one organization which gave every developer the standard training and then washed their hands of it.  After having thrown it over to the developers, it was no wonder that a year later hardly anyone was using it.  One of the biggest problems was the tool was given to the developer with no workflow and no purpose.  No queries were set up.  No screens were customized. The developers had to suddenly become experts at using the tool -- a foolish use of time given tool usage was certainly not a core competence of the organization.  Spoon feeding a simple, easy-to-follow process doesn't mean you think your developers are dumb.  It means that they get to spend less time wrestling with the tool and more time doing what they are paid to do - write great code that solves the company's mission.  Again, without the right process, you'll lose support.
  • Automation is key.  Having multiple steps in the process across multiple tools in the toolchain mean added time, human error and frustration.  Integrating static analysis with bug tracking, source control management, LDAP, policy databases, build scripts, etc. lowers the ongoing investment.
  • Another area ripe for improvement is performance.  We've run into codebases where the analysis takes 30 hours or more.  Static analysis has to analyze a lot of stuff but there are ways to improve the performance dramatically.  We've tuned some analyses down to minutes depending upon the deployment model.  This significantly lowers the "cost" and opens up new possibilities for when and where analysis can be performed
  • One of the biggest complaints you'll hear from developers is that the results have false positives - meaning results that are incorrectly noted as issues.  Every static analysis tool will generate false positives but the more the tool understands your code (usually through configuration), the more it will produce better results.  Every time a developer has to examine an issue that is eventually is false is wasted time (although some will argue that not all of the time is wasted because false positives can signal poorly written code).  Lower this cost and raise the value of the tool through tuning.  
  • If you aren't a developer get them involved.  There's nothing like having a champion or two help you get buy-in.  Involve them in defining the process - after all, nobody knows the process better than the developers themselves.  Developers have to see and believe in the true value, not the inflated value.  Do this by running mentoring sessions where everyone can plainly see the value and the limitations of the tool.  Set expectations all around accordingly.
In concept, static analysis tools are simple to use.  In addition, your developers are smart, so it's tempting to just give them the tools and let them figure out the value.  It's a given your developers have too many features to deliver in too short a time.  You need to lower the cost of adding static analysis to their workflow.  You'll lower resistance and improve their efficiency while improving the overall quality and security of the code.

No comments:

Post a Comment