False Positives versus Intentional's

Bugs need to be categorized so that the appropriate action can be taken.  The traditional bug reported by customers, support, field engineers, etc. are examined by developers to determine if it is a bug.  If it is a bug, its priority and severity are usually assessed.  If it is not a valid bug report it may have been rejected as not a bug, unfixable, already fixed, duplicate or any of a number of different categories that say that no further work will be done on it.

Static analysis defects have a slightly different flavor.  Static analysis reports potential defects, meaning an inconsistency or flaw has been detected that needs to be verified by a developer or review team and handled appropriately.  Bugs determined to be real ("true positives") need to be assessed for priority - usually a combination of type of bug reported, whether it is on a specific conditional path/exception path, what part of the code it is found in (new code/old code/component/3rd party library), etc.  Static analysis solutions may also include a "score" to give further information to help prioritize it.

Some static analysis results however may be "false positives" which are essentially "not a bug" characterizations.  Herein lies a distinction.  Vendors often like to make a distinction between a "false postive" and an "intentional" status.  A "false positive" signals a situation where the analysis incorrectly identified a problem as a bug.  If you will, it's a bug or limitation of the static analysis tool.  An "intentional" status signals a situation where a valid bug is reported but that the developer had intentionally coded it this way and so no further action is required.  The presumption here is that for "intentional's", the analysis did what it was supposed to do correctly.

Some organizations lump everything that they don't want to fix into the "false positive" category.  If they weren't going to change any code, then it shouldn't have been reported.  Other organizations take a more sophisticated approach by making this distinction - the presumption being that intentional's should be reviewed, like a sanity check.  A variety of reasons why they categorize it as such include:

• wanting to get an accurate "false positive rate" for the tool.  Lumping "intentional's" artificially increases the false positive rate
• the workflow for "intentional" and "false positive" may be different.  While both should be reviewed/audited, a static analysis administrator may review "false positives" for tuning and configuration opportunities.  False positives should be reported back to the vendor for fixing as well.  "Intentional's" may be reviewed by architects to understand what strange coding practices are taking place.  Intentional's typically need a good reason to explain why it was coded in the way it was.

Of course the advantage to finer grain categorizations have to be weighed with the cost of explaining the distinctions.  There is a cost to data acquisition, but if you value the ability to measure the tool's effectiveness and have a process for improving your analysis and improving your code through examination of intentional's then it may be well worth it.