The Value of Static Analysis
On the Daily Dave mailing list there’s an interesting discussion about the value of static analysis. For those unaware of what static analysis is, static analysis is analyzing source code to find potential vulnerabilities. Like every technology, static analysis has it’s pros and cons. I don’t actually subscribe to the mail list (I only use RSS), so I’m going to write a little about my views on static analysis.
Pros
In the security world, the big fight is between static versus dynamic analysis. By dynamic, most people talk about penetration testing. The results from automated penetration tools usually contain low amounts of false positives, but your test is dependant on the tests ran by the tool. Pen-testing tools can and do miss vulnerabilities. Static analysis on the other scans ALL code. If there are vulnerabilities in the corners of your application that are typically not used, then static analysis has a higher probability of finding these vulnerabilities. In addition to finding obscure vulnerabilities, static analysis can also find more categories of vulnerabilities. Automated pen-test tools are limited because they can only see http responses. Static analysis tools can apply rules that are more focused on your development platforms.
Cons
The biggest argument against static analysis is it produces too many false positives. The common misconception here is that the tool is not saying “this is vulnerable”, the tool is actually saying “this is potentially vulnerable and needs to be audited to be sure”. Yes this creates a lot of work, but this argument really only applies to first time scans. Most of the major static analysis applications are rule based and give better results over time. After the initial triage, you suppress false positives and create custom rules to make the scan more context specific. For example, on the mailing list someone referred to static analysis tools producing false positives on custom memory management libraries. This is true, out of the box most scanners are going to flag this because they don’t know what the library does and want human eyes to verify. If you’re using Fortify SCA, you can write a custom rule to eliminate those false positives in the future. Because I’m a Fortify consultant, I know that the more you tailor our static analysis software to your application the better your results set. Static analysis shouldn’t be a one shot scan, it should be used continually throughout development and testing.
Conclusion
In the end, it’s not static analysis versus dynamic analysis. In reality, you should be using BOTH. Static analysis is going to give you a sense of how secure you code is. Penetration testing is going to find easily exploitable vulnerabilities. If you are concerned about false positives with static analysis, check out Fortify Program Trace Analyzer (PTA). PTA does static analysis automatically as you are doing functional testing. The results are extremely conservative. If PTA finds a vulnerability, you can usually take it to the bank.