Reducing false alarms in static buffer overflow analysis

Abstract

Buffer overflow detection using static analysis can provide a powerful tool for programmers to find difficult bugs in C programs. Although more precise abstraction can reduce the number of false alarms in general, the cost to perform such analysis is often too high to be practical for large software. On the other hand, less precise abstraction is likely to be scalable in exchange for the increased false alarms. In order to attain both precision and scalability, we present a method that first applies less precise abstraction to find buffer overflow alarms fast, and selectively applies a more precise analysis only to the limited areas of code around the potential false alarms. We present two effective methods to reduce false alarms in our buffer overflow analyzer. One is state refinement to remove redundant alarms in the fast buffer overflow analysis phase. When the origin of a group of alarms are same, our method shows only the first alarm in the group and automatically filters out the rest. Our experiment with several open source programs shows that our method can reduce about 27% of buffer overflow alarms on average. It suggests that the efforts to examine or fix the problem are reduced by the same degree. The other is symbolic execution over the potential alarms found in the first phase to filter out false alarms. Taking advantage of a state-of-art SMT solver, our precise analysis efficiently filters out a substantial number of false alarms. Our experiment with the test cases from three open source programs shows that our filtering method can reduce about 68% of false alarms on average.