Automatic fuzzing grammar generation through API-level symbolic execution

Abstract

Blackbox fuzz testing is commonly used to find security bugs of a program whose source code is unavailable. Blackbox fuzz testing can only test a small portion of code when rigorously checking the well-formedness of input values. To overcome this problem, blackbox fuzz testing is sometimes performed using a grammar that delineates the format information of input values. However, it is almost impossible to manually construct a grammar if the input specifications are not known. We propose an alternative technique: the automatic generation of fuzzing grammars using API-level dynamic symbolic execution. API-level dynamic symbolic execution collects constraints at the library function level rather than the instruction level. This idea is based on the fact that developers generally prefer to use well-known, string-related library functions over self-implemented code when processing input strings. While API-level dynamic symbolic execution may be somewhat less accurate than instruction-level dynamic symbolic execution, it is highly useful for speedily generating fuzzing grammars that enhance code coverage for real-world programs. Fuzzing grammars explicitly differentiate fields that affect paths from those that do not. Therefore, by replacing fields that do not affect paths with random strings, lengthy strings, file paths, urls, etc, fuzzing grammars can be used to generate concrete test cases that can easily trigger security bugs such as buffer overflow vulnerabilities, etc. To verify the feasibility of the proposed concept, we implemented a system for generating ActiveX control fuzzing grammars, named YMIR. To the best of our knowledge, the YMIR system is the first tool ever developed to carry out whitebox fuzz testing on ActiveX controls. The experiment results showed that the YMIR system was capable of generating fuzzing grammars that can raise branch coverage for ActiveX control methods using highly-structured input string by 15-50\%. In addition, the YMIR system found three vulnerabilities revealed only when input values are well-formed. Automatic fuzzing grammar generation through API-level dynamic symbolic execution is not restricted to the testing of ActiveX controls

it should also be applicable to other string processing programs whose source code is unavailable.