Fuzzing – A Powerful Technique for Software Security Testing
I was participating in a code review today and was reminded by a senior architect, who started working as an intern for me years ago, of a testing technique I had used with one of his first programs. He had been assigned to create a basic web application that collected some data from a user and wrote it to a database. He came into my office, announced it was done and proudly showed it to me. I walked over to the keyboard, entered a bunch of junk and got a segmentation fault in response.
Although I didn’t have a name for it, that was a standard technique I used when evaluating applications. After all, the tried and true paths, expected inputs and easy errors will be tested early and often as the developer exercises the application using the basic use cases. As Boris Beizer said, “The high-probability paths are always tested if only to demonstrate that the system works properly.” (Beizer, Boris. Software Testing Techniques. Boston, MA: Thomson Computer Press, 1990: 76.)
It is unexpected input that is useful when looking to find untested paths through the code. If someone shows me an application for evaluation the last thing I need to worry about is using it in an expected fashion, everyone else will do that. In fact, I default to entering data outside the specification when looking at a new application. I don’t know that my team always appreciates the approach. They’d probably like to see the application work at least once while I’m in the room.
These days there is a formal name for testing of this type, fuzzing. A few years ago I preferred calling it “gorilla testing” since I liked the mental picture of beating on the application. (Remember the American Tourister luggage ad in the 1970s?) But alas, it appears that fuzzing has become the accepted term.
Fuzzing involves passing input that breaks the expected input “rules”. Those rules could come from some formal requirements, such as a RFC, or informal requirements, such as the set of parameters accepted by an application. Fuzzing tools can use formal standards, extracted patterns and even randomly generated inputs to test an application’s resilience against unexpected or illegal input.
Fuzzing falls into the security testing domain. Although when I carried out that test years ago I was actually not looking for a security issue, I was looking for reliability flaws. I don’t like programs to fail in the field. What is interesting is that unreliability for a user often signals opportunity for an attacker. In that regard fuzzing is a great technique since identification of vulnerabilities likely identifies a bug that could irritate a user.
Developers and testers should use fuzzing in the process of validating software. It doesn’t replace other testing, such as unit, integration, performance and so on. Instead it seeks to find unknown flaws whose presence won’t be identified by using existing test cases. Essentially, fuzzing identifies vulnerabilities that would become zero day exploits.
A key reason that fuzzing should be a required part of the testing process is that attackers use fuzzing to find vulnerabilities. Attackers are smart and know that an easy way to find flawed applications is to use fuzzing techniques to automatically poke at program interfaces. The fuzzing tools will identify flaws in responses and the attacker can then focus on trying to find a workable exploit.
There are several tools, commercial and open source, that support fuzzing. In Blue Slate’s security training course we introduce students to an open source tool, JBroFuzz which is sponsored by the OWASP. The tool is great for learning about fuzzing and allows for fuzzing of applications that use HTTP and HTTPS.
Commercial tools in the space include Codenomicon’s Defensics and Security Innovation’s Holodeck. Both are very powerful tools that simplify creation of test scenarios and reporting of identified errors.
I strongly urge any software shop to learn about and apply this technique. Attackers will “test” our applications with this approach in the wild. Doesn’t due diligence require us to actively pursue the identification and mitigation of flaws in our software before attackers find and exploit them? Being proactive when it comes to security shouldn’t be optional and given the availability of these tools we have no excuse not to utilize them.
Fuzzing is a powerful technique, along with peer reviews, code analysis tools, log monitoring and so forth to minimize the number of vulnerabilities in our applications.
Does your shop use fuzzing tools as part of the development and testing process? What is your experience with their effectiveness? Do you have best practices around defining templates for the fuzzing tool to use? Feel free to share your thoughts and best practices regarding these types of tools.
Tags: application security, enterprise applications, Information Systems, Internet, linkedin, programming, Security, Testing, vulnerability