Automatically deduplicating program crashes by test case simplification and root-cause clustering

What is it about?

Fuzzing has become a key driver to discover bugs in complex programs. A key advantage of fuzzing is that each discovered program crash comes with a witness, the program input that allows reproducing the crash. Due to its probabilistic nature, fuzzing may find many different inputs that trigger the same bug, resulting in additional work for the developer to triage, analyze, and prioritize the different issues. Our paper presents a novel approach---Igor---that automatically deduplicates these crash inducing witnesses by first minimizing them (through coverage-reducing fuzzing---shrinking test cases to the approximately shortest path to the bug) and then clustering these paths to group them by similarity. In Terry Pratchett's Ankh-Morpork, the "Igors" are a group of humble professional servants (often to mad scientists) that are proficient transplant surgeons, a fitting name for a tool that transplants discovered crash inputs to make them more useful to the analyst. Our approach manages to reduce 254,000 unique crashes into 48 unique clusters that ultimately map to 39 ground truth bugs, reducing the amount of developer work by four orders of magnitude.

Featured Image

Photo by Peter Neumann on Unsplash

Photo by Peter Neumann on Unsplash

Why is it important?

Automatic testing discovers large amounts of bugs but several inputs may be discovered for each bug. Developers spend large amounts of time to triage and analyze the many generated test cases, wasting expensive developer time. Given that only finite developer time is available, this will reduce the number of bugs that can be fixed and increase the time until a fix is available. Our approach allows the developer to focus on actually fixing the bugs instead of triaging them.

Perspectives