What is it about?

A new testing tool, called TerzoN, extends the NaNofuzz fuzzer and its human-focused user interface to integrate fuzz, example-based, and property-based testing into a new "Composite Oracle." TerzoN's unique Composite Oracle allows users to articulate correctness in various ways in a single user interface that can call attention to certain conflicts among a user's assertions that might otherwise be easily overlooked. In a randomized controlled human trial of professional software engineers using TerzoN and the popular "fast-check" testing tool, participants using TerzoN elicited 72% more bugs, accurately described more than twice the number of bugs, and finished testing 16% more quickly (p < 0.05) than participants using fast-check.

Featured Image

Why is it important?

Thinking up and articulating properties (or invariants) of programs is hard for software engineers. This difficulty has limited the adoption of powerful property-based testing tools to a narrow group of specially-trained software engineers. The net take-away is that a Composite Oracle, when surfaced appropriately in a human-focused user interface along with lots of examples, can help software engineers articulate expected behavior more accurately and, in doing so, also find more bugs in less time than with a widely-deployed state-of-the-art industry tool.

Perspectives

Property-based testing tools have a lot of usability problems that have not been adequately explored since Quickcheck was created more than 25 years ago. Many design decisions from QuickCheck have been copied from one property-based testing tool to the next without much thought for how human software engineers actually interact with these tools. We hope that TerzoN can be a step toward a more usable property-based testing future that helps a wider range of software engineers build and ship better code.

Matthew C. Davis
Carnegie Mellon University

Read the Original

This page is a summary of: TerzoN: Human-in-the-Loop Software Testing with a Composite Oracle, Proceedings of the ACM on Software Engineering, June 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3729359.
You can read the full text:

Read

Contributors

The following have contributed to this page