Generating Tests by Example

The Problem

Developers write too few tests. Creating a comprehensive test suite requires thinking of many edge cases — empty inputs, null values, boundary conditions, concurrent access, unusual orderings — which is time-consuming and easy to miss. A single test that verifies the "happy path" may cover only a fraction of the code's actual behavior.

The result is fragile software with large gaps in test coverage. Bugs lurk in precisely the scenarios that nobody thought to test, and adding thorough tests after the fact demands the same deep understanding of the API that writing them from scratch does.

The Key Idea

Instead of asking the developer to enumerate every scenario, the system takes a single example test and uses it as a template for generating more. It analyzes the example to understand the tested API's interface and behavior, then systematically produces new tests by:

• Varying inputs — substituting different values, types, and sizes to explore how the API handles diverse data.
• Exploring different code paths — constructing inputs that exercise branches, loops, and error-handling logic not reached by the original test.
• Targeting edge cases — generating tests for boundary conditions such as empty collections, null references, maximum-size inputs, and duplicate elements.

All of this is guided by the structure of the example test: the sequence of API calls, the assertions made, and the relationships between setup, action, and verification.

Interactive Demo

Start with one example test, then click "Generate Tests" to watch the system produce new tests targeting different scenarios. Each generated test is derived from the structure of the original.

How It Works

Example Analysis

The system begins by parsing the example test to extract its structure: what objects are created, which methods are called, in what order, and what assertions are checked. This produces a test template — an abstract skeleton that captures the pattern of interaction with the API under test.

Input Variation

Using the template, the system generates new concrete inputs by varying the values supplied to the API. If the example test adds a single element to a list, the system tries adding zero elements, many elements, duplicate elements, and null values. Each variation is chosen to exercise a different aspect of the API's contract.

Path Exploration

Beyond simply changing values, the system analyzes the API's implementation to identify code paths not covered by the original test. It constructs inputs specifically designed to reach these uncovered branches — for instance, triggering a resize operation in a dynamic array, or hitting an error-handling path when capacity is exceeded.

Results

Starting from a single example test, the system automatically generates meaningful test suites that substantially increase code coverage. The generated tests are readable, well-structured, and target genuine edge cases that developers frequently miss.

The generated tests uncovered real bugs in well-tested libraries, confirming that even mature codebases have blind spots that systematic test generation can expose.