Don't Panic: Property-Based Testing

Property-based testing is the habit of testing a rule instead of assembling a long guest list of preferred inputs. A named example says that sorting this particular list should return that particular order. A property says that sorting keeps the same elements, keeps the same length, and puts them in order for every generated list the test can reasonably explore. The examples are still welcome. They carry business meaning. The property is there because software has an alarming talent for meeting the invitation and misbehaving everywhere else.

The machinery has five jobs. A property states the rule. A generator describes the input domain. The runner repeats the test, and a shrinker tries to reduce any failure to a smaller counterexample. The report then preserves the reduced case and replay information. This is why a failure involving a five-hundred-item structure can eventually arrive wearing a two-item hat and looking much less mysterious.

The surprising part is that randomness does not confer wisdom. A generator that produces mostly empty arrays can produce an impressive stack of passing results while avoiding the behavior that matters. A broad generator followed by many discarded cases has the same problem in a different coat. Construct valid values directly, generate related fields together, and inspect classifications or coverage statistics. The generator is not a vending machine for test data. It is part of the argument that the test examined something useful.

A counterexample also needs stable surroundings. Hidden time, scheduling, network responses, shared mutable state, or another random source can make the same input pass on Tuesday and fail on Wednesday. Shrinking cannot make a moving target polite. Control those dependencies, then retain the concrete counterexample along with a replay token or seed. The seed can help repeat a generation path, but the counterexample is the evidence that survives changes in library settings.

For a map of the moving parts, open the Slides. The Intro explains property patterns, generator design, shrinking, and stateful testing in full. The Cheatsheet is the quick reference when a discard rate rises or a counterexample refuses to reproduce. The Practice tab turns a compact contract into a reviewable test design, and the Exercise uses sorting to make the whole arrangement fail safely before it passes usefully. The Quiz checks whether the vocabulary has stopped disguising itself as magic.

Don't Panic: Property-Based Testing

Relevant careers