Don't Panic — Continuous Integration

Continuous integration is a cadence before it is a tool, and the whole practice fits inside one claim about frequency: nobody's work stays unmerged for longer than a day, and nothing lands on the branch everyone shares without an automated build having run the tests across it first.

Buy Jenkins, adopt GitHub Actions, install whatever you fancy — where changes still ripen on branches for a fortnight, none of it qualifies. The service runs builds. The practice is the interval.

What it exists to prevent is the private theory. A branch is one developer's guess about the product, and two clean guesses can merge without a single textual conflict and still be wrong together.

That is a semantic conflict: the text combines perfectly, the behaviour does not, and version control cannot see it. The longer branches live, the more of them exist and the staler everyone's assumptions have grown.

Shrink the interval and the batch, and integration stops being an event. Failures point at something recent, the repair surface is narrow, and whoever caused it still remembers why.

A credible loop is a short list. One mainline, the single shared branch representing the product as it currently stands. Everything needed to build and configure it under version control. A build that is automated, reproducible and self-testing — it runs the tests itself and returns an unambiguous pass or fail — triggered on every integration.

Then the part that is a social agreement rather than a setting: a broken mainline becomes the team's immediate business, fixed or reverted. Keep stacking work on a broken base and every result after it is ambiguous evidence. Ten minutes is the usual ceiling for the quick suite, with slower end-to-end and performance work in later stages.

Now the arithmetic nobody expects. Flakiness is a rate, and rates compound. One test that passes 99.99% of the time looks beyond criticism; ten thousand such tests, each independently that reliable, produce a red build roughly 63% of the time.

That is why large suites go red with nobody having broken anything, and why chasing individual flaky tests — ones that fail without any product change — never converges. What decides the outcome is the suite-level failure rate, and it is a product of the parts. Measure per-test flake rates and budget them.

All of which makes automatic retry the most tempting wrong answer available. Retrying until green turns a visible cost somebody would have fixed into an invisible one paid in compute, and once retries are policy, red no longer means broken and green no longer means working. It also conceals race conditions, which look exactly like flakes and are the defects most worth catching.

Two numbers to leave with. Mainline is a queue: once build duration multiplied by merge rate approaches one, every merge waits behind another and the pressure to batch changes returns — precisely the thing this was meant to remove. And count the minutes per week mainline spends red, rather than build success rate, which improves whenever people stop pushing.

The Intro carries the loop, and a useful list of the things CI is not. Checklist on the Cheatsheet. Read Field Notes before pinning a third-party action by tag.