I build tools that tell you the truth about your systems.

Expired debt deliberately does not fail your pull request.

If a passing date could red a build, an engineer with an unrelated one-line change eats a failure on a Monday for someone else's debt — on a day nobody changed anything. That check gets deleted within a month. So expiry is enforced by a scheduled run instead: the PR gate stays deterministic, and the deadline still has teeth somewhere that can't block an unrelated merge.

Publishing the negative result.

The evaluation criteria were registered before the strategies ran, and nothing is revised after the fact. The experiment did not find an edge — and the honest way to report that is to publish the ledger anyway, losses included. A portfolio of only successes tells you nothing about how someone handles being wrong.

The client is never trusted to score its own game.

An Edge Function loads the canonical puzzle server-side, derives the outcome and mistake count itself, and marks the row server_validated. Scoring on the device would have been far less work — and would have made every leaderboard number meaningless the first time someone opened the debugger.

Auditing it hard enough to conclude it wasn't ready.

A structured adversarial review of the first-contact experience produced 30 claims; 28 survived a refutation pass. It found that the diff engine — the actual product — silently mishandled dates and shared references, meaning it could report "no fields changed" when a field had changed. Shipping was the easy option. Every finding was fixed the same day, in the order that made the next one observable, and the fixed tool was then pointed at a real ORM, which found four more, one of them a credential leak in its own redaction. The demo now builds and runs from a clean clone in under a minute. Before the first release, every number and absolute statement in its public documentation was checked: 306 claims, 70 corrected, 2 removed. Still unpublished, and it says so.

No accuracy number is published until there is a number worth defending.

The README separates what runs from what is designed, and publishes three numbers with error bars instead of one accuracy figure: recall of 43 of 52 defect fixtures (82.7%), 0.49 false alarms per clean diff, and a 68.9% clean pass rate, every finding judged by hand and the useless ones included. There is no precision figure on purpose: false alarms can only be counted on clean fixtures, and a number built from two populations would flatter the tool. The first verify pass was measured, found to cost more recall than it bought, and replaced before it shipped; CI holds a floor below the measurement.

Synthetic data must be unique under exactly the normalization the pipeline applies.

Identifiers were first deduplicated as raw strings. Gmail ignores dots, so two "different" addresses were one mailbox issued to two people, and a small phone space handed hundreds of households the same number. Both made a correct resolver look wrong and held the clean control at 0.82 precision where it belongs at 1.0 — a dataset that manufactures impossible collisions measures nothing.

Making the product evaluable at zero cost.

Bring-your-own-key AI plus cached demo outputs, so someone can walk the entire workflow without a key, a bill, or a sales call. An AI product that requires spending money before you can tell whether it's useful mostly measures how much you already trusted it.

It says "I don't have anything on that" before the model is ever called.

The refusal is a measured threshold on the retrieval, not a prompt asking the model to be honest. The cutoff was tuned against the question set: every unanswerable question sat past it and every answerable one inside it, and the eval fails the pipeline if that stops being true. A confident wrong answer on a resume page costs more than a plain refusal ever could.