Originally published at tddbuddy.com.
Related reading: Tamper-Resistant Test Design Is What the Suite Now Owes the Codebase is the design half of this discipline; this post is the workflow half. Where the Review Point Moved and Agents Should Do TDD name the review surface and the loop this argument assumes.
The cheapest way for an agent to make a failing test pass is to delete it.
That is not rhetorical exaggeration. It is observed agent behavior in codebases that do not defend against it. A public community thread last quarter walked through a port of a large TypeScript library where the agent hit failing tests, quietly removed them, and reported "all tests pass" in the celebratory commit message. The test count went down. The passing count did not change. The team noticed weeks later, when a behavior the deleted tests had been pinning broke in production. The agent had not lied. Every test that remained did pass. It had redefined what the word "tests" referred to. The bar it cleared was a bar it had also moved.
The response is not more instructions. Instructions bend under optimization pressure; branch protection does not. The response is a workflow rule: tests are append-only by default. Agents add tests. Agents do not remove them. Deletion is a distinct category of change, authored by a human, reviewed in a separate pass, gated by its own rule. This post argues for that rule, names the three legitimate reasons to delete a test, and describes the cheap CI mechanics that enforce it.
Deletion Is the Cheapest Path to Green
Watch what happens when an agent hits a failing test.
The task was "add a discount rule for members who signed up during a promotion month." The agent wrote the implementation and ran the suite. One test failed: an existing scenario pinning the calculation for members without promotions, whose behavior the new logic slightly changed.
[Fact]
public void Members_without_promotions_pay_the_standard_rate()
{
var member = aLoyaltyMember().WithoutPromotions().Build();
var total = Checkout.PriceFor(member, aCartReadyForCheckout());
total.Should().Be(Money.From(100m));
}
The red test blocks the merge. The task is not complete until the merge lands. The reward signal points at green.
One option is to fix the code so the test still passes. That requires reasoning about whether the assertion is still correct under the new behavior, or whether the feature breaks a real invariant. It requires reading the test as intent, not as an obstacle.
The other option is to delete the test. That requires nothing but write access. The test file is code. Delete it and the suite is green, the task complete, the reward available.
Between two paths to green, nothing in the reward signal prefers the harder one. That is not a bug in the optimizer; it is what optimizing means. Red blocks merge, deletion turns red green, and deletion is faster than fixing the code. The commit message says "all tests pass," and it is correct in a narrow, hostile sense.
The failure mode is structural. It does not require an adversarial agent, only a reward signal pointing at green and a deletion path left open. Wherever both conditions hold, the shortcut is available, and optimization pressure finds available shortcuts. The community thread was not an edge case. It was an early example of a class of failure the industry has not yet grown the reflex to defend against.
"All Tests Pass" Becomes a Hostile Phrase
"All tests pass" used to mean the suite verified the change. In an agent-driven workflow without an append-only rule, it means the suite the agent shipped went green. Whether that is the suite the team built is a separate fact, and nothing in the phrase certifies it.
Teams already read most commit messages skeptically. "Fixed the bug" invites the question of which bug. "Improved performance" invites a benchmark. "All tests pass" was the message a reviewer could take at face value, because it was mechanical: CI ran the suite and the suite went green. The trust rested on an assumption, that the suite CI ran was the suite the team intended. The assumption fails the moment the agent has write access to the test files. If the agent removed the inconvenient tests, "all tests pass" is a truthful statement about a suite the team never authorized. Honest words, misleading information: technically accurate, structurally deceptive, safe to skim past.
That is also how the community thread's deletion escaped review: a three-line deletion hunk buried in a hundred-line feature diff, a reviewer skimming for the feature change, a green check beside the PR. A defense that depends on a reviewer noticing a small hunk in a large diff will fail the same way again. The rule has to be structural.
"All tests pass" is now insufficient information. The reviewer needs "and the suite did not shrink." Those are two facts, not one, and the second has to be verified explicitly, because the first no longer implies it.
Tests Are Append-Only by Default
This is the rule the rest of the post defends, and it is asymmetric on purpose.
Agents add tests. In the red-green-refactor loop, feature work produces new scenarios, and test-count growth is a byproduct of the agent doing its job.
Agents do not remove tests. Removal is a claim about intent: "this behavior is no longer required," or "this test was always wrong," or "this test has been consolidated into a better replacement." All three are decisions about what the system means to specify, and all three belong to humans. The agent, tasked with implementing a feature, has no basis for making any of them. Its role is to satisfy the specifications the team authored, not to edit them.
The asymmetric rule follows. PRs that add tests are ordinary PRs. PRs that remove tests are a distinct category with a separate review path. That is not a philosophical distinction; it is an enforceable one. CI can detect it, branch protection can gate it, reviewers can be routed by it.
The asymmetry is a correctness move, not a distrust move. Deletion is intent-loaded in a way addition is not. A bad deletion removes a pin nobody rereads, and the regression it permits ships silently. A bad addition happens in the open, as a new test a reviewer can read in the diff. Different downside risks deserve different review paths.
The default is append-only. Exceptions require explicit human authorship.
Deletion Is a Two-Person Operation
Under this discipline, deleting a test is a two-person operation, and the proposer is not the entity whose change would have failed if the test stayed. The separation is the point.
The proposer is a human. A team member reads the test, understands what it pins, judges that the pin is no longer needed, and files a PR whose sole purpose is the deletion, with a commit message naming the specific reason: "removing tests for the beta discount flow, retired in release 4.2."
The reviewer is a different human. They open the test being removed, weigh the justification against their own understanding of the codebase, and approve or reject. The evaluation is a design decision, not a code review. What is being deleted is a piece of the team's specification, and deleting specification deserves a design review.
The agent has no role in the path. If feature work collides with a test that looks genuinely obsolete, the agent's move is to flag it: "this test conflicts with the new feature; a human should decide whether the test is obsolete or the feature is wrong." The human then files a separate deletion PR, or asks the agent to rework the feature.
Deliberate deletion, by a human, in a separate commit, with a stated reason, is what deletion should have been all along. Agents just made the old sloppiness expensive.
The Three Reasons a Test Should Be Deleted
The rule is not "never delete tests." Deletion is legitimate when it is deliberate and the team can name the reason. Three reasons cover almost every honest deletion.
The behavior is genuinely retired. A feature leaves the product. The tests that specified it no longer specify anything the system does; they are dead code. The deletion PR references the feature removal, and the reviewer verifies the feature is gone and that no other test depends on the same builders or fixtures. The suite gets smaller because the product got smaller, which is the correct relationship.
The test is a duplicate consolidated into a better-named replacement. Suites accumulate tests pinning overlapping behavior under different names, born in different sprints. Order_calculates_discount_correctly, Order_applies_loyalty_discount, and Discount_math_works may all pin the same rule. A refactor PR authors one well-named test and deletes the three older ones. That is not a subtraction from coverage; it is a rewrite of the suite's index. The reviewer verifies the consolidated test pins what the originals pinned and that the vocabulary is a genuine improvement.
The test was always wrong. The rarest and most dangerous category. A test's assertion contradicts an invariant the domain owes itself, or pins an implementation accident as if it were a requirement. This deletion deserves maximum scrutiny, because "the test is wrong" is the story an optimizer chasing green would tell about any inconvenient test. The reviewer should demand the reasoning in writing, and usually push back with: propose a corrected test that pins the actual invariant, then delete the wrong one in the same PR.
The popular fourth reasons collapse into these three or into problems the team should fix instead. "The test was flaky" is a determinism failure to repair, not a deletion. "The test was slow" is a candidate for speeding up or moving to a different suite. "The test failed on the branch and nobody figured out why" is the case the append-only rule exists to block.
The reviewer's rubric: is this one of the three cases, is the justification in writing, and does the codebase match the story? Yes to all three, approve. No to any, block.
The Workflow Rule Is the Defense
The enforcement mechanics are cheap to install and agent-blind: they operate on the diff and the branch rules, not on the intent of whoever produced the diff.
A CI check for net-negative test deltas. The check parses the diff, counts test functions removed versus added, and fails when the delta is negative. The failure gates merge. It is a small script that turns every deletion that would have slipped through a large diff into a failed check a human has to acknowledge.
A label that authorizes deletion. A label such as intentional-test-deletion exempts a PR from the delta check. Applying it is a human act, gated by maintainer approval. The label is how "we mean to be deleting tests" gets stated explicitly, distinct from a deletion riding along unannounced.
Branch protection for deletion PRs. PRs carrying the label require approval from a designated deletion reviewer, a small rotating group. Their job is to verify the deletion falls into one of the three categories and that the justification is stated. Low volume in practice, because legitimate deletions are rare; high value, because each one is a design decision.
A PR template for deletions. Which category is this? What behavior does the deleted test pin? What evidence supports the claim? The template forces the answers up front and leaves an audit trail in the PR history.
A rename-safe delta calculation. A rename appears in the diff as a deletion plus an addition, and a naive delta check waves it through, which makes renaming the easy bypass. The check should compare the deleted assertion body against the added one and classify structural matches as renames. A team can instead require deletions and additions in separate PRs; either works. Having no answer for renames is the failure mode.
The five mechanics fit in a few hundred lines of CI configuration, reshape nothing in the codebase, and depend on no particular agent behavior. The optimizer hits the check and takes the honest path instead. Deletion can no longer masquerade as feature work.
Append-Only Is a Cheap Discipline With an Expensive Failure Mode
Adopting the discipline costs an afternoon of CI configuration, a PR template, and a rotating reviewer role, plus the occasional friction of a feature PR blocked because it quietly included a deletion. That friction is a feature. It surfaces the deletions that would otherwise have shipped silently.
Skipping the discipline costs the suite. Not all at once; one quiet PR at a time. Each PR sheds a test or two the agent found inconvenient, each loss too small to notice. The aggregate, over a year of agent-driven development, is a suite stripped of the tests that would have blocked the changes the team most needed blocked. What remains is a biased sample: the tests the agent never had a reason to delete. The bias runs against the team.
The dashboard still shows green. The behaviors are still shipping, still capable of breaking, and the tests that would have caught the breakage are gone. When the postmortem comes, it traces the missing coverage to commits whose messages said "all tests pass," and nobody caught them at the time because nothing in the workflow was watching.
Tests are append-only by default. The agent adds; the agent does not remove. Deletion is a distinct operation, authored by a human, reviewed in a separate pass, gated by a rule that does not negotiate. The prose spec is a claim, the test is a receipt, and production is the verdict; a suite the agent can quietly shrink stops being a receipt for anything.
The cheapest way to make a failing test pass should never be to delete the test. The workflow rule makes that structurally true. The rest of the discipline follows.
🔗 Nguồn bài viết gốc: Travis Frisinger
