The Test That Must Fail Before We Trust the Green

Scope: Research and architecture note. KodeProof development is frozen; this is not a release announcement.

A test suite can pass before and after a refactor because the change is safe. It can also pass because the suite never observed anything meaningful.

KodeProof’s bounded automatic adapter addresses that ambiguity by requiring existing assertions to fail on a product-generated behavior control. The point is not to make a green result harder to obtain for its own sake. It is to establish that the selected observation has some ability to disagree.

KodeProof development is frozen in the current Kalu direction. This article describes source-level research, not a new qualified product release.

Passing twice is not enough

Imagine a synthetic test that imports a module but never asserts a result. Running it against the original and candidate can produce two successful exits. The equality says little about whether a behavior changed.

A constant-green assertion has a similar problem. It can create the appearance of observation without depending on the subject the refactor is supposed to preserve.

Challenge the observation itself

The current adapter compares actual assertion observations before and after and requires a behavior control to be detected. Absent tests, skipped tests, empty assertions, and insufficient controls request characterization and block publication.

That does not make the test suite exhaustive. A control can demonstrate sensitivity to one relevant difference while leaving other behaviors unobserved. The scope and limitations of the existing assertions still need to be understood.

Keep the test authority outside the proposed edit

The adapter authenticates source, test, and configuration bytes and runs bounded scratch copies. An agent should not be able to change the expected result, remove a difficult assertion, or supply a constant success envelope while presenting the same refactor claim.

The source supports a specific existing-test shape rather than arbitrary test frameworks and project configurations. That narrowness is part of the implementation boundary, not an invitation to imply universal refactor safety.

For an evaluator, ask to see the control failure as well as the unchanged passing baseline. A verifier becomes more credible when it demonstrates the ability to reject the kind of error it claims to guard against. Green is useful only after we know something meaningful could have made it red.