Runtime Characterization Catches What Static Analysis Cannot See
Captured calls, receivers, values, effects, readiness, and state coverage can expose refactor regressions beyond the source graph—when their limits stay explicit.
Static analysis can prove a great deal about a refactor. It cannot know every value that crossed a dynamic boundary or every effect produced by a live application.
Runtime characterization records selected behavior from the before-state and turns it into a portable comparison surface for the candidate.
A trace needs more than arguments and return values
Two calls with the same arguments can behave differently because of their receiver, mutation, scheduling, or host effects.
A useful record can include:
- function identity;
- canonicalized arguments;
- receiver snapshot;
- return value or thrown error;
- captured side-effect shape;
- state and readiness provenance;
- serializer and support versions.
Deduplication should consider all of these dimensions so distinct behaviors are not collapsed into one case.
Canonicalization must preserve meaningful identity
Runtime values can contain cycles, typed arrays, maps, sets, class instances, scene objects, geometry, materials, and textures.
The serializer needs explicit rules. Session-local numeric IDs may be noise, while a stable UUID can be an intra-run reference witness without being compared across runs. Scene transforms should retain position, rotation, quaternion, scale, and a matrix witness rather than attempting a lossy decomposition later.
Calling arbitrary toJSON() methods is unsafe because their output can be version-dependent or omit important fields.
Bounds should fail visibly
Large collections and trace volumes need caps. Truncation must retain size metadata so different-length collections do not compare equal accidentally.
The remaining blind spot should be documented: equal-size collections that differ only beyond the retained prefix may compare equal. A replay using a truncated argument may also be rejected because the function sees incomplete input.
Failing closed is better than manufacturing an exact claim from a sample.
Readiness is part of the observation
A trace captured before hydration or during an unstable render loop may not represent the intended state.
A readiness ladder can progress from document readiness through DOM and frame stability, framework hydration, application hooks, semantic snapshot stability, and render-loop calm. Timeout remains an explicit failure rather than a fallback to ready.
The run should report its weakest readiness evidence.
Coverage can regress while assertions pass
Suppose every replayed trace matches, but the candidate reached fewer states, lost natural navigation, skipped a transition, or fell back to a weaker readiness signal.
That run exercised less than the accepted baseline. It should be rejectable even though the assertions it did execute passed.
Coverage regression reasons can include shrunk state or transition coverage, fewer functions observed, lost natural reachability, readiness regression, lost semantic snapshots, new skipped states, or bridge-contract failure.
Hooks are optional and disclosed
An application can expose bounded hooks for readiness, state selection, semantic snapshots, or interactions. These can improve coverage but should never be disguised as natural user reachability.
Each visit records whether it was natural, scripted, hook-assisted, or synthetic. One assisted step downgrades the provenance of that visit.
KodeProof uses runtime characterization to complement structural verification, not replace it. The harness is valuable because it can say both “this observed behavior matched” and “this is exactly how much behavior we were able to observe.”