Independent evaluation
A product that verifies agents must not grade itself.
The KodeProof Evidence Lab uses a separate evaluation framework. KodeProof contributes native artifacts and treatment behavior; an independent sealed grader decides whether the final candidate satisfies the task.
What is frozen for a campaign
A material change creates a new campaign identity.
- task and fixture snapshot
- hidden grader
- prompt
- treatment build and contracts
- upstream provider builds
- model identity and reasoning configuration where exposed
- agent harness and configuration
- budgets
- repository before-state
- condition
- rubric and scorer version
Historical raw events are not overwritten.
Independent outcomes
Do not compress distinct failures into one safety number.
- Candidate task success: did the final candidate satisfy the sealed task and hidden requirements?
- KodeProof trust outcome: did KodeProof pass, block, stop, refuse, or transfer?
- Delivered success: did a valid candidate reach the declared delivery boundary?
- False confidence: did the agent or product claim trusted completion while an independent grader found material failure?
- False block: did KodeProof block a candidate that satisfied the full declared transaction and independent task contract?
Four separate safety claims
Original-worktree corruption rate. Transfer-policy violation rate. Independent hidden-task regression rate. False-confidence rate.
Cost and usability
A safety layer that prevents errors but makes the workflow unusable may fail commercially.
- wall time
- fresh and cached input tokens
- output and reasoning tokens where available
- monetary cost
- tool and command activity
- retries and timeouts
- human intervention
- agent confusion
- false-block repair cost
- time to candidate completion
- time to trusted and delivered completion
Publication policy
Methodology before headline claims.
We intend to publish product and campaign versions, data completeness, catches and false blocks, limitations, invalidated campaigns, and field evidence separately from preregistered trial evidence. We will not publish a simple leaderboard that hides condition, task, cost, coverage, or invalid trials.
Take the next step
Evaluate the whole transaction.
The private alpha is for teams willing to review catches, false blocks, missing coverage, workflow cost, and transfer behavior—not only the happy path.