Every figure, with the command that produced it.
Harald is measured on the public CUAD set (The Atticus Project, CC BY 4.0): 510 commercial contracts with the clauses marked by lawyers.
Ten questions on 510 contracts
One run on 11 November 2026: ten questions on every contract, 5,100 cells.
2,661
Verified quotes
Each one found in its contract by code.
95.5%
Recall
Of the clauses CUAD marks, the share Harald found.
78.6%
Precision
Of Harald’s answers, the share CUAD also marks. F1 on verified cells: 86.3%.
LIVE=1 pnpm exec tsx packages/eval/src/cli.ts --set full --client live --seed 1 --columns packages/eval/fixtures/columns-all10.json --cap 5
How to read the figures
What was measured, and what each number means.
- The set
- CUAD holds 510 commercial contracts in English. Lawyers marked where each type of clause appears, and those marks are the reference.
- The questions
- Ten clause types, among them governing law, termination for convenience, exclusivity, non-compete and change of control.
- Verified
- A cell counts as verified only when code finds its quote in the contract as one continuous passage.
- Recall and precision
- Recall is the share of marked clauses that Harald found. Precision is the share of Harald’s answers that the lawyers also marked.
- Your contracts
- These are results on a public benchmark. Results on a firm’s own contracts may differ.
The check itself is on the first page.
Change one word of a quote and see what the rule does with it.