Official benchmark
The honours board
Big claims. Small carpets.
Standard tools · Photo only
Exact-pub accuracy
The board is awaiting its first results.
Nothing appears here until a complete run is scored, adjudicated, and published with its manifest. We are not going to eyeball it.
Columns
- accuracy
- Exact establishment, averaged per pub then across pubs.
- cost
- Per attempt and per correct answer, tools included.
- latency
- Median and p95, plus tool calls.
- coverage
- Abstentions, malformed output, timeouts, budget exhaustion, each counted.
- uncertainty
- Intervals clustered by pub. Ties reported as ties.