Overview
Benchmark control room
Every number here is a view over the run store. Click through any model, run or task to walk the recording ladder down to individual spans.
Models
3
3 benchmarked
Runs
5
all settled
Task results
330
151 passed · 47 failed
Spans recorded
759
463 events · 0 judge calls
Total spend
$0.5851
106.7k tokens · priced at write time
Score landscape
Latest finished run per model · pillar composites 0–100 · drag to orbit, hover for values
Rendering 3D view…
Leader · Swift Budget (simulated)
Omni score and pillar gauges
Leaderboard
Latest finished run per model · Elo from pairwise task outcomes
Pillar profiles
Stacked glass radar · top models
Rendering 3D view…
Omni score across runs
Every finished run, chronological
Spend per run
USD, priced at write time
Task outcomes
Latest run per model, pooled
76%
pass rate
Recent runs
| Run | Model | Pillars | Status | Omni | When |
|---|---|---|---|---|---|
| f2f54103 | sim:local-llama | CapabilityReliabilitySafetyAgencyEconomics | finished | 58.4 | 22s ago |
| f5e098e1 | sim:local-llama | CapabilityReliabilitySafetyAgencyEconomics | finished | 58.4 | 22s ago |
| 1504bf5f | sim:swift-budget | CapabilityReliabilitySafetyAgencyEconomics | finished | 82.0 | 23s ago |
| 1d732c0d | sim:swift-budget | CapabilityReliabilitySafetyAgencyEconomics | finished | 82.0 | 23s ago |
| 422ad29b | sim:atlas-frontier | CapabilityReliabilitySafetyAgencyEconomics | finished | 79.5 | 24s ago |
Pillars
What each score composes
- CapabilityKnowledge, math, code, reasoning, instruction-following and tool-calling on an unsaturated battery.
- ReliabilityTruthfulness, hallucination, calibration (ECE), consistency across seeds and abstention behaviour.
- SafetyAttack-success-rate across injection, jailbreak, leakage, toxicity, bias & agentic probes; OWASP mapped.
- AgencyFixed ReAct / Plan-and-Execute scaffolds over a tiered task battery: useful vs. useless turns, HHI, recovery.
- EconomicsTokens, cache hits, cost-per-solved-task, quality-per-dollar, TTFT / TPOT — priced at write time.