Ω OmniBench
back to run 1504bf5f
agency · T4 · agency · scaffold L2

Aggregate across files passed

ag_t4_002

Score
100%
Turns
8
8 useful · 0 useless
Interventions
0
harness hints (HHI)
Tool calls
3/3
matched expected trajectory
Errors
0/0
recovered / total
Latency
2.08s
Tokens
1,614
0 reasoning
Cost
$0.00021

Trajectory timeline

Turn-by-turn replay · useless turns annotated with the classifier reason

  1. #0modeluseful
    plan
  2. #1modeluseful
    → read_file
  3. #2tooluseful
    read_file({"path":"README.md"}) → # Project Kestrel Status: green Release: 2.4.0
    → # Project Kestrel Status: green Release: 2.4.0
  4. #3modeluseful
    → read_file
  5. #4tooluseful
    read_file({"path":"notes.txt"}) → Meeting moved to Thursday. Budget cap is 12000 USD. Owner: Priya
    → Meeting moved to Thursday. Budget cap is 12000 USD. Owner: Priya
  6. #5modeluseful
    → kv_set
  7. #6tooluseful
    kv_set({"key":"report","value":"v2.4.0|12000"}) → OK
    → OK
  8. #7modeluseful
    done

Prompt / output inspector

instruction
Find the release version in README.md and the budget cap in notes.txt. Store a summary in the key-value store under key 'report' formatted exactly as v<version>|<budget> (e.g. v1.0.0|500). Reply 'done'.
final output
done
expected
v2.4.0|12000
expected trajectory
read_file:README.md
read_file:notes.txt
kv_set:report=v2.4.0|12000

Spans

8 recorded · tokens & cost per span

  • plan468ms
    97 in · 42 cached · 14 out · $0.00002 · ttft 342ms
  • turn#1448ms
    218 in · 93 cached · 14 out · $0.00004 · ttft 332ms
  • read_file8ms
  • turn#2363ms
    237 in · 102 cached · 14 out · $0.00005 · ttft 238ms
  • read_file7ms
  • turn#3405ms
    260 in · 111 cached · 19 out · $0.00005 · ttft 234ms
  • kv_set6ms
  • turn#4385ms
    274 in · 118 cached · 1 out · $0.00004 · ttft 376ms

Events

  • task_started{"trial":0,"taskId":"ag_t4_002","scaffold":"L2"}
  • replan{"plan":"1. read_file({\"path\":\"README.md\"})"}
  • task_finished{"score":1,"passed":true,"taskId":"ag_t4_002","failureLabel":null}

Scorer metadata

{
 "toolLog": [
  "read_file({\"path\":\"README.md\"})",
  "read_file({\"path\":\"notes.txt\"})",
  "kv_set({\"key\":\"report\",\"value\":\"v2.4.0|12000\"})"
 ],
 "scaffold": "L2"
}
Agencystate_checkdifficulty 3/5read_filelist_fileskv_set