What’s the next bottleneck?

  1. The picture after look counter: The basin won't drain.look counter
  2. The picture after inspect C1: Close-up of the cleanout plug.inspect C1
  3. The picture after open_joint T1: Trap off, over the bucket.open_joint T1
  4. The picture after run_tap both 1: It drains.run_tap both 1
Handsnot tested
briefinggolookinspectrun_tapwaitset_valveplace_bucketopen_jointclose_jointrepairfinish
World modeltested
Kitchen basin before running the tap run_tap cold 3 The same basin after: the water rose instead of draining
Contexttested
kitchen, counterkitchen, under sinkbathroom, counterbathroom, under sinkutility, wall

Six faults

trap clog, T2 up close
trap_clog
branch clog, C1 up close
branch_clog
supply leak, J7 up close
supply_leak
drain leak, J6 up close
drain_leak
aerator clog, A1 up close
aerator_clog
valve not open, V4 up close
valve_not_open

Solvable. Not gameable.

00.250.50.751Level 1Level 2Level 3Level 4FaultsHidden behind anothernonoVague complaintnoDecoysnoneTime budget45455565
Table
Reference plumberServices everything, never testsFive exploits
Level 10.96 (0.93–0.98)0.41 (0.00–0.85)0.00 (0.00–0.00)
Level 20.96 (0.93–0.98)0.37 (0.00–0.85)0.00 (0.00–0.00)
Level 30.95 (0.93–0.97)0.43 (0.00–0.85)0.00 (0.00–0.00)
Level 40.94 (0.91–0.98)0.51 (0.06–0.85)0.00 (0.00–0.00)

Mean (lowest–highest) over 100 houses per level; the five exploits pooled.

Where models break

00.250.50.751Level 1Level 2Level 3Level 4

5 of 8level 3–4 jobs reported done with a fault still in the house

ModelLevelsJobsScoreEvery fault fixedFaults named rightTokens read per jobWrittenTool callsTime
Codex, default model1–240.904 of 44 of 4428k (92% cached)1.4k1850 s
Codex, default model3–480.512 of 82 of 8731k (94% cached)2.0k2657 s

Scored by replay

  1. AgentOpenCode + model
  2. Housesimulator + renderer
  3. Verifierreplays it from the seed
  4. Score 0–1
repair ×

repair: share of faults fixed, −0.25 per part left worse

harbor run --repo valuhaka/plumberbench -d plumberbench -a opencode -m MODEL

Try it

Junction 2026 · AfterQuery, Break the Agent