TRY THISKeep the model fixed. Switch between 80% and 50% success.
THE RELIABILITY GAP METR · TH 1.1
A higher bar. A shorter horizon.
1.2 hrClaude Opus 4.6 · 80% success
Task difficulty, measured in human-expert time Log scale · model release date →
Reported interval: 27 min–2.8 hr. Selected model released 2026-02-05.
Human task time, not AI runtime
Source: METR Data updated 8 May 2026 · incomplete frontier coverage
50% PREDICTED SUCCESS12.0 hr
80% PREDICTED SUCCESS1.2 hr
What changed?
The success requirement is 80%. The selected model configuration and dated evaluation snapshot stay fixed when you switch thresholds.
Keep the limits in view
Human-expert task time, not AI runtime. Results depend on the evaluated software tasks and setup. Neither threshold guarantees success on your task.
Exact configuration: claude_opus_4_6_inspect Scaffold: mtb/start_metr_task,metr_agents/react · ai_rd_fix_embedding/init_solver,use_tools,metr_agents/react · re_bench_common/init_solver,use_tools,metr_agents/react · metr_tasks_swaa/init_solver,generate · metr_agents/react. Snapshot updated 8 May 2026; checked against the published file on 2 October 2026.
Reported measurement · METR snapshot
What this helps you see
This chart measures task difficulty in human-expert time. The horizon is a fitted estimate at a chosen success rate—not how long the AI ran.
Keep the limits in view.
Software-task evaluation does not establish reliability on our teaching receipt example or on every kind of work. It does not establish shutdown resistance.
