There is a specific kind of trust that engineering teams extend to a new hire. You don't give them a critical production task on day one. You watch them handle something small. Then something medium. You look at how they handle an edge case they weren't prepared for. Only after months of evidence do you let them work unsupervised on something that matters.
Silas Alberti, SVP of Research at Cognition, has applied that same standard to every AI model his team has tested inside Devin, Cognition's autonomous AI software engineer. He has run nearly every Claude generation through it. And Claude Fable 5 is the first model he'd trust to leave running overnight.
That is not a marketing claim. It is a specific, earned threshold — and understanding how Cognition got there tells you something important about where agentic AI actually stands right now.
What Cognition is building and why the bar is high
Devin takes on the engineering work that never quite gets done: codebase migrations, backlogs of bugs, features that keep slipping from one sprint to the next. Its customers range from high-growth startups to Fortune 500 companies. Code written by Devin has to be reliable and production-ready. A quiet bug introduced at 2am causes real problems by 9am.
Alberti's team trains and evaluates the models behind Devin and has developed a sharp instinct for the gap between benchmark performance and real-world reliability. They have watched models score well on standard evaluations and then fall apart the moment a Cognition engineer tried to use them in practice.
"We trust no eval," Alberti says — meaning their own engineers, their highest-taste developers, put each new model through a real day of work. The bar is whether the code produced is something they'd actually keep.
That standard is why his assessment of Claude Fable 5 is worth paying attention to.
Where earlier models hit their ceiling
Cognition traces the first real jump in agentic reliability to Claude 3.6 Sonnet in late 2024. It was the first model that could chain tools reliably and hold a multi-step task. When the team integrated it into Devin, internal usage tripled. That was a signal.
But one ceiling remained across every model since: how long an agent could run before it lost the thread.
"Before Fable, you could delegate agents that could stay on-task for a couple of minutes, maybe an hour," Alberti says. Beyond that, sessions drifted. Give an earlier model five ideas to weigh simultaneously and it would lose track and get confused. A prior Opus model, tasked with a database migration, technically finished — but introduced a series of subtle bugs along the way.
Incident triage showed the same pattern. Earlier models stayed at the surface of logs instead of digging for the relevant line. Worse, they were trained to give an answer regardless of confidence. They would, as Alberti puts it, "confidently claim the first plausible thing they discover and then stop." Engineers learned to discount that signal.
The length-of-task ceiling was not a prompting problem or a workflow problem. It was a model problem.
How Cognition tests for real-world reliability
Before discussing what changed with Fable 5, it is worth understanding how Cognition measures it — because their evaluation methodology is more demanding than most.
Cognition built FrontierCode, a proprietary benchmark, specifically because existing evaluations kept rewarding code that passed isolated tests but would not survive a real production codebase. Alberti calls it an "anti-slop" standard. The benchmark grades models on whether their code is production-ready and mergeable — not just whether it runs.
On FrontierCode's hardest subset, the prior Opus model scored around 10%. Claude Fable 5 scored approximately 30% — a tripling on a benchmark that Cognition's own engineers cannot easily game.
The team's first reaction was suspicion. "Is there a bug? This can't be true." Usually a benchmark jump of this size triggers weeks of internal debate over whether the model is genuinely better in practice. This time the engineers using it agreed with the numbers.
"It was kind of a shocker, honestly," Alberti says.
The overnight test — and what it actually means
The specific capability that changed with Fable 5 is what Alberti calls the horizon — how long the model can run autonomously before it drifts, loses context, or makes a consequential mistake it doesn't flag.
"The biggest thing we noticed was the horizon, how long it can be self-sufficient," he says. "There have been tasks where I was about to go to bed and I was like, 'Okay, just please keep working on this and don't stop until I wake up.' And then I wake up, and it's been working for eight hours straight and actually making real progress. I hadn't seen that before."
Eight hours of autonomous work that produces something reviewable in the morning is not a minor capability improvement. It represents a qualitative shift in what it means to delegate engineering work to an AI agent.
Two things made the longer horizon possible.
First, contextual clarity in noisy environments. Claude Fable 5 was the first model Cognition tested that could properly use their internal debugging tools — paging through messy logs in a browser and drawing reliable conclusions despite the noise. Earlier models stayed at the surface. Fable 5 digs to the relevant line.
Second, honest uncertainty. On incident triage tasks, Fable 5 pinned down root causes and — critically — stated what it did not know. That last part is what rebuilt trust with Cognition's engineers. A model that knows the limits of its own conclusions is more useful than one that sounds confident about everything. "It checked its own work before handing it back," Alberti says, "like an employee who re-reads before hitting send."
The database migration that earlier models got wrong
One specific test illustrates the difference clearly.
A database migration that had tripped up earlier Opus models — one where the previous model technically completed the task but introduced subtle bugs throughout — was assigned to Claude Fable 5. Before starting, the model stated the invariants it would hold itself to during the migration. Then it executed against them.
Stating invariants before execution is a significant behaviour. It means the model is encoding its own constraints explicitly, which makes deviations visible and auditable. It is the AI equivalent of a careful engineer writing down the assumptions they're working from before they touch the codebase.
The migration completed without the class of subtle bugs that had appeared before.
What Fable 5 changes for engineering teams
Cognition's founding bet was that AI agents should be able to run in the cloud for hours at a time, handling real engineering work autonomously. For the company's first year of existence, the models were not capable enough to make that bet viable.
Alberti says Claude Fable 5 makes the full version of that bet viable — and some of it is already in the product. Devin can now watch a Slack channel and jump into an issue without being tagged. It can monitor production and triage a spike independently. When it gets one of those right, Alberti says it feels "like a real engineer on the team."
His expectation for the near term: in a year or two, 90% of agent sessions will be proactive ones — the agent finds a problem, scans the codebase, and messages the team with a proposed fix, unprompted.
"A lot of these things we've always wanted to build at the company are now possible," Alberti says.
What this means beyond engineering
The Cognition case study is specific to software engineering, but the underlying dynamic applies more broadly.
The threshold Alberti describes — the point at which you trust an agent to work unsupervised on something that matters — is a threshold that every team working with AI agents is navigating right now. What specific evidence would shift your level of trust? How do you test for it without using your most critical work as the testing ground?
Cognition's approach is worth borrowing: build your own evaluation that reflects your actual quality bar, not someone else's benchmark. Use your highest-judgment people to validate the numbers. And watch specifically for whether the model knows what it does not know — because confident wrongness is far more dangerous than honest uncertainty.
The models capable of genuinely long-horizon autonomous work are arriving. The question for every team is whether you have built the evaluation infrastructure to know when to trust them — and with what.
Frequently asked questions
What is Claude Fable 5? Claude Fable 5 is Anthropic's most capable generally available model as of mid-2026, designed for long-horizon agentic tasks. It supports a one-million-token context window and is built for complex, multi-step autonomous work including software engineering, research, and knowledge tasks.
What is Devin and how does it use Claude Fable 5? Devin is an autonomous AI software engineer built by Cognition. It handles engineering tasks like codebase migrations, bug fixes, and feature development. Cognition integrated Claude Fable 5 as the model powering Devin's Ultra agent — its most capable tier, designed for long-horizon tasks and complex debugging.
What is FrontierCode and why does it matter? FrontierCode is Cognition's proprietary benchmark for evaluating AI models on real-world engineering quality. Unlike standard benchmarks that measure whether code passes tests, FrontierCode grades on whether code is production-ready and mergeable into a live codebase. Claude Fable 5 scored approximately 30% on FrontierCode's hardest subset, compared to around 10% for the prior Opus model.
What does "long-horizon agentic task" mean? A long-horizon agentic task is a complex, multi-step piece of work that an AI agent runs autonomously over an extended period — hours rather than minutes — without human intervention at each step. The challenge is maintaining coherence, accuracy, and honest uncertainty over a long session. Claude Fable 5 is the first model Cognition's team trusted to run unattended for up to eight hours on production-relevant work.
Can Claude Fable 5 work unattended overnight on real engineering tasks? According to Cognition's Silas Alberti, yes — it is the first model their team has trusted to do so. The key improvements are its ability to navigate noisy debugging environments reliably and its tendency to state what it does not know rather than producing confident but incorrect conclusions.
How is Claude Fable 5 different from Claude Opus for coding tasks? On Cognition's FrontierCode benchmark, Fable 5 scored roughly three times higher than the prior Opus model on the hardest tasks. More importantly, Fable 5 can sustain autonomous work for significantly longer without losing context or drifting — a qualitative capability difference, not just a benchmark improvement.
This blog is part of Bitroot's ongoing coverage of AI tools, workflows, and strategies for founders and builders. All insights, guides, and resources are available free at Bitroot — no paywall, no fluff.