Claude runs a 50,412-account revenue forecast overnight, then backtests itself | Aravind Arumugam
ai5 min read
Claude runs a 50,412-account revenue forecast overnight, then backtests itself
Anthropic published a demo this week showing Claude Fable 5.1 maintaining a company's revenue forecast overnight, without anyone watching it. The interesting part is not that it runs unattended. It is that it grades its own past predictions when asked, and it still cannot send anything until a person approves it.
Here is what actually happens between one working day and the next, and why the last step matters more than the automation.
What actually runs overnight?
A single analyst at a B2B company owns the revenue forecast for 50,412 accounts. That business has a mix of contract revenue, which is predictable, and consumption revenue, which is not. Forecasting it by hand is a week of work that is stale by the time it is finished.
In the demo, Claude rebuilds the forecast every night on Anthropic's API. It reads the actual source material rather than a summary of it: signed contracts, invoices, and usage records.
That distinction matters. In one account it picks out an invoice for $105,838.42 and notices the customer has exceeded its included usage four months running, and that the overage is accelerating: $15.25, then $412.60, then $1,211.77, then $2,120.40. It also reads the contract clause confirming the agreement renews automatically for successive twelve-month terms.
That is one account out of fifty thousand, and it does the same across all of them.
How does it group fifty thousand accounts?
Not by region, industry, or which sales team owns the relationship. It sorts every account by how it actually consumes, which produces five behavioural cohorts:
Seasonal
Erratic
Step function
Plateau
Linear
This is the quiet insight in the whole demo. Most forecasting is organised around how a company is structured internally, because that is how the reporting lines run. Behaviour is a better predictor of next quarter's revenue than the org chart is.
What does ten thousand scenarios per cohort buy you?
Each cohort is then run through 10,000 scenarios. The output is not a single number but a distribution, which is why the interface reports a spread alongside the forecast: a P90 to P10 range of 8.8%.
A forecast without a spread invites false confidence. "Sixty four million" and "sixty four million, give or take nine percent" are very different statements to put in front of a board, and only one of them is honest about what is known.
By morning, the Q3 P50 forecast has moved from $63,962,532 to $64,232,806, with $35.3M already locked in under contract, and six accounts have quietly moved into the erratic cohort overnight. Every figure is traceable to a specific run, labelled in the interface as run #471.
Can it tell you how wrong it has been?
This is the part worth stealing regardless of what tools you use.
In the morning the analyst asks a straightforward question: the CFO wants to know how accurate we have actually been. Claude backtests itself on the spot, comparing four quarters of its own nightly forecasts against what actually happened, and reports:
Average error 6.6%
Down from 8.9% a year ago
5.4% last quarter
Note the honesty in that middle figure. The error is down year on year but up against last quarter, and it says so rather than quoting only the flattering comparison. A system that reports its own regression is considerably more useful than one that only reports improvement.
Most forecasting tools never answer this question at all, because nobody stored the old predictions next to the eventual actuals. If you build one thing out of this article, make it that: keep every forecast you publish, so the system can be marked against reality later.
What stops it from sending the forecast itself?
Nothing technical. That is the point.
The work is finished, checked and formatted by the time the analyst opens it. And the forecast still does not leave the building until a human reads it and presses Mark as reviewed.
It would be trivial to remove that step. Plenty of teams will. But the demo is more instructive with it in, because it draws the line in the right place: the system is trusted to do the work and not trusted to decide the work is correct.
That maps onto a rule worth applying to any automation you build. Automate the labour. Keep the sign-off. The failure mode of an unattended forecast is not that it crashes, it is that it is confidently wrong at scale and nobody looked.
What would I take from this?
Three things, in order of how easy they are to copy:
Report a spread, not a number. Any forecast you publish should carry its own uncertainty next to it.
Store your predictions. You cannot backtest what you did not keep. This costs almost nothing and pays for itself the first time somebody asks how accurate you have been.
Group by behaviour, not by org chart. How a customer spends predicts how they will spend. Which team owns them does not.
Summary
Claude Fable 5.1 rebuilds a 50,412-account revenue forecast overnight, unattended, reading real contracts and invoices rather than summaries. It groups accounts into five behavioural cohorts, runs 10,000 scenarios per cohort, and reports a P90–P10 spread of 8.8% alongside a $64.2M Q3 figure. Asked how accurate it has been, it backtests four quarters of its own forecasts and reports an average error of 6.6%, down from 8.9% a year ago but up from 5.4% last quarter. And it still cannot publish anything until a person presses Mark as reviewed.
The automation is impressive. The approval gate is the part worth copying.
