Technology · Field notes — REF / HORIZON

Will AI Prevent Another Horizon?

Everyone wants to know whether smarter software would have caught the bug. It is the wrong question. The fatal fault in Horizon was never in the code.

---

For a quarter of a century the Post Office held to one position with the conviction of scripture: the computer was right, and the people were wrong. More than nine hundred sub-postmasters were prosecuted on that basis. Lives were ruined, some of them ended. It remains the most widespread miscarriage of justice in British legal history, and it turned on a single, fatal article of faith — that the machine does not lie.

So now, with a far cleverer breed of machine arriving in every workplace, the natural question is whether it would have saved them. Would a modern AI, let loose on the Horizon system, have caught what the courts could not? It is a comforting question, and I understand why people ask it. I think it misses the point so completely that answering it on its own terms does more harm than good.

Horizon was not a software failure

Let me be clear, because this is where most takes go wrong. Yes, Horizon had bugs. Phantom shortfalls that appeared from nowhere. Transactions that reversed themselves. A remote-access capability that let Fujitsu reach into branch accounts and alter figures without the sub-postmaster ever knowing. These were real defects, and they mattered.

But bugs are not remarkable. Every non-trivial system ever shipped has them. I have said before that there are still bugs in Horizon today — they simply haven't been found yet. That is not cynicism; it is how software works. A buggy accounting system is a Tuesday. It is not a national scandal.

What turned defects into a catastrophe was not the defects. It was the decision — taken by people, knowingly, and repeated for years — to trust the machine over the human in front of them, to withhold what was known, and to prosecute anyway. The disaster was a choice. Several thousand of them, in fact, made by individuals with names and salaries and the option, at every turn, to do otherwise.

What AI would genuinely have caught

I want to be fair to the optimistic case, because there is one. Point a modern anomaly-detection model at the Horizon transaction logs and it would have raised its hand almost at once. The same impossible shortfall surfacing across unconnected branches. Balances that reconciled to no human action. Corrections applied with no corresponding entry. This is precisely what such systems are good at: finding the one inconsistency in a mountain of data, tirelessly, at three in the morning, without getting bored or being told to stop looking.

Run a capable model over the codebase and it might well have flagged the remote-access function and the silent edits as exactly the kind of thing that should never exist in a system used as courtroom evidence. So on the narrow technical question — could AI have spotted the problem sooner — the honest answer is yes. It probably could.

But the machines already knew

Here is the part the optimists skate straight past. The discrepancies were never hidden from the computers. They were in the data. Fujitsu's own engineers could see them. The system logged the remote access. The information that would have exonerated hundreds of innocent people sat inside the technology the entire time, available to anyone who cared to look.

Horizon did not fail to detect the problem. People failed to act on what the detection plainly showed them — and, worse, chose to suppress it. The bottleneck was never detection. You cannot fix a refusal to look with a better pair of eyes.

AI will tell you the numbers don't add up. It will not phone the regulator.

AI does not come with a spine

A model will tell you, instantly and without fear, that the figures are impossible. It will not then walk into a manager's office and say we are prosecuting innocent people. It will not refuse to sign the witness statement it knows is misleading. It will not put its own career on the line to stop a prosecution. Those are acts of conscience, and conscience is the single component Horizon lacked — and the one thing no model ships with.

I have sat in a witness box on IT matters. I know what it costs a person to tell an uncomfortable truth under oath, and I know what it costs everyone else when they decline to. None of that arithmetic changes because the software got cleverer. The hard part of Horizon was never the part a computer could do. It was the part where a human being had to be brave, and too few were.

The new danger is worse than the old one

The Post Office's entire institutional defence reduced to four words: the computer is reliable. It rode on a long-standing legal presumption — that a computer was working properly unless you could prove otherwise — which Horizon exposed as dangerous and which has since, belatedly, come under review.

Now consider what we are about to do. We are replacing the merely opaque computer with one that writes its own code, reasons in ways nobody fully audits, and cannot always explain itself even to the people who built it. The "computer says" defence does not weaken in this world. It gets stronger, more impressive, and far harder to challenge from the witness box — because now even the engineers can only shrug. Automation bias was the accelerant last time. We are preparing to pour a great deal more of it onto the fire. When a sub-postmaster in 2032 says "the AI got it wrong," who in that courtroom will be equipped to prove they're right?

Where AI could actually earn its keep

None of this means the technology is useless against the next Horizon. It means we are aiming it at the wrong target. Stop pointing it at the accounting engine and point it at the human layer — the place the failure actually lived.

Tamper-evident, immutable audit logs that no engineer can quietly amend. Anomaly reports routed to somewhere a frightened middle manager cannot intercept them — to an independent board, not up a chain that punishes bad news. Whistleblower channels that surface a pattern after five branches, not nine hundred prosecutions. The value is not a smarter till. It is using the technology to remove the human discretion that let the cover-up breathe — to make looking away harder than facing the thing. That is the Root Cause Analysis instinct: repair the system that produced the failure, not merely the symptom it threw off. AI can help build that scaffolding. It will not choose to build it. We have to.

A year on from the first volume of Sir Wyn Williams's final report, the scale of the human damage is set out in unsparing detail. What is still not settled is the bill for it. Fujitsu has conceded a "moral obligation" to contribute and set aside nothing against it, pending the remainder of the report it says it is waiting on. The total redress is now valued in the region of two billion pounds.

Where it stands, mid-2026 — the suffering catalogued, the contribution still unquantified. The defects we understood years ago. It is the accountability that remains outstanding.

The honest answer

So — will AI prevent another Horizon? It will probably prevent a few. It will catch defects sooner, flag the impossible faster, and spare some people the version of this that begins with a software bug. That is worth having, and I won't pretend otherwise.

But the Horizon that mattered did not begin with a bug. It began with a culture that prized the institution's reputation above the truth, leaders who did not want to be troubled with detail, and a presumption that the machine does not lie. AI touches none of that. If anything, it hands that same culture a more dazzling machine to hide behind, and a better answer to give when someone asks how they could possibly have known.

The next Horizon will not be stopped by a cleverer model. It will be stopped, if it is stopped at all, by someone willing to say "this is wrong" — and by an organisation, for once, willing to listen. We have built remarkable tools for finding the truth. We have never been much good at the part that comes after, where we are supposed to act on it.

Ends