Technology · Field notes — REF / HORIZON

Will AI Prevent Another Horizon?

Everyone wants to know whether smarter software would have caught the bug. It is the wrong question. The fatal fault in Horizon was never in the code.

---

For a quarter of a century the Post Office held to one position with the conviction of scripture: the computer was right, and the people were wrong. More than nine hundred sub-postmasters were prosecuted on that basis. Lives were ruined, some of them ended. It remains the most widespread miscarriage of justice in British legal history, and it turned on a single, fatal article of faith — that the machine does not lie.

So now, with a far cleverer breed of machine arriving in every workplace, the natural question is whether it would have saved them. Would a modern AI, let loose on the Horizon system, have caught what the courts could not? It is a comforting question, and I understand why people ask it. I think it misses the point so completely that answering it on its own terms does more harm than good.

Horizon was not a software failure

Let me be clear, because this is where most takes go wrong. Yes, Horizon had bugs. Phantom shortfalls that appeared from nowhere. Transactions that reversed themselves. A remote-access capability that let Fujitsu reach into branch accounts and alter figures without the sub-postmaster ever knowing. These were real defects, and they mattered.

But bugs are not remarkable. Every non-trivial system ever shipped has them. I have said before that there are still bugs in Horizon today — they simply haven't been found yet. That is not cynicism; it is how software works. A buggy accounting system is a Tuesday. It is not a national scandal.

What turned defects into a catastrophe was not the defects. It was the decision — taken by people, knowingly, and repeated for years — to trust the machine over the human in front of them, to withhold what was known, and to prosecute anyway. The disaster was a choice. Several thousand of them, in fact, made by individuals with names and salaries and the option, at every turn, to do otherwise.

What AI would genuinely have caught

I want to be fair to the optimistic case, because there is one. Point a modern anomaly-detection model at the Horizon transaction logs and it would have raised its hand almost at once. The same impossible shortfall surfacing across unconnected branches. Balances that reconciled to no human action. Corrections applied with no corresponding entry. This is precisely what such systems are good at: finding the one inconsistency in a mountain of data, tirelessly, at three in the morning, without getting bored or being told to stop looking.

Run a capable model over the codebase and it might well have flagged the remote-access function and the silent edits as exactly the kind of thing that should never exist in a system used as courtroom evidence. So on the narrow technical question — could AI have spotted the problem sooner — the honest answer is yes. It probably could.

But the machines already knew

Here is the part the optimists skate straight past. The discrepancies were never hidden from the computers. They were in the data. Fujitsu's own engineers could see them. The system logged the remote access. The information that would have exonerated hundreds of innocent people sat inside the technology the entire time, available to anyone who cared to look.

Horizon did not fail to detect the problem. People failed to act on what the detection plainly showed them — and, worse, chose to suppress it. The bottleneck was never detection. You cannot fix a refusal to look with a better pair of eyes.

AI will tell you the numbers don't add up. It will not phone the regulator.

AI does not come with a spine

A model will tell you, instantly and without fear, that the figures are impossible. It will not then walk into a manager's office and say we are prosecuting innocent people. It will not refuse to sign the witness statement it knows is misleading. It will not put its own career on the line to stop a prosecution. Those are acts of conscience, and conscience is the single component Horizon lacked — and the one thing no model ships with.

I have sat in a witness box on IT matters. I know what it costs a person to tell an uncomfortable truth under oath, and I know what it costs everyone else when they decline to. None of that arithmetic changes because the software got cleverer. The hard part of Horizon was never the part a computer could do. It was the part where a human being had to be brave, and too few were.

The new danger is worse than the old one

The Post Office's entire institutional defence reduced to four words: the computer is reliable. It rode on a long-standing legal presumption — that a computer was working properly unless you could prove otherwise — which Horizon exposed as dangerous and which has since, belatedly, come under review.

Now consider what we are about to do. We are replacing the merely opaque computer with one that writes its own code, reasons in ways nobody fully audits, and cannot always explain itself even to the people who built it. The "computer says" defence does not weaken in this world. It gets stronger, more impressive, and far harder to challenge from the witness box — because now even the engineers can only shrug. Automation bias was the accelerant last time. We are preparing to pour a great deal more of it onto the fire. When a sub-postmaster in 2032 says "the AI got it wrong," who in that courtroom will be equipped to prove they're right?

Where AI could actually earn its keep

None of this means the technology is useless against the next Horizon. It means we are aiming it at the wrong target. Stop pointing it at the accounting engine and point it at the human layer — the place the failure actually lived.

Tamper-evident, immutable audit logs that no engineer can quietly amend. Anomaly reports routed to somewhere a frightened middle manager cannot intercept them — to an independent board, not up a chain that punishes bad news. Whistleblower channels that surface a pattern after five branches, not nine hundred prosecutions. The value is not a smarter till. It is using the technology to remove the human discretion that let the cover-up breathe — to make looking away harder than facing the thing. That is the Root Cause Analysis instinct: repair the system that produced the failure, not merely the symptom it threw off. AI can help build that scaffolding. It will not choose to build it. We have to.

A year on from the first volume of Sir Wyn Williams's final report, the scale of the human damage is set out in unsparing detail. What is still not settled is the bill for it. Fujitsu has conceded a "moral obligation" to contribute and set aside nothing against it, pending the remainder of the report it says it is waiting on. The total redress is now valued in the region of two billion pounds.

Where it stands, mid-2026 — the suffering catalogued, the contribution still unquantified. The defects we understood years ago. It is the accountability that remains outstanding.

The honest answer

So — will AI prevent another Horizon? It will probably prevent a few. It will catch defects sooner, flag the impossible faster, and spare some people the version of this that begins with a software bug. That is worth having, and I won't pretend otherwise.

But the Horizon that mattered did not begin with a bug. It began with a culture that prized the institution's reputation above the truth, leaders who did not want to be troubled with detail, and a presumption that the machine does not lie. AI touches none of that. If anything, it hands that same culture a more dazzling machine to hide behind, and a better answer to give when someone asks how they could possibly have known.

The next Horizon will not be stopped by a cleverer model. It will be stopped, if it is stopped at all, by someone willing to say "this is wrong" — and by an organisation, for once, willing to listen. We have built remarkable tools for finding the truth. We have never been much good at the part that comes after, where we are supposed to act on it.

Ends

We've opened a secure channel. Please email us the password.

Consumer · Field notes REF / SECURITY — FOLLOW-UP

A follow-up. Of the two banks that asked me to email my identity and income documents, one has now run its full course — a refusal, a threat, a climbdown, an apology, and then, at the very last step, a request to email the one thing that should never be emailed.

What happens when you decline to email your identity documents, cite the regulator, and copy in the data protection officer.

A while ago I wrote about two banks that, in the same stretch of weeks, asked me to send proof of identity and proof of income by email — the one channel they spend a fortune warning me never to trust. This is what happened when I stopped complying with one of them and started pushing back. It has, I’m pleased to report, a sort of happy ending. It also has a punchline neither of us should be proud of.

Where we left off

The secure upload portal they pointed me to returned a dead link. The mobile app I authenticate into with my face offered no way to upload a document. And when I pointed both of these out, the answer came back that email was, regrettably, the only way.

Exhibit — the only solution

“We kindly ask you to send the requested documents by email, as we do not have any other solution to proceed with and complete this case.”

“No other solution” is doing a lot of work there. It describes a gap in their own systems — a portal that doesn’t load, an app that can’t upload — and reaches for the insecure channel as though it were a law of nature rather than a thing they hadn’t finished building.

The refusal

I declined to send a complete identity-and-income bundle as an unencrypted attachment, and proposed the method the national data-protection regulator itself recommends: an encrypted file, with the password shared separately, by telephone. I cited the law — the provision that requires appropriate technical measures, including encryption where appropriate, for personal data of this kind — and the regulator’s own guidance that email is rarely a safe way to move it. I mentioned, for good measure, that the same regulator had already fined a company precisely for asking its customers to email personal data.

The threat

The reply engaged with none of that. Instead it recast my position — a “decision not to provide the required information” — which was simply untrue; I’d offered to provide everything, securely. And it attached a list of consequences.

Exhibit — the consequences

Rejection of certain banking transactions. Inability to access certain services. Blocking of payment methods. And, “possibly, termination of the business relationship.”

Nothing quite concentrates the mind like being told your account is on the line over the question of which envelope you use. I could have folded there, and I suspect most people do — which is rather the point of putting the sentence in.

The climbdown

So I escalated, and copied in the data protection officer. That changed everything. Within a few days: an apology, an explicit acknowledgement of the security concern, a note that the relevant teams had been reminded of good practice, confirmation that the portal fault was being investigated, and — the thing I’d been asking for the whole time — a secure workspace, opened for thirty days, for me to upload the documents into.

This is the part where the system worked. The desk whose actual job is data protection understood the problem in a sentence. The people who’d been demanding the data hadn’t understood it at all.

The one person whose job was data protection got it at once. The people asking for the data never did.

The last step

And then, having built the secure channel, they asked me to send the password to it — by email.

Exhibit — the last step

Upload the documents to the secure workspace; then “send the password separately” by email. And — this is the part that stings — “please note that any text entered in comment or message fields is not encrypted during transmission… do not include the password or any sensitive information.”

Read that twice. They understood channel security well enough to warn me off the unencrypted comment box — and in the very same breath, asked me to put the password in an email. The link to the secure workspace had also arrived by email. So both halves — the way in, and the key that opens it — would end up sitting in the same inbox, one compromise away from being lifted together.

The entire point of sharing a password “separately” is that it travels by a different channel, so that no single breach hands over both the lock and the key. A password by email, when the link came by email too, is not a different channel. It’s the same envelope, posted twice.

The oldest secure channel

By this point I was, as it happened, in the country where the bank lives. So I did the one thing in this entire saga that turns out to have no security holes at all: I put copies of the documents in a folder, walked into a branch, and handed them across the counter against a dated receipt.

No transport encryption to argue about. No attachment left decrypted in anyone’s sent folder. No password to send by any channel, secure or otherwise. Just paper, a human, and a stamp. The oldest method in banking quietly outperformed every digital channel the institution owns — not because those channels are bad ideas, but because not one of them had been finished to the point where it actually worked from end to end.

That’s the whole story, in the end. The security was never missing. It was there in pieces — a portal, an app, a secure workspace, a data protection officer who understood all of it — and it stopped, every single time, one step short of the place I actually needed it.

The security was never missing. It just stopped, every time, one step short of where I needed it.

That’s one of the two closed out. The other bank — the one that assured me its email was “securely encrypted,” then admitted it had no portal at all — is still, at the time of writing, thinking about it. That’s a story for when it ends.

Ends