Monday 15 January 2024

Horizon

I’ve been following the Post Office Horizon scandal as I imagine you have too. The victims have suffered unimaginable stress, some over twenty years or more. I’m not sure I would have survived it as most of them have. Sadly, as we know some did not. I believe four postmasters or mistresses took their own lives and it isn’t hard to see why. It is indeed the “greatest miscarriage of justice in British legal history.” But apart from the appalling scandal of lives wrecked, there is another angle that has been quite understandably overshadowed by the human tragedy of it all.

It’s also a tale of British industry. The computer company at the heart of it was Japanese-owned but I am pretty sure the staff are virtually all British. Fujitsu simply acquired ICL, a company started by Harold Wilson in 1968.

ICL started to use Fujitsu technology in the 1980s and the Japanese company (itself started by the Japanese government money by the way) in 1990 bought 80% of ICL and became the sole shareholder in 1998.

Horizon was a failed project twice. It began in 1996 under John Major as a contract to modernise payments made through the Post Office by the Benefits Agency. That went terribly wrong and was abandoned in 1999 when the DWP got fed up with all the delays. ICL wrote off £180 million and the Post Office £571 million. The reputational damage to the ICL brand was so bad it was renamed Fujitsu Services in April 2002, and later to just Fujitsu.

Horizon was then salvaged and re-purposed for the Post Office as a general-purpose system to automate the nation's 21,000 post offices. This was like trying to resurrect the Sinclair C5.

So, it's no use blaming the Japanese, it was a very British failure and I think the whole saga tells us something unique about us as a country. The way we're unable to manage big projects (HST is the latest example) or even recognise failure early on.

I read several accounts about software engineers arriving at Fujitsu in the early 2000s to work on Horizon and saying "Everyone knew it was a bag of s**t."  Everyone that is except the people at the top. Or perhaps they too were aware that it was going to be a failure but hoping to take the money and the bonuses and get out before it all went south.

The Horizon system contained thousands of errors with 5,000 entries in the Known Error Logs (KELs) and 218,000 PEAKS  - records of software issues kept by Fujitsu.

In short, it's clear there were plenty of staff who knew it was all rubbish but nobody went public and I bet they didn't shout very loud inside the business either - would you if your mortgage and children's future depended on your job? We are far too reserved when it comes to that sort of thing, aren't we?

I’m not an expert on computers by any means but I worked for a company that provided process control systems and which started to use microprocessors as early as 1973. We employed software engineers and I worked internally in sales support and later on in external sales.

We had some decent products that used Assembly code - a very low-level computer language used for products that would be sold in the 100s or thousands, and we also had some dogs. We also had products that used high-level languages like Fortran and had code developed for every single job. Invariably, our in-house programmers made mistakes, usually a lot and sometimes serious ones.

We did use one external software supplier who was excellent and as far as I recall never provided software with errors and which never crashed. He was the exception rather than the rule.

The Horizon system is not that unusual in Britain, is it? Government IT systems invariably cost a fortune, far more than originally estimated, and often don’t work. But don’t run away with the idea that it’s a problem confined to the government because it isn’t. Private industry suffers from the same problems.

As I understand it, the repurposing of Horizon to replace the paper functions in Post Offices was started without a proper design specification and was essentially cobbled together. The prototype, developed to show how the system might work, went on to become the actual system and was added to over a period of years.

In short, it appears the whole system was ill-conceived and poorly executed with bad decisions being made by irrational individuals who weren’t qualified to make them. The worst was to launch it before it was anything like reliable. After about ten years in 2010 the Legacy system was rejigged and relaunched as Horizon Online which was an improvement but it still had errors and bugs.

The Legacy version used a piece of messaging software called Riposte, which some people have suggested was the wrong thing to do. The Online version from 2010 dropped Riposte altogether.

To give you a glimpse into the problem, in the 2019 judgment, Mr Justice Fraser places some weight on the evidence of Torstein Godeseth, who admits to having been the designer of Horizon.  He has a physics degree, worked for Rolls Royce before joining the Royal Navy in 1977, and started working on IT systems. He joined the PO in 1987 and worked on technology to automate the branches.

He was a technical advisor when the Post Office and Benefits Agency first bought the Horizon system and was involved in the whole thing for years, before joining Fujitsu in 2010 when the Online version of Horizon was launched. 

He describes himself as “having a pretty good knowledge of Riposte”.

But listen to this. It wasn't until the legal case in 2019 that Mr Godeseth realised that Fujitsu could remotely access branch computers and insert transactions without the SPM's knowledge: See paragraphs 317 onward:

 "Mr Godeseth only found out the true position when Mr Parker was preparing his subsequent witness statement in the weeks prior to the commencement of the Horizon Issues trial, in other words in 2019. He had not known that before. His explanation about this was as follows.

“Q. You were finding out a detail that you didn't know before in quite a controversial area, weren't you?

A. It was clearly an area that was going to be of interest because of the fact that we were inserting transactions into Riposte."

This is quite extraordinary to me and not just because it goes to the heart of the problem. This would simply never happen in the European companies I worked for. Their engineers knew every detail of their particular field whether it was electrical, software or mechanical/pneumatic and they took pride in knowing it.

The point I am making is that failures on the level of Horizon seem to me to be something peculiar to Britain. I think other countries have probably made similar mistakes, choosing the fundamentally wrong system for example but I don’t believe it would have ever seen the light of day.

If it had, the system would have been scrapped very quickly. Our problem is the inability to speak out and call a spade a spade. I don't believe it could have happened anywhere else in the world except Britain and that should worry us.