Mr Bates vs The Post Office vs Software Engineering

Like many people who live in the UK, I spent some of the Christmas period watching ITV’s Mr Bates vs The Post Office, which from a computer scientist’s point of view can be seen as a dramatisation of the real world consequences of buggy software. There were many fist-clenching moments, but one that specifically annoyed me as a computer scientist was the assertion that the Post Office’s software couldn’t possibly contain bugs.

This TV programme tells the story of how bugs in the Post Office’s software led to postmasters being unfairly prosecuted for false accounting and theft, with deeply unpleasant consequences. The dramatisation mentions a bug that appeared when multiple terminals in a single post office were used at the same time, causing transactions to get lost. I’m going to focus on this one too, but it’s worth knowing that there were numerous others¹. To me, this sounds like a failure of both requirements engineering and software testing, two important pillars of the software engineering process which are key to delivering reliable software that’s fit for use.

Requirements engineering involves working out how software will be used and consequently what it needs to do. Given that this particular bug appeared when multiple terminals were used, it makes me wonder whether this situation was adequately considered during the requirements engineering phase, or just tacked on at a late stage. Software testing involves testing the software in all situations in which it might be used, with a particular focus on corner cases. Again, if they’d thought about use cases involving multiple terminals, then this bug should have been picked up in testing.

Though, having said that, the great Dutch computer scientist Edsger Dikjstra once said that testing can only prove the presence of bugs, not their absence. That is, even if your testing process is thoughtful and industrious, it can never prove the absence of any bugs in your program, only the absence of bugs that you’ve thought about looking for. The one exception to this is when formal methods are used. These involve writing the program’s desired behaviour as a bunch of maths, and then proving that the maths is correctly refined into code — something that is difficult, time consuming, and unfortunately not to many people’s taste².

So, in essence, we can very rarely say that software is bug free, but we can attempt to engineer our software in a way that limits the likelihood of bugs. In fact, software engineering is there to ensure that software works. Individual programmers and designers will make mistakes, but the software engineering process should prevent those mistakes from making it into the finished product — at least when correctly adhered to. This story shows how software engineering can go wrong, and how awful the consequences can be for people, society and organisations. However, serious failings are fortunately now the exception rather than the norm, and a lot of this is due to improvements in how we do software engineering.

One key improvement is agile development. Back before the turn of the century, it was commonplace for requirements engineering to be done only once, at the start of a project. In practice, requirements almost always change during the course of a software development project, and so these projects often produced software that was not fit for purpose. This explains many of the large failed software projects of the 80s and 90s. But then, in the late 90s and early 00s, agile development started to take off, based on the idea that change should be an integral part of any software development project. Central to this was the involvement of potential users within the development cycle, asking them to try out and give feedback on the developing software at regular intervals.

I don’t know much about the development project which led to the Post Office’s software, but I’m guessing it wasn’t based around agile development³. If it was, then it’s likely that the many bugs would have been detected and ironed out during repeated user testing.

Anyway, to sum up, this incident shows the importance of not only doing software engineering correctly, but also the importance of non-computer scientists being aware of the limitations of computers and their programmers — so that next time someone suggests their software can’t possibly contain bugs, they can whip out their copy of Dijkstra’s best quotes and laugh in their face.

1. See https://clarotesting.wordpress.com/2020/05/27/the-post-office-horizon-it-scandal-part-1-errors-and-accuracy/ for an account of some of these.

2. Though I quite like it. A good example of its use is within the Paris metro, where it was used to prove that driverless trains would always function as expected.

3. But from what I’ve read elsewhere, it sounds like the software engineering process was messy, to say the least.