Computers, Algorithms and Artificial Stupids.

Oh yes, and the inability to admit mistakes...

Way back in 1972 when I was about fifteen, my mum showed me an advert in the local paper, the Huddersfield Examiner, for a two week long introduction to computer programming course at Huddersfield Polytechnic, running in the school holidays.

above: Leeds University in the early 1970s, the Great Computer!

Some background. At this time digital technology did not really exist. I had never seen a computer but had read about them in the science fiction books I devoured (an addiction...) and an occasional article in the New Scientist my Dad got (an English lecturer but with an interest in science) and of course, saw in the memorable film 2001 when Hal¹, described as a computer rather than an AI, goes psychotic (and incidentally, is the most interesting character in the movie).

I signed up for the course, the only requirements being to turn up armed with a pencil that had a rubber eraser on the blunt end. I did hope that I might actually get to see a computer, but no. The course leader told us, rather proudly, that there were currently three computers in Yorkshire, one at the DVLA (Department of Vehicle Licensing Agency) in Pontefract, another had been bought by a wealthy woollen mill owner to run the staff payroll and the third was at Leeds University. It was this machine that Huddersfield Polytechnic had time on, between 2am and 4am each night.

above: HAL from 2001, the psychotic AI. More of these to come?

So we learned some machine code, assembly language and a little Basic, the latter being a programming language whose arguments looked vaguely like logic rather than obscure code. The programs we were asked to write were simple, such as, input three numbers and decide if they formed a triangle or not and if they did, was it a right angle triangle, an isosceles, an equilateral or just an ordinary triangle.

More to the point, we learned how to break a task down into the most basic steps possible and then translate those steps into statements a computer could process. We also learned the acronym GIGO, or Garbage In, Garbage Out, meaning if your code was rubbish, so too would be the output.

We'd hand in our bits of paper at the end of the afternoon. A night shift at the Poly would then transfer our code to paper tape with holes punched in it, then shine a laser through the tape, translate the flashes into an electrical signal and send it down the phone line to Leeds University where it would be turned back into code (basically a modem or modulator/demodulator) and finally fed into the Great Computer.

In the morning I'd come back in and collect the output, usually something like “ERROR at line 35”. I'd recheck my code, note that at line 35 I'd mistakenly used a comma rather than a semicolon, use my pencil eraser, make the correction and hand the paper back in at the end of that afternoon. Next morning it might have got as far as “ERROR at line 55” or something similar, so repeat until it works; that is, a feedback loop of 48 hours in all. Some poor folks never got a single program working during the two weeks. For some reason by about day three it all clicked and I whizzed through all the tutor's challenges and he had to keep coming up with tougher ones to keep me occupied.

above: simple basic program from Wikipedia

Ten years later, 1982, in Dolgellau, chatting to a new friend, Wolfy, computers got mentioned and he said he'd got one- I nearly fell off my chair in surprise! Well, a mate of his had lent him a ZX81². I could hardly believe it- In just ten years going form only three computers in Yorkshire to here's one in my hand...and what's more, it ran a version of basic and in a few minutes I had it displaying the classic “Hello World”. When Wolfy showed me the Basic code book that came with it, I soon had it displaying “Hello World” ten times in different colours (including flashing colours...) and with varying sound effects. Wolfy was as surprised as I was.

Note here that with the ZX81 I would type in the Basic commands then run the program, whereupon it would say something like “ERROR at line 45”. I'd check the code, make the adjustments (along the same lines of a colon instead of a full stop), run the code again and repeat until it worked. So the feedback loop was now seconds rather than days. What a difference!³

Why am I on about this? Well, the ITV series Mr Bates Vs. The Post Office has finally brought the disastrous, flawed Horizon IT software out into the open, though as Radio Four listeners, we had already been aware of this awful miscarriage of justice for over a decade after Nick Wallis damning radio series of 2011⁴.

One of the key difficulties in bringing this disaster to light was the absolute insistence by management that the Horizon software was infallible, that it could not be blamed for any errors, therefore the sub-postmasters, over 700 of them, must have been dipping into the till.

This fantasy that computers can't make mistakes has, hopefully, now been dispelled but it does not stop errors from occurring. Any programmer will tell you that all complex software, like computer operating systems, aircraft guidance systems, Horizon etc. etc. will contain bugs, that is, coding mistakes that might only show up in use or in very particular situations.

Microsoft, for example, release new versions of Windows, followed by regular updates to fix bits that are only found to be broken when in use. In may ways, what used to be called beta-testing, when software was issued to a limited number of specialist users who then fed back to the manufacturer any bugs they found, happens now in the wild, with us as the beta-testers. This might be all fine and dandy for games or similar types of software but mission critical stuff, where lives might depend upon the software functioning perfectly is another matter.

Boeing's 737 MAX flight control system, for example, led to two plane crashes that killed 346 people. The software, called Manoeuvring Characteristics Augmentation System (MCAS), was added to compensate for the larger, more powerful engines that had been added onto the existing 737 airframes, which changed the in-air flight characteristics of the aircraft.

above: one of the Boeing 737 MAX’s crash sites. As with Horizon, at first the pilots were blamed rather than the software.

One consequence of the new software was to force the aircraft into a nose dive given certain conditions. The flight data recorder from one of the aircraft, Lion Air 610, for example, revealed that the plane had gone out of control and had moved up and down over 24 times, a result of the pilots desperately and repeatedly trying to pull the plane up out of a dive, before it finally dove into the sea at full speed. Investigators were surprised to learn that Boeing had installed the flight control software program that could force the plane into a dive without the pilots’ knowledge.

Horizon and MAX are just two examples of a long list of terrible programming failures. An old friend of ours, Elsa, learned programming in the 1980s, proved very good at it and became involved in a decade long venture to integrate all NHS patient records digitally and make them available at any hospital or surgery. Sounds simple, useful and practical. Not! This vastly expensive software program never saw the light of day and was eventually abandoned after throwing millions of pounds at it. I remember Elsa telling us how it wasn't going to work but she was still kept on for years before the truth finally came out.

Back in Huddersfield Polytechnic writing a program with maybe thirty or so lines of code to decide whether something was a triangle of not and if so, what type, is a relatively easy task to check for bugs, even when the feedback loop was a couple of days. The fact is that today's software can run to millions of lines of code with thousands of different code modules that interact with each other. These cannot be checked completely by human beings and may run for days, months or even years before a particular combination of circumstances throws up a problem. In some ways, the safest bug is one that just crashes the system. Its the ones that don't do anything immediately obvious that present the greatest risk.

Another example is a world famous software company's spreadsheet, which millions of folk use. For most activities it adds everything up just fine but start throwing large amounts of data at it and errors begin to appear. This is a known fault and currently unfixable.

It's said that getting the AI's working on producing software will mean they will be able to create perfect, faultless code. But then, the AIs are “just” computer programs made up of millions of lines of code....uh oh.

I had a brush with one the other day....living in a place where you can't get a TV signal and with an internet speed that until recently made it impossible to stream anything, Lyn and I have not watched any form of telly for over thirty years and haven't missed it. We get regularly hassled by the Beeb TV licensing folk who can't really get their heads around the idea that we're not interested. This used to mean filling in and returning a paper form every two years, declaring that we don't watch any broadcasts. This time round I chose the option to phone an automated service.

The AI at the other end used a woman's voice and referred to itself as “I”, along the lines of “Can you tell me the reason you are phoning me today?”. All went well until it (not “she”!) asked me for my mobile phone number so it (not “she”!) could send me a text confirmation. Well, until our new Emergency Services Network mast goes live (and there's no sign of that happening anytime soon) we live in a mobile not-spot but could I explain that to the computer program? Not a chance. In the end it (not she!) hung up on me! Aaargh!

Fujitsu are the company behind Horizon and are currently embedded in various Government contracts, as is Horizon, despite repeated cock-ups. In 1999, for example, the firm won a £184m contract to develop Libra - a software meant to standardise case management transactions across more than 300 magistrates' courts. In the end, it cost nearly three times more than expected, and the National Audit Office concluded that it was not able to produce even basic financial information.

above: DPD’s AI Chatbot attempts a Haiku. Its not even good at Haikus…

So what's the message here? GIGO. Computer software that is perfect does not exist and may never appear. This includes AI's. For a laugh at the more harmless results, have a look at this BBC news story about the courier company DPD's attempt to add AI to its chatbot. Users found that apart from the AI chat bot being useless at solving users' problems, it was quite easy to get it to misbehave, such as writing poor Haiku's about how crap DPD were and getting it to swear...

Thanks for reading. Next up is a look at the UK and Welsh Governments’ generous grant schemes for heating peoples homes with Air Source Heat Pumps, which has got to be a good move, surely! Um, better look a bit closer before you decide….

Then we’ll be back to Konsk- sorry to leave loyal readers on a cliff hanger. ‘Fraid there’s more of those to come! Hwyl!

1. You may already know that Arthur C. Clarke, who wrote the book and the screenplay, chose the acronym HAL for the mad computer in order to be one step (or letter) ahead of IBM…

2. The ZX81 was produced by Sinclair Research, run by Clive Sinclair and was the first home computer to come in at under £100. It had no screen and plugged into a television set. It could generate eight colours and eight flashing colours, had sound but only 1k of memory- that is, the memory could hold 1,000 characters, so programs were necessarily short and compact, to say the least. The only was to save a program was onto casette tape which was notoriously unreliable. But it was a computer you could hold in your hand and about as powerful as the one that got Apollo to the moon and back in 1969. How things have changed…

3. There is an important lesson here. Whatever we're doing, we should try to keep the feedback loops as short as possible- that is how we can make rapid progress. David Holmgren, when he was teaching at our place, pointed out that you should never just give instructions to a machine operator (say digging a swale) and then leave them to it because you'll come back and find they've not quite done what you wanted. You should rather sit in the cab with them, giving them immediate feedback on what they are doing until you are satisfied that they have got it.

Otherwise you come back to check and wow, a lot can go wrong in an hour on a big machine! With gardening, we usually have to use an annual feedback loop. So we try something new in the spring, find out it doesn't quite work by the summer but then have to wait till next spring to make the adjustments. A good reason for a beginner to work closely with someone who has been round the loop many times!

4. Nick Wallis continues his excellent in depth account of the scandal, bringing out yet more damning evidence of the conspiracy that is not covered in the ITV drama. It can be found here.