Back to...

GET VISIBLE! Advertise Here. Find Out More





Share Our Stories! - Click Here

Software Curse On Programmers Few Know About

By Ted Twietmeyer
3-20-19

I have been increasingly concerned about how software engineering is now blindly implemented. My career background includes being a programmer, engineer and project engineer/manager for many decades in industry, defense and aerospace. It gave me a broad base of inside knowledge.

The software curse is now causing problems with everything from vehicles to aircraft.

In the mid 1970s the first microprocessor chip was sold by Intel. Here is a brief history of control systems used in everything throughout our society, from vehicles to traffic signal controllers and railroad crossings to factory equipment and commercial/military aircraft, tanks, missiles etc...:

Here is a brief history to understand the software curse:

At first, simple 4 bit microprocessor chips ("micro") were used in various products to replace numerous circuit boards full of discrete electronic components. General Railway and Signal (GRS) was among the first large scale users of micros. These systems were originally programmed in machine code. GRS manufactures signaling systems used on railroads and controls most the traffic lights you see at intersections today around North America.

GRS has had fault detection in traffic signals for many decades. This detects any unsafe fault in the control system such as an attempt by circuitry to turn on all green lights together. Safety system will then immediately change a traffic signal into a blinking red and yellow light when a fault of any kind is detected until a technician comes out to fix the problem.

· In the early 1980s the first micro development systems were implemented to replace tedious hand coding. These systems directly created code in machine language. To write code, programmers were required to intimately know the architecture inside the micro used in the product. Programmers had easy access to all data/hardware that application software controlled.

· In the mid 1980s, the first high level language came out for micro systems by third party companies. It was known as C and is still used today. Now it has evolved into C++, C# ("C sharp") and other language variants. C and various high level languages require using development software known as a compiler. A compiler runs on PCs and other computers. It is used to convert a specific high level language to machine code which a micro can run.

At first, high level languages like C required the programmer to create variables, strings and arrays to store user program data. Everything was transparent. Then object-oriented high level languages came along like C++, C# and Java. High level language compilers of all types are used by millions of programmers today, and are among the first artificial intelligence programs ever created. Software that controls airbags, active stability systems, anti-skid brakes and many other safety-critical systems we use in our everyday lives depends on bug-free machine code. As someone who has used numerous compilers in my career, I can testify that compilers are NOT free of bugs.

THE CURSE

Here is how the curse works: When a programmer needs to store data using object-oriented programming, most compilers store the programmer's data somewhere in memory. The programmer has no idea where the data resides, and must implicitly trust the compiler. A programmer must access the data as objects. This is similar to backing up your computer's data to a cloud, where you have no idea at all where it physically resides but trust it will be safe. Any programmer must blindly trust the compiler that it will create FLAWLESS machine code (huh?) to read/write data, allowing the compiler to decide where it wants to store it in memory.

Now software bugs have made world news which can crash planes and other vehicles, making pilots and drivers helpless to take over control and prevent crashes. Are data storage errors occurring which are caused by defective compiled code? Following are examples of software bugs that have crashed planes.

DC9 AIRCRAFT IN SWEDEN

When commercial aircraft crash, government authorities investigate the cause of the crash and may require a manufacture to create safety upgrades. Outside of pilot error and training issues, aircraft upgrades can be hardware modifications, procedure changes, software changes etc...

Some years ago, the FAA identified a problem with DC9 aircraft. Some pilots would not use sufficient engine thrust during takeoff causing the aircraft to stall and crash. To prevent this, a software change was implemented. Known as the ATR (Automatic Thrust Restoration,) during takeoff this system will override the pilot's engine thrust settings whenever insufficient thrust was detected by the ATR software.

But a serious conflict arose with DC9 Flight 571 in Sweden on a winter day. This aircraft had a working ATR system. Unlike other commercial aircraft which have engines position under the wings, the DC9 had the engines mounted high up on each side of the rear vertical stabilizer. Investigators discovered on that winter day when it was -20C outside, the wings were not properly de-iced before takeoff. Right after takeoff, wings bend as they take on the full weight of the aircraft. This caused large chunks of ice to fly off the wings. Whenever this happens, pilots are trained to reduce power until the problem clears up. Pilot did exactly as he was trained to do. Continuing to keep engines at full power will destroy them.

Pilot reduced power, but after removing his hand from the throttle levers the ATR system forced engine power back up to 100%. Ice from the wings entered both engines at 100% thrust destroying them. The skillful pilot landed the aircraft with no engine power on a frozen lake. Despite the crash breaking the fuselage into three segments, no serious fires occurred and all passengers survived.

What happened? No one took the time to evaluate the effect the ATR would have under ALL possible operating conditions for the DC9 aircraft. Investigation revealed the flight crew was never told by the airline the ATR "upgrade" was installed, and that it could take control of engine throttles.

Boeing 737 MAX 8 SOFTWARE BUG THAT KILLS

The Maneuvering Characteristics Augmentation System, software known as MCAS, was included on the Max 8 model aircraft as a safety mechanism that would automatically correct a plane entering a stall condition. If the plane loses lift under its wings during takeoff and the nose begins to point upward, the system kicks in, over-rides the pilot and automatically pushes the nose of the plane down.

After the Lion Air crash, the FAA's issued an airworthiness directive that said: "This condition, if not addressed, could cause the flight crew to have difficulty controlling the airplane, and lead to excessive nose-down attitude, significant altitude loss, and possible impact with terrain."

Officials did not immediately determine what caused Ethiopian Airlines 302 to nosedive into the ground, but many experts have noted similarities between this week's crash and the one in Indonesia.

This and other software issues can be seen at https://www.philly.com/ business/boeing-max-pilots- complained-suspected-safety- flaw-20190312.html

SUMMARY

It's past time for programmers writing any software to stop blindly trusting compiler-created machine code without performing extremely thorough testing under all conditions.

Two kinds of software bugs can crash planes. Either one can create defective machine code for computers:

1. COMPILER BUGS - Few people know about these. Imperfect machine code can be created by the compiler itself which is NOT bug free. Some compiler manufacturers do not perform thorough testing. Implications of defective compiler-created code are enormous.

2. APPLICATION BUGS - Imperfect machine code created by application programmers, who blindly trust compilers to generate correct code for safety-critical systems. Programmer code must absolutely compile and run properly without error under ALL conditions.

Defective code whether caused by 1, 2 or both described above has already created related fatalities across many other industries such as cars, trucks, trains etc... During my career, I have found many programmer errors compilers cannot detect. When this happens defective code is generated and most anything can happen.

There is an arrogance among some compiler manufacturers who steadfastly REFUSE to take blame. I have personally encountered it. When informing the compiler manufacturer about a bug, I was told point blank "There is nothing wrong with the generated code. It's your software that's wrong." For a long time that company refused to believe me and replicate the simple software test I gave them over the phone. Finally they were forced to realize I was right and quietly issued an update.

Are Boeing and other manufacturers fighting a similar compiler issue, application program bug created by their programmers, a bug created by instrumentation electronics programmers, etc...?

Time will tell.

Ted Twietmeyer