Modern Software Problems - The Technical and Social History of Software Engineering - page 359

1987: Wall Street Crash
On Monday, October 19, 1987, the Dow Jones average dropped 508 points for the
greatest 1-day loss in history.
The problems are somewhat murky, but apparently the long-running bull mar-
ket had been shaken by various Securities and Exchange Commission (SEC) in-
vestigations and other reasons for loss of confidence. As live human investors
began to sell stocks, programmed trading software that followed patterns began to
generate so many sell orders that various stock-trading systems crashed and mil-
lions of shares were put up for sale, which deepened the panic.
Lessons learned: The key lessons from this problem are that, in today's world,
software controls so many critical financial and government operations that bugs
or errors can have vast consequences and cause problems almost instantly.
Problem avoidance: This problem might have been found through thoughtful in-
spections that included limits analysis. Requirements modeling might also have
found the problem.
Static analysis probably would not have found the problem because it was a
problem of logic and trends rather than a syntactic issue.
Testing might have found the problem but did not. Some of the modern forms
of testing such as risk-based testing might have found this problem.
1990: AT&T Telephone Lines Shutdown
In 1990, a widespread shutdown of AT&T telephone lines lasted for about nine
hours and caused major disruption of telephone traffic. Many airline reservations
could not be made, and millions of calls, including some emergency calls, could
not be connected.
What seemed to have happened is that one of AT&T's 114 switching centers
had a minor mechanical problem (not software) and had shut down briefly. When
this center came back up, it sent a message generated by software to all of the other
centers, which caused all of them to shut down. Apparently, there was a bug in a
single line of code that caused the shutdown.
Lessons learned: The lesson from this failure is that large interconnected systems
governed by software are intrinsically risky and need elaborate buffers and error-
correction protocols.
Problem avoidance: This error could have been caught by either code inspections
or static analysis tools. The error appears to be one of syntax rather than one of lo-