Recently, historians of computing have written a lot about the so-called
“software crisis,” first articulated in the 1960s, and the emergence of software
engineering, structured programming, and formal verification of program correctness as possible solutions to the crisis (Abbate 2012; Ensmenger and Aspray 2002;
Ensmenger 2010; Mahoney 2004; Slayton 2013a; Tomayko 2002). By 1968, a number of different problems seemed to converge, creating a sense in the computer industry that software was becoming a problem. For one, software costs were outstripping hardware costs; a widely plagiarized graph projected software to make up 80% of computer costs by 1978 (Slayton 2013a, 155–7). Moore’s Law would seem to exacerbate this, with hardware improving by orders of magnitude over software. Secondly, a number of large, complex software projects had failed, most famously that of IBM’s OS/360 effort, but also software on NASA’s Mariner I spacecraft (Abbate 2012, 92) and MIT’s timesharing MULTICS operating system, which inspired Unix at Bell Labs. (Slayton 2013a, 111–2) Thirdly, demand seemed to be causing a shortage of programmers and fourth, the programmers that were available seemed to be unmanageable, as programming had the reputation of a “black art” and a craft skill, rather than a quantifiable, scientific or engineering discipline (Abbate 2012; Ensmenger 2010).
These latter two were related. As Ensmenger notes, the labor shortage was not
“an absolute shortage of programmers but rather a shortage of a particular kind of programmer. [emphasis in original]” (Ensmenger 2010, 18) “An early study at IBM suggested that exceptional programmers were ten times more efficient than their merely average colleagues. The alleged 10:1 performance ratio quickly became firmly embedded in the cultural wisdom of the industry.” (Ensmenger and Aspray 2002, 6) Another IBM study declared “that a good programmer was at least twenty-five times more efficient than his or her merely average colleague. Whether the exact ratio of performance was precisely twenty-five to one (or a hundred to one—another commonly quoted figure) did not much matter. What did matter is that whatever its
111
deficiencies, this study and others seemed to confirm plentiful anecdotal evidence that good programmers appeared to have been ‘born, not made.’” (Ensmenger 2010, 19) Fred Brooks, who had led the failed OS/360 effort and famously wrote The Mythical Man-Month to examine the software crisis and catalog possible solutions, cited a similar study: “Within just this group [of experienced programmers] the ratios between best and worst performances averaged about 10:1 on productivity
measurements and an amazing 5:1 on program speed and space measurements! In short the $20,000/year programmer may well be 10 times as productive as the
$10,000/year one.” (F. P. Brooks 1995, 30) Clearly, there appeared to be an order of magnitude difference in programmer skill and productivity.
A number of different solutions were proposed over the years to address the software crisis. Some were explicitly managerial solutions. Brooks expanded on a top-down proposal by IBM’s Harlan Mills, known variously as the Chief
Programmer Team, Surgical Team, or Superprogrammer approach. As the title of Brooks’ book indicates, Brooks argued that throwing more manpower onto a
software project did not speed it up but rather slowed it down, as the organization got more complex. Programmer teams should be kept small, with 10 people or less.
Moreover, because of the order of magnitude difference in programmer productivity, all of the high level design decisions should be done by the chief programmer a.k.a.
head surgeon, with subordinate implementation tasks delegated to the supporting programming staff. Brooks argued that this top-down “aristocratic” division was necessary to maintain the conceptual integrity of the program’s architecture. It did not deskill the work of the supporting staff, as implementation still required creativity and skill, though not as much as architecture. “The opportunity to be creative and inventive in implementation is not significantly diminished by working within a given external specification, and the order of creativity may even be
enhanced by that discipline.” (F. P. Brooks 1995, 48)
Another proposed solution, which came to be known as “structured
programming” came from Dutch computer scientist Edsgar Dijkstra. “The main tools of structured programming were abstraction, modularity, and the use of conditional
112
loops (rather than “go to” statements) to model the logical structure of the process that was being automated.” (Abbate 2012, 99) Dijkstra published a letter to the editor of Communications of the ACM in 1968 proclaiming “Go To Statement Considered Harmful.” (Dijkstra 1968) Dijkstra’s point was that the Go To statement, a command that allowed a program to arbitrarily jump around to any line of code, created a spaghetti-like maze that would be impossible for other programmers to follow. By restricting programmers to using only conditional branches, loops, and subroutine calls, programs would be much easier for human readers to follow. Structured programming also incorporated programming practices that called for making software more abstract (less machine-dependent), modular and reusable. More importantly for Dijkstra, however, is that these techniques would also make programs more amenable to formal, mathematical proof of correctness. Dijkstra promoted the formal verification of program correctness as one way to combat the reliability problems of software that had led to the sense of crisis (MacKenzie 2001, 37–61).
Both the chief programmer team approach and structured programming came to be seen under the larger umbrella of “software engineering,” a term that began to pick up steam for its association with “systematic, disciplined, quantifiable”
techniques. (Institute of Electrical and Electronics Engineers (IEEE), IEEE Standard Glossary of Software Engineering Terminology: IEEE Standard 610.12-1990 (New York: IEEE, 1990), 67, quoted in Abbate, 2012, 99). 1968 saw the first NATO Conference on Software Engineering at Garmisch, Switzerland. Douglas Mclroy, a member of Bell Labs’ Multics team at the NATO conference explicitly linked software engineering to mechanical engineering and the industrial revolution:
We undoubtedly produce software by backward techniques. We undoubtedly get the short end of the stick in confrontations with hardware people because they are the industrialists and we are the crofters. Software production today appears in the scale of
industrialization somewhere below the more backward construction industries. I think its proper place is considerably higher, and would like to investigate the prospects for mass-production techniques in software. (M.D. McIlroy, “Mass Produced Software Components,” in Naur and Randell, pp. 138-150. Quoted in (Mahoney 2004, 11))
113
Much recent discussion of software engineering by historians sees in this industrial rhetoric a managerial drive to discipline and deskill unruly programmers.
For example, Rebecca Slayton cites Department of Defense managers as having significant influence on driving the development and adoption of software engineering techniques. “While anticipating the rising costs of software, [DoD managers] also confronted shrinking budgets, perpetual scrutiny, and growing demands for flexibility, interoperability, and security. In this environment, defense department managers began to embrace a particular vision of software engineering, a means of quantifying software development and ensuring accountability.” (Slayton 2013a, 160) Under this influence, software engineering reemerged in the 1970s under a new community focused on the reduction of complexity through programming and design methodologies (Slayton 2013a, 162–4). Nathan Ensmenger, likewise, saw in much of the manufacturing rhetoric a call for Taylorist scientific management of programmers:
The solutions to the “software crisis” that [are] most frequently
recommended… are not fundamentally different from the four principles of scientific management espoused by Frederick Taylor in an earlier era.” (Ensmenger and
Aspray, 2002, 15-16)
Other historians, however, see software engineering as a rhetorical strategy used by programmers themselves to increase their status as a profession. “For the major organizations that were trying to define computing as a profession, embracing the term software engineering [emphasis in original] could signal a claim to
professional turf rather than any commitment to engineering as such.” (Abbate 2012, 102) Abbate agrees with Ensmenger that this process of professionalization involved defining it as masculine, and relabeling programming as “software engineering” did precisely this. Nevertheless, Abbate notes that most of the proposed “engineering”
solutions came from those who had deep knowledge of programming themselves, trying to improve programming practice from within. “But not everyone had aimed to deskill programmers in the first place. Software engineering advocates such as Dijkstra and Brooks identified as programmers themselves and had no desire to
114
downgrade their peers. Many programmers actively took up techniques such as structured programming as a way of easing their work and enhancing their own value in the job market. Rather than making programmers obsolete, software engineering methods became simply another skill that programmers could claim.” (Abbate 2012, 108) Thomas Haigh goes further, making the case that the circle of people who directed the software engineering agenda at the NATO Conferences were not managers but the pillars of academic computer science, especially Edsgar Dijkstra.
Dijkstra thought that the software crisis was really the fault of managers who thought programming was easy and hired hordes of programmers with low skill. Dijkstra sought to elevate programming as a formal, mathematical discipline that required intelligence (Haigh 2010).
Within this environment, programming languages emerged as one possible solution to the software crisis. Dijkstra’s structured programming methodology promoted a number of techniques that could be applied, with self-discipline, to existing programming languages. Such practices included maintaining modularity of code. “Essentially, Dijkstra wanted programs to be made up of blocks of code, where each block did only one task.” (Zepcevski 2012, 185) Nevertheless, soon afterwards, computer researchers began to devise programming languages that would enforce structured programming principles, by, for example, omitting the “Go To” statement.
According to Thomas Haigh, the same circle of computer scientists who had participated in the NATO Conferences had previously been working on ALGOL, a language that many had hoped to promote good programming practice. This circle had dissented from the ultimate direction ALGOL had taken, but one among them, Niklaus Wirth, turned his passed-over ALGOL proposal into Pascal (Haigh 2010).
Pascal was a language that would help produce programs amenable to formal proof (Zepcevski 2012, 261–2). The Department of Defense sponsored the development of Ada in order to standardize on a single language throughout the military (Slayton 2013a, 164–5; Zepcevski 2012, 262–3). Both Pascal and Ada were procedural languages developed to foster structured programming practices.
115
Object-oriented programming languages likewise enforced some of these same practices. Joline Zepcevski, in her recent dissertation, argues that
object-oriented programming and structured programming arose in parallel. Modularity had long been practiced before either structured or object-oriented programming.
“Segmentation, or creating modules of code, is one of the most frequently cited solutions to the problem of complexity. When working in teams, if programs are segmented or modularized, programmers can work on individual segments, with little interaction between segments. Modularity has long been perceived as a method of decreasing complexity and improving programming as an industrial activity. As early as the work being conducted on the SAGE project, techniques were being used to create modular code.” (Zepcevski 2012, 199) Nevertheless, object-oriented
programming made modularization of code mandatory for all programs. Objects are essentially modules of code that are completely separate from other objects. They can only exchange data with other objects by sending messages to them. Otherwise, the data contained in an individual object is hidden, black-boxed, or “encapsulated,”
in a way that prevents other code from accidentally or maliciously changing them, which enhances the reliability and security of programs. Zepcevski argues that object-oriented programming languages such as Smalltalk and C++ developed as a confluence of ideas from various sources; ideas pervasive in computer science became incorporated into these languages and eventually became part of the
definition of object-orientation itself (Zepcevski 2012, 137–141). Alan Kay himself recounted that Smalltalk was influenced not only by Simula-67, the only previous object-oriented language, but also the functional language LISP, and ideas from Ivan Sutherland and Seymour Papert (Kay 1993; Zepcevski 2012, 237).
As a result, looking at such languages only as agglomerations of their components, structured programming advocates could claim that object-oriented programming was nothing new, just another language enforcing structured programming methodologies. “Donald Knuth, author of The Art of Computer Programming, has insisted that he has always thought of programming using the techniques that have been subsumed into the object oriented paradigm… ‘but I
116
haven’t used the languages that help enforce the discipline; I’ve always enforced the discipline myself in other languages.’” (Dan Doernberg, Interview with Donald Knuth for the Computer Literacy Bookshop, December 1993,
http://tex.loria.fr/litte/knuth-interview, quoted in Zepcevski, 2012, p. 272) Zepcevski argues that, for older computer scientists as Niklaus Wirth and Donald Knuth, object-orientation did not add any new concepts but merely combined existing concepts and techniques into a single language that enforced those practices. However, other computer scientists such as Antony Hoare saw, correctly in Zepcevski’s view, that object-orientation was in fact a paradigm shift in seeing programs not as processes but as collections of objects (Zepcevski 2012, 274–5).
Nevertheless, Zepcevski notes that Alan Kay created Smalltalk to directly address the issue of software complexity, which, along with the problem of program verification, was the key issue in the software crisis. This occurred in parallel with, and despite, advances made by structured programming.
[The examples of the structured programming languages Pascal and Ada] illustrate that, while language designers enforced structured programming techniques in their languages, the problems of complexity and verification were still an important part of the discourse on programming theory… Constant discussion of the software crisis abounded… The object-oriented methodology was created in response to problems of complexity… Alan Kay often talks of a bet where he argued he could create “the most powerful
programming language in the world” in a single page of code. The outcome of this bet was the creation of Smalltalk. The inference here is that other languages may have also been powerful, but the
complexity of their long definitions was detrimental. (Zepcevski 2012, 263–5)
The enforcement of programmer discipline mechanically through language design was thus one solution to the software crisis that promised order of magnitude gains in programmer productivity. Fred Brooks notes at one point in The Mythical Man-Month, “Programming productivity may be increased as much as five times when a suitable high-level language is used.” (F. P. Brooks 1995, 94) What Brooks meant, however, was that high-level languages could produce this gain when compared to programming in assembly languages, as it frees the programmer from
117
having to deal with the complexity of the low-level hardware, which is hidden by higher level abstractions. “To the extent that the high-level language embodies the constructs wanted in the abstract program and avoids all lower ones, it eliminates a whole level of complexity that was never inherent in the program at all.” (F. P.
Brooks 1995, 186) This argument is commonly extended to object-oriented
languages, which states that because object-oriented languages work at a higher level of abstraction than procedural languages, they will inevitably reduce complexity and improve productivity. This is often connected to a narrative of linear progress—the more abstract and higher-level a language is, the more advanced it is. Computer scientist Tony Hoare apparently thought of object-orientation in this way: “Hoare sees the change in programming methodologies as paralleling his own scientific progress moving from sequential code with hierarchical construction towards an object oriented worldview…” (Zepcevski 2012, 275)
By the 1980s, Brooks himself, however, was skeptical that object-oriented programming, or indeed anything, was truly a silver bullet that could solve the software crisis. In “No Silver Bullet—Essence and Accident in Software
Engineering,” first published in 1987, he argued that software development was inherently hard because it had certain essential characteristics: complexity,
conformity, changeability, and invisibility or unvisualizability (because it involves abstract concepts which are difficult to visualize). Following Aristotle, he
categorized all other programming difficulties as “accidental,” or “those difficulties that today attend its production but that are not inherent.” (F. P. Brooks 1995, 182) Addressing both structured languages like Ada and object-oriented languages, he notes that, like all other high-level languages, they have produced incremental improvements through removing one or more of the accidental difficulties of programming, but that continuing advances in high-level languages will only
produce diminishing returns, as the largest win had been gained from the initial move away from assembly in the first place (F. P. Brooks 1995, 188–9). He is also
skeptical of attempts to realize graphical programming, of which NeXT’s Interface
118
Builder is an early example that came out after Brooks’ article (F. P. Brooks 1995, 194).
Brad Cox disagreed. In a direct rejoinder to Brooks, Cox wrote an article in BYTE magazine in 1990 claiming that a silver bullet indeed existed, and further insisted that this solution could bring about a Software Industrial Revolution, echoing McIlroy’s comment over twenty years prior (Cox 1990a). There were two components to this solution. The first was object-oriented programming, which required programs to be written in modular, even piece-meal fashion. Second was an economic arrangement that this technology would make possible: an open market of software objects that could be bought and sold off-the-shelf, and used by developers to build their own software without rewriting everything from scratch. Reusable objects would be like interchangeable parts, which made possible the first Industrial Revolution.
I use a separate term—software industrial revolution—to mean what
“object-oriented” has always meant to me: transforming programming from a solitary cut-to-fit craft into an organizational enterprise like manufacturing. This means letting consumers at every level of an organization solve their own software problems just as home owners solve plumbing problems: by assembling their own solutions from a robust commercial market in off-the-shelf subcomponents, which are in turn supplied by multiple lower level echelons of producers.” (Cox 1990a, 212)
Indeed, Brooks had envisioned a similar economic arrangement where coding could be outsourced to the market, although he did not couple it with the
technological solution of object-oriented programming:
Buy versus build. The most radical possible solution for constructing software is not to construct it at all… While we software engineers have labored on production methodology, the personal computer revolution has created not one, but many, mass markets for software…
Even software tools and environments can be bought off-the-shelf. I have elsewhere proposed a marketplace for individual modules.
Any such product is cheaper to buy than to build afresh… The
development of the mass market is, I believe, the most profound long-run trend in software engineering… The cost of software has always
119
been development cost, not replication cost… (F. P. Brooks 1995, 197–199)
Why was object-oriented programming necessary to create this market? Cox argued such software components needed to be produced at a much higher level of abstraction than previously. Here Cox shifted from the metaphor of 18th century firearms production to modern computer hardware production. Computer hardware was made up of different levels of components, with each level made up of smaller components from the level just underneath it. Thus, a computer might be made up of several racks of printed circuit boards, or cards. Each card contained multiple
integrated circuits, or chips. Each chip was designed modularly, as distinct combinations of logical “blocks.” Each block was made up of even more
fundamental building blocks, logic “gates” that represented AND, OR, and NOT.
And these gates themselves were made up of individual transistors. A computer hardware manufacturer need not produce its own chips, but merely needed to buy the chips it needed from other companies. This, in fact, had made the personal computer
And these gates themselves were made up of individual transistors. A computer hardware manufacturer need not produce its own chips, but merely needed to buy the chips it needed from other companies. This, in fact, had made the personal computer