Donald Knuth gave an apparently different answer to the question of what CS is:
[C]omputer science is . . .the study ofalgorithms. (Knuth, 1974b, p. 323; my boldface, Knuth’s italics)
Historical Digression:
Knuth is the Turing Award-winning author of a major, multi-volume work on algorithms (The Art of Computer Programming(Knuth, 1973)), as well as developer of the TEX computer typesetting
system that this book’s manuscript was prepared in. For an interview with Knuth, see Roberts 2018.
3.6.1
Only Algorithms?
He cited, approvingly, a statement by Forsythe (1968) that the central question of CS is: What can be automated? Presumably, a process can be automated—that is, done automatically, by a machine, without human intervention—if it can be expressed as an algorithm. (We’ll return to this in§3.15.2.1.1.)
Further Reading:For a book-length discussion of this, see Arden 1980.
Knuth (1974b, p. 324) even noted that the name ‘computingscience’ might be bet- ter than ‘computerscience’, because the former sounds like the discipline is the science of computing(what youdowith computers) as opposed to the science ofcomputers
(thetoolsthemselves). Others have made similar observations: Foley (2002) says that “computing includes computer science” but goes beyond it. Denning (2013b, p. 35) says: “I have encountered less skepticism to the claim that ‘computing is science’ than to ‘computer science is science’.”
As Knuth pointed out,
a person does not really understand something until he [sic] teaches it to someone else. Actually a person does notreallyunderstand something until he can teach it to acomputer, that is, express it as an algorithm. (Knuth, 1974b, p. 327)
The celebrated cellist Janos Starker once said something similar: “When you have to explain what you are doing, you discover what you are really doing” (Fox, 2013).
Further Reading:
Schagrin et al. 1985, p. xiii, and Rapaport and Kibby 2010,§2.4.2, also discuss this idea.
And expressing something as an algorithm requires “real” understanding, because ev- ery step must be spelled out in excruciating detail:
It is a commonplace that a computer can do anything for which precise and unam- biguous instructions can be given. (Mahoney, 2011, p. 80)
That is, a computer can do anything for which analgorithmcan be given (for, after all, isn’t an algorithm merely “precise and unambiguous instructions”?). Thought of this way, the comment is almost trivial. But consider that to give such instructions (to give an algorithm) is to be able to explicitlyteachthe computer (or the executor, more generally) how to do that thing. (For a humorous commentary on this, see Figure 3.3.) But there is a potential limitation to Knuth’s theory that weteachcomputers how to do something—more specifically, to the theory that, insofar as CS is the study of what tasks arecomputable, it is the study of what tasks areteachable. The potential limitation is that teaching is “propositional”, in the sense that it requires sentences (propositions) of alanguage. Hence, it isexplicitor conscious. It is what psychologist and Nobel laureate Daniel Kahneman has called a “System 2” task:
System 2allocates attention to the effortful mental activities that demand it, includ- ing complex computations. The operations of System 2 are often associated with
Figure 3.3: http://xkcd.com/894/
the subjective experience of agency, choice, and concentration. (Kahneman, 2011, p. 21)
But there is another algorithmic way of getting a computer to do something: by
training it, either via a connectionist, neural-network algorithm, or via a statistical, machine-learning algorithm. ‘Learning’, in this sense of ‘machine learning’, is differ- ent from being (propositionally)taught. Such training isimplicitorunconscious. It is “System 1” thinking:
System 1operates automatically and quickly, with little or no effort and no sense of voluntary control. (Kahneman, 2011, p. 20)
We, as external, third-person observers, don’t consciously or explicitly know how to do a System-1 task. Knowinghowis not necessarily the same as knowingthat.
Further Reading:
The knowing-how/knowing-that distinction was first discussed in Ryle 1945. Stanley and Williamson 2001 argue that knowing howisa form of knowing that; Pavese 2015 takes up their view in the context of computer programs. For a survey of current work on the distinction, see Cath 2019.
The two “systems” or “types” of thinking are discussed in much greater detail in Evans and Stanovich 2013. There, “Type 1” (or “intuitive”) processing is characterized as indepen- dent of working memory (which is a kind of short-term, conscious memory) and as “au- tonomous”. And “Type 2” (or “reflective”) processing is characterized as “requir[ing] work- ing memory” and involving “cognitive decoupling” and “mental simulation”. (Cognitive de- coupling is, roughly, the ability to mentally represent a mental representation, so that the second-order representation can be thought about separately from the original representa- tion. For more on the two “systems” of thinking, and on unconscious cognition more gen- erally, see http://en.wikipedia.org/wiki/Dual process theory#Systems and the bibliography at: http://www.cse.buffalo.edu/∼rapaport/575/rules-connections.html#uncs-cognition. We’ll return
to these ideas in§12.4.4.1.2.2, when we discuss the difference between following rules and be- having in accordance with them, and again in§18.4, when we discuss whether computers can make decisions. On the difference between classical symbolic programming (where the pro- grammer “teaches” the computer how to do something) and machine-learning systems (where the computer “learns” how to do something without being explicitly taught), see Seabrook 2019.
Here is an example that might help to explain the difference: Consider the game of tic-tac-toe. A computer (or a human player) might be programmed—that is, explicitly “taught”—to play winning tic-tac-toe by using a “conscious” or “System 2” algorithm that it explicitly follows. Most older children and adults have been taught a version of this algorithm (Zobrist, 2000):
For player X to win or draw (that is, to not lose),do:
begin
ifthere are 2 Xs in a row,thenmake 3 Xs in a row
else ifthere are 2 Os in a row, then block with an X
else if2 rows intersect with an empty square such that each row contains 1 X, no Os,
thenplace X at the intersection
else if2 rows intersect with an empty square such that each row contains 1 O, no Xs,
thenplace X at the intersection
else ifthere is a vacant corner square,thenput X there
elseplace X on any vacant square.
end
Alternatively, a computer can be programmed tolearn howto play winning tic-tac-toe in a “System 1” manner, without expressing (or being able to express) that strategy propositionally, that is, in a “System 2” manner. Such a learning mechanism can be found in Michie 1961. Briefly, the computer is “rewarded” for each random move that
leads to a win or draw, and such moves are thus caused to be made more frequently in future games.
Further Reading:
There are at least two implementations of Michie’s method online: 1. “MENACE: Machine Educable Noughts and Crosses Engine”,
http://www.mscroggs.co.uk/blog/19
2. “MENACE 2, an Artificial Intelligence Made of Wooden Drawers and Coloured Beads”,
http://we-make-money-not-art.com/menace-2-an-artificial-intelligence-made-of-wooden-drawers- and-coloured-beads/
An algorithm in the form of a System-1–style artificial neural network is akin to building in to the computer the ability, as if “innate”, to do that thing. Such a computer could not necessarily tell ushowit was doing it; it would not necessarily have any “conscious” access to its algorithm. An algorithm in the form of an explicit machine- learning program that would enable the computer to learn how to do that thing is some- where in the middle; it would be conscious of its ability to learn, but not necessarily of how to do the thing; it might not necessarily be able to teach someone or something else how to do it, unless it could observe itself doing it and develop a theory of how to do it (which theory would be expressed in a System-2–style, explicit algorithm). (We’ll return to these issues in§§3.9.5 and 3.14.4.)
Let’s say for now that something is computable just in case “precise and unam- biguous instructions can be given” for it. (We’ll be more precise and unambiguous(!) in Chapter 7.) So, the question becomes: What tasks are amenable to “precise and unambiguous instructions”? Presumably, chess is computable in this sense, because there are explicit rules for how to play chess. (Playingwinning chess is a different matter!) But vision would seemnotto be thus computable. After all, one cannot give “precise and unambiguous instructions” that would enable someone to see. Yet there
arecomputer-vision systems (see http://aitopics.org/topic/vision for an overview), so visiondoesseem to be computable in a different sense: A behavior is computable if it can bedescribedin terms of such instructions. The entity that exhibits that behavior naturally might notuse, or be able to use, those instructions in order to behave that way. But we might be able to give those instructions to another system thatcoulduse them to exhibit that behavior. So, for instance, the human brain might not literally compute in the sense of executing an algorithm in order to see, but a computer using that algorithm might be able to exhibit visual behavior. (Whether it “sees”, phenomenologically, is a philosophical question!) Similarly, the solar system might not be executing Kepler’s laws, but an artificial solar system might. (We’ll look into this issue in§9.8.2.)
3.6.2
Or Computers, Too?
Knuth goes on to point out, however, that you need computers in order to properly study algorithms, because “human beings are not precise enough nor fast enough to carry out any but the simplest procedures” (Knuth, 1974b, p. 323). Indeed, he explicitly copies Newell, Perlis, & Simon’s strategy, revising his initial definition to include computers, that is, the phenomena “surrounding” algorithms:
When I say that computer science is the study of algorithms, I am singling out only one of the “phenomena surrounding computers,” so computer science actually includes more. (Knuth, 1974b, p. 324)
Are computers really necessary? If they are, does that mean that CS is (as Newell, Perlis, & Simon claim) the study of computers? Let’s consider some similar questions for other disciplines: Do youneeda compass and straightedge to study geometry (or can you study it just by proving theorems about points, lines, and angles)? After all, the mathematician David Hilbert wrote a completely axiomatic treatment of geometry without any mention of compass or straightedge (Hilbert, 1899). Do youneeda micro- scope to study biology? I doubt that Watson and Crick used one when they discovered the structure of DNA. Do youneeda calculator (or a computer!) to study physics or mathematics (or do they just help you perform calculations more quickly and easily)? Even if you do need these tools, does that make geometry the study of compasses and straightedges, or physics and math the study of calculators, or biology the study of microscopes? I think most people would say that these disciplines are not studies of those tools. On the other hand, “deep learning” algorithms do seem to need computers in order to determine if they will really do what they are intended to do, and do so in real time (Lewis-Kraus, 2016). (We’ll return to this in§3.12.)
About ten years later, Knuth (1985, pp. 170–171) backed off from the “related phenomena” definition, more emphatically defining CS as “primarily the study of al- gorithms”, because he “think[s] of algorithms as encompassing the whole range of concepts dealing with well-defined processes, including the structure of data that is be- ing acted upon as well as the structure of the sequence of operations being performed”, preferring the name ‘algorithmics’ for the discipline. (Gal-Ezer and Harel 1998, p. 80 say that the “heart and basis” of the field is “algorithmics” even though this does not “cover the full scope of CS”.) Knuth also suggests that what computer scientists have in common (and that differentiates them from people in other disciplines) is that they are all “algorithmic thinkers” (Knuth, 1985, p. 172). (We will see what it means to “think algorithmically” in§3.14.5, below, and in Chapter 7.)
Further Reading:
Chazelle 2006 and Easton 2006 discuss the nature of CS as the study of algorithms.