Variation and Selection

and a few other places did many experiments using the operant-conditioning method and explored the world of reinforcement schedules. Almost none of these experiments were a direct test of Skinner’s theoretical position, how-ever. Skinner’s proscription of the hypothetico-deductive approach—“As to hypotheses, [my] system does not require them” ¹ —seemed to make hypothesis testing unnecessary.

Increasingly, Skinner’s own gaze was directed elsewhere, to human behav-ior, which was always his chief interest. He used experimental results from the animal laboratory largely to provide scientifi c support for his analysis of human problems. Skinner’s classic 1948 paper “‘Superstition’ in the Pigeon” is one of the earliest and most dramatic examples of this strategy. ²

The experiment was incredibly simple. Skinner took a hungry pigeon that had learned to eat from an automatic feeder (Plate 6.1) but that had received no other training. The pigeon was placed in a Skinner box and given brief access to food every 12 s. This is response-independent reinforcement. The procedure is in fact classical or Pavlovian conditioning, not operant condi-tioning ( temporal condicondi-tioning —see Figure 2.1). After a few dozen feedings, every pigeon Skinner tried began to show vigorous, stereotyped activity. This

Plate 6.1 A typical Skinner box for pigeons. Food—the reinforcer—is delivered via a hopper that is raised for a few seconds. The pecking key was disconnected in the “superstition” experiment.

Foodhopper

Variation and Selection 75 result is surprising because the bird had no control over food delivery: All it was required to do was wait, and eat when food became available every few seconds. The pigeon’s vigorous activity was completely unnecessary. No con-tingency, hence no reinforcement. So how come all this vigorous activity?

We’ll see how Skinner turned this unanticipated and potentially damaging result to his own advantage. The “superstition” paper shows Skinner’s fl exibil-ity as an experimenter. It also shows his rhetorical genius. And it shows how he could vault effortlessly from an animal-laboratory curiosity to the most rarifi ed aspects of human culture.

The superstition experiment is almost unique in Skinner’s oeuvre because it presents observational, rather than automatically gathered, data. Nearly all subsequent experimental papers by Skinner and other operant condition-ers eschewed all but counter readings and cumulative records. Observational data were generally regarded as unacceptable. Yet the master, more adapt-able than his followers, presented simple, anecdotal observations without apology. To study a classical conditioning procedure, using such an informal method to gather data, was a creative departure for someone whose whole career until that time had been devoted to automated instrumental learning experiments.

Skinner’s rhetoric was brilliant because the results of the experiment—

vigorous activity, despite the absence of any operant response requirement—

would by most observers surely have been interpreted as contrary to his ideas about the indispensability of “consequences” as a molder of behavior. The pigeon’s behavior had no consequences. So why did it occur? The “condi-tioning” of vigorous skeletal activities by response-independent food delivery might well have been taken then as strong evidence against Skinner’s view that contingency (his term for dependency or causal linkage ) between response and reinforcer is essential to operant conditioning. The obvious interpretation is one that draws on the principles not of operant but of Pavlovian condition-ing. In fact, as we’ll see, the behavior in the superstition experiment manifests the properties of variation rather than selection.

Skinner solved the contradiction in the fi rst paragraph of the paper, by predict-ing (yes, predictpredict-ing ) the outcome of the experiment as a deduction from operant reinforcement theory (yes, theory ). Presenting new data, whether anticipated or not, as a logical deduction from existing theory had long been standard in many areas of experimental science. But Skinner’s choice of this method of presenta-tion is surprising because he elsewhere argued repeatedly against the hypothetico-deductive approach (“As to hypotheses, [my] system does not require them”) and, as far as I know, followed it in no other paper. Skinner usually advocated induction, not deduction. Consequently, it is hard to believe that his use of the method here was for anything but purposes of persuasion—a rhetorical device rather than an accurate account of why he actually did the experiment.

Skinner begins the paper as follows:

To say that a reinforcement is contingent upon a response may mean nothing more than that it follows the response. It may follow because of

some mechanical connection [contingency] or because of the mediation of another organism; but conditioning takes place presumably because of the temporal relation only, expressed in terms of the order and proximity of response and reinforcement. Whenever we present a state of affairs which is known to be reinforcing at a given level of [food] deprivation, we must suppose that conditioning takes place even though we have paid no atten-tion to the behavior of the organism in making the presentaatten-tion. A simple experiment demonstrates this to be the case. (italics added)

This is a remarkable paragraph. The terms “may” and “must suppose” tend to expel from the reader’s mind all sorts of inconvenient counterexamples. In hindsight, these questions pop into the foreground: To say that a reinforcement is contingent on a response does not mean only that it follows the response ( contiguity ). It also means at least two other things: that reinforcement cannot occur (or occurs less often) in the absence of the response and that reinforce-ment occurs more often if the response occurs more often. Skinner’s rhetoric had the effect of focusing readers’ attention on contiguity to the exclusion of the other properties of a reinforcement contingency. Persuading a behaviorist audience in this way was no mean feat because in all practical matters, causal dependency (the response is necessary for the reinforcer)—which implies all three properties—is the prevailing view. In clinical behavior modifi cation, for example, an undesirable behavior is abolished by omitting reinforcers nor-mally dependent on it, or by delivering punishment for undesirable behavior (self-injurious behavior, for example). Desirable behavior is strengthened by making reward dependent on it. In each case, there must be a real causal depen-dence of reinforcer or punisher on the target behavior.

Response contingency in Skinner’s usual sense (i.e., dependency) is a proce-dural feature, not a behavioral process—contingencies must act through some proximal mechanism. Skinner’s argument for the superstition paper relied on the fact that the mechanism for reinforcement was then widely thought to be nothing more than closeness in time: response-reinforcer contiguity. Contigu-ity was also thought to be the key to learning CS-US relations in classical conditioning. At this time, the inadequacy of simple contiguity theory as an account of either classical or operant conditioning was not fully understood.

Skinner explained the vigorous, stereotyped behavior of his pigeons in between periodic food deliveries by means of what he called adventitious reinforcement —that is, accidental contiguity between food and a behavior that originally occurs for “other reasons.” His argument begins with the reasonable assumption that a hungry pigeon is not passive in a situation where it receives occasional food. Suppose it happens to be doing something toward the end of an interfood interval and food is delivered. The behavior will be contiguous with the food and so, by the reinforcement principle, will be more likely to occur again. If the next food delivery comes quite soon, this same behavior might still be occurring, and so receive another accidental pairing with food, be further strengthened, occur again in the next interval and so on. By means

Variation and Selection 77 of this positive-feedback process, some behavior might be raised to a very high probability. Since there is no real causal relation between behavior and reinforcer, Skinner called the behavior superstitious —implying a parallel with human superstitions, which he believed to arise in a similar way.

This plausible account was not based on direct observation of the process.

No one had actually recorded these accidental response-reinforcer contigui-ties or the progressive increase in response strength that was supposed to fol-low them. Indeed, the contiguity view, in this impressionistic form, is almost immune to disproof.

Contiguity learning was in fact generally accepted as an explanation for the effects of contingency itself. The idea had been used to explain the effect of “free” (response-independent) reinforcers on a schedule of contingent rein-forcement. Even in 1948 there was some understanding that free reinforcers on a response-contingent schedule might tend to weaken the reinforced response. ³ This weakening was explained, in effect, by adventitious reinforcement. If, for some unexamined reason, a behavior other than the instrumental response should happen to occur, then by chance it would sometimes be contiguous with the occasional free reinforcers. This would tend to strengthen it and thus, by competition, weaken the reinforced response.

Few observations had been done to back up this hypothesis: Did such com-peting behaviors actually occur? How often were they contiguous with free food? What is the form of competition between different acts? Did the fre-quency of the instrumental response change before or after such accidental pairings? Skinner also offered no quantitative details: How many accidental pairings are needed to produce how much increment in strength? How often should unreinforced “other” behaviors be expected to occur? Nevertheless, the adventitious reinforcement hypothesis view was plausible enough to be accepted for more than twenty years.

The simple contiguity account of operant reinforcement poses a real meth-odological problem. It is a hypothesis where the putative cause is not a stimu-lus but a relation between the response (which is controlled by the animal, not the experimenter) and a reinforcing consequence. Inability to control the occurrence of the response makes it impossible to be sure of the effect of any particular pairing between response and reinforcer. For example, suppose the response occurs and we at once deliver a reinforcer, thus ensuring response-reinforcer contiguity. Suppose that a few additional responses then occur; have we demonstrated a strengthening effect of contiguity? Not at all; perhaps this response just happens to occur in runs, so that one response is usually followed by others, quite apart from any reinforcing effect. This is not uncommon—a pigeon will rarely make just one peck, for example. Suppose we reinforce a second response, just to be sure. After a few repeats, no doubt the pigeon is pecking away at a good rate; have we then demonstrated an effect of contigu-ity? Again, not really. By repeating the pairings of response and reinforcer, we have now established a real dependency between response and reinforcer. Per-haps the increase in pecking is just the result of some other process that allows

the pigeon to detect such molar contingencies (covariation between two rate measures), as they are called.

In fact, there is little doubt that animals often learn to repeat a response after just one response-reinforcer pairing—just like the law-of-effect (LOE) model I discussed in Chapter 4 . The question is whether persistent behavior that is in fact unnecessary for the delivery of the reinforcer—superstitious behavior—is the automatic consequence of a simple contiguity process. The fi rst does not imply the second. The point, of course, is that in the absence of any proposal for a specifi c real-time mechanism for the action of reinforcement, the hypoth-esis of adventitious reinforcement is almost impossible to disprove. Unless behavior in the absence of reinforcement is highly predictable, unless a single reinforcement reliably produces an immediate enhancing effect, preferably on more than one response type, the meaning of those occasional instances where responding seems to increase rapidly following a single contiguous reinforcer is ambiguous. A few apparently confi rming observations have appeared. ⁴ But there have been no convincing experimental proofs of the adventitious-reinforcement hypothesis as a process with anything like the generality often assumed for it.

The very obscurity of these theoretical points, along with the simplicity of his experimental method, combined to add plausibility to Skinner’s simple account of the vigorous behavior induced in pigeons (not all species show such effects) by periodic free food delivery. But I believe the main reason for the success of this paper, and of Skinner’s other writings, lies elsewhere: in his easy extension of dry data from the animal operant laboratory to highly salient aspects of human culture—like superstition. The leap here is substantial, from pigeons posturing in a box to rituals in card games and body language by bowlers and, in Skin-ner’s later writings, to religious beliefs. It rests not so much on any convincing isomorphism between the pigeon in the box and the human situation—where is temporal contiguity and frequent reinforcement in a card game, for example?—

as on the reader’s empathy with the pigeon. “The bird behaves as if there were a causal relation,” writes Skinner, ⁵ and the reader nods assent to an argument that has more in common with cognitive than behavioristic psychology.

Many possibilities are suggested by the two words “as if”! Is reinforce-ment the source of all knowledge? Is human knowledge as much an illusion as the pigeon’s mistaken causal inference in the superstition experiment? Deists behave “as if” there were a God, scientists “as if” truth exists, and all of us “as if” the fl oor will not open up under our feet. Are all these “superstitions”? Are all to be explained by reinforcement? Skinner would probably have answered,

“Yes. . . .”

Experimental Problems with Skinner’s Account

Skinner’s account of laboratory “superstition” stood unchallenged for twenty years. The fi rst cracks appeared in 1968 with the publication of a paper by Brown and Jenkins on what they called autoshaping. A subsequent paper

Variation and Selection 79 by Williams and Williams using the same procedure established that operant key pecking could be produced and maintained by purely Pavlovian proce-dures. ⁶ In these experiments, pigeons untrained to peck a response key are simply allowed to experience occasional pairing of a 7-s key illumination with access to food (Pavlovian delay conditioning—see Figure 2.1). The pro-cedure reliably produces key pecking after a couple of dozen pairings—even if key pecks have no effect at all. Indeed, the pigeon will continue to peck even if pecks actually prevent food delivery, a procedure termed omission training.

Brown and Jenkins’s result was surprising not because they got condition-ing, but because of the kind of response that was conditioned. Not the usual respondent, like salivation in Pavlov’s dogs, but the prototypical operant response, key pecking. This was important partly because Skinner had pro-posed a distinction between operant and respondent behavior—the former

“controlled” by its consequences, the latter by Pavlovian pairing (see Chap-ter 2 ). But if skeletal, “operant” responses can be produced through classical conditioning procedures—as autoshaping and the superstition experiment both show—then Skinner’s distinction between operant behavior (conditionable only by consequential reinforcement) and respondent behavior (conditionable only by Pavlovian means) loses its generality. Salivation, the prototypical Pav-lovian response, is of course physiologically different from pecking and lever pressing (the prototypical operant responses). Salivation, a respondent, is con-trolled by the autonomic nervous system; lever pressing, an operant response, by the skeletal nervous system. The point is that the terms “operant” and

“respondent” add nothing to these physiological differences if the terms fail to correspond to sharp differences in susceptibility to operant and Pavlovian con-ditioning. Pecking, and a number of other activities in other species, has both operant and respondent properties. Hence, the operant-respondent distinction becomes less useful.

A couple of years before Brown and Jenkins published their arresting results, Virginia Simmelhag and I had repeated Skinner’s 1948 experiment and observed the pigeons’ behavior from the very beginning of training. We looked second by second at the brief interval between feedings. A few years later we published the results in a long theoretical paper. ⁷ Our interest was purely exploratory; we had no hypothesis. The experiment was inductive, not hypothetico-deductive. Skinner would have approved! We wondered about two things: Would you get “superstitious” behavior even if the schedule were variable, as opposed to fi xed time? (Answer: yes.) And what are the details—

what is the bird doing second by second in each interfood interval?

The answer to the second question is very interesting. We found three things that differ from Skinner’s account:

1. The activities that develop are of two kinds: interim activities, which occur in the fi rst two-thirds or so of the interfood interval, are rarely con-tiguous with food and vary from subject to subject, and a single terminal

response, which occurs during the last third of the fi xed interval. The ter-minal response is restricted to times of high food probability and interim activities to times of low probability.

2. The terminal response is either pecking or a stereotyped pacing activity obviously related to it. The topography (form) of the terminal response does not differ from animal to animal in the irregular way Skinner described.

3. Neither the interim activities nor the terminal response develops in the accidental way implied by the adventitious-reinforcement hypothe-sis ( Figure 6.1 ). Pecking is never contiguous with food in the fi rst few days of the procedure. It suddenly supplants the behavior that had been contiguous—“head in feeder”—after a few days. In other words, there is no way that the pecking that comes to predominate can be explained by adventitious reinforcement. Indeed, it behaves just like a classically con-ditioned response, not the standard operant response it was thought to be.

The interim activities, on the other hand, occur at times of low reinforce-ment probability. They are almost never contiguous with food, hence they also cannot be explained by adventitious reinforcement.

These fi ndings have been repeated many times by others, and the details are now quite familiar. In the meantime a number of other superstition-like effects had been reported.

The most dramatic is something called instinctive drift. Keller Breland and his wife, Marion, were students of Skinner who learned their trade train-ing pigeons to guide missiles on “Project Pelican” durtrain-ing World War II and later went into the business of training animals for commercial purposes such as advertising. ⁸ They found, and reported in 1960, many years before the autoshaping experiment, that animal behavior is not so malleable—

“reinforcement shapes behavior as a sculptor shapes a lump of clay”—as Skin-ner had proposed. ⁹ They discovered that even though an animal may be trained to get food by making Response A, it will under some conditions after a while switch to (ineffective) Response B—even though it thereby misses food deliv-eries. Their work was essentially ignored until autoshaping and reanalysis of the superstition experiment allowed operant conditioning to be looked at in a new way—a way that had a place for instinctive drift.

In one demonstration, the Brelands trained a raccoon to pick up a wooden egg and drop it down a chute. At fi rst the animal readily released the egg and went at once to the feeder for its bite of food. But after a while the animal was

In one demonstration, the Brelands trained a raccoon to pick up a wooden egg and drop it down a chute. At fi rst the animal readily released the egg and went at once to the feeder for its bite of food. But after a while the animal was