Darwin's Dangerous Idea (79 page)

always make its way into the central processing unit in normal fashion, What was it about red strings that turned on the red light, and green strings where it would provoke a few billion operations to be performed in a few that turned on the green light? Of course, in each particular case, there was seconds, ending, always, with either of two output signals, a 1 (which turned no mystery at all. They could trace the causation of each particular string on the red light) or a 0 (which turned on the green light). In every case, they through the supercomputer in B and see that, with gratifying determinism, it found, they could explain each step of the causation at the mi croscopic level produced its red or green or amber light, as the case might be. What they without any difficulty or controversy. No occult causes were suspected to be couldn't find, however, was a way of predicting which of the three effects a operating, and, for instance, when they arranged to input the same sequence new string would have, just by examining it (without "hand-simulating" its of ten thousand bits again and again, the program in box B always yielded effect on box B ). They knew from their empirical data that the odds were the same output, red or green.

very high that any new string considered would be amber— unless it was a But this was mildly puzzling, because although it always gave the same string known to have been emitted by box A, in which case the odds were output, it didn't always yield the same output by going through the same better than a billion to one that it would be
either
red or green, but no one intermediate steps. In fact, it almost always passed through different states could tell which without running it through box B to see how the program before yielding the same output. This in itself was no mystery, since the settled out.

program kept a copy of each input it received, and so, when the same input Since, in spite of much brilliant and expensive research, they found them-arrived a second or third or thousandth time, the state of the memory of the selves still utterly unable to predict whether a string would turn out to be red, computer was slightly different each time. But the output was always the green, or amber, some theorists were tempted to call these properties same; if the light turned red the first time a particular string was input, it
emergent
properties. What they meant was that the properties were (they always turned red for the same string thereafter, and the same regularity held thought)
unpredictable in principle
from a mere analysis of the micro-for green strings ( as the scientists began to call them ). All strings, they were properties of the strings themselves. But this didn't seem likely at all, since tempted to hypothesize, are either red strings ( cause the red light to flash) or each particular case was as predictable as any deterministic input to a green strings (cause the green light to flash). But of course they hadn't tested deterministic program could be. In any event, whether or not the properties all possible strings—only strings that had been emitted by box A.

of red, green, and amber were unpredictable in principle or merely in So they decided to test their hypothesis by disconnecting A from B tem-practice, they certainly were surprising and mysterious properties.

porarily and inserting variations on A's output strings to B. To their puzzle-Perhaps the solution to the mystery lay in box A. They opened it up and ment and dismay, they discovered that almost always when they tampered found another supercomputer—of a different make and model, and running a with a string from A, the
amber
light flashed! It was almost as if box B had different gigantic program, but also just a garden-variety digital computer.

detected their intervention. There was no doubt, however, that box B would They soon determined that whenever you pushed button a this sent the readily accept man-made versions of red strings by flashing red, and man-program off in one way, by sending a code (11111111) to the CPU, and made versions of green strings by flashing green. It was only when a bit—or whenever you pushed button (3 this sent a different code ( 00000000 ) to the more than one bit—was changed in a red or green string that the amber light CPU, setting in motion a different set of billions of operations. It turned out usually—almost always—came on. "You've killed it!" somebody once that there was an internal "clock" ticking away millions of times a second, blurted out, after watching a "tampered" red string turn into an amber string, and whenever you pushed either button the first thing the computer did was and this led to a flurry of speculation that red and green strings were in some take the "time" from the clock (e.g., 101101010101010111) and break it up sense
alive
—perhaps male and female—whereas amber strings were
dead
into strings it then used to determine which subroutines to call in which strings. But, appealing as this hypothesis was, it did not turn out to lead order, and which part of its memory to access first in the course of its anywhere, although further experimentation with a few billion random preparation of a bit string to send down the wire.

variations on bit strings of ten thousand bits in length did strongly The scientists were able to figure out that it was this clock-consulting (which was as good as random) that virtually guaranteed that the same bit 416 THE EVOLUTION OF MEANINGS

Two Black Boxes
417

string was never sent out twice. But in spite of this randomness, or pseudo-to
choose at random (or pseudo-random) one of its "beliefs" (either a stored randomness, it remained true that whenever you pushed button a the bit axiom or a generated implication of its axioms), translate it into English (in a string the computer concocted turned out to be red, and whenever you pushed computer, English characters would already be in ASCII), add enough button (3 the bit string eventually sent turned out to be green. Actually, the random bits after the period to bring the total up to ten thousand, and send scientists did find a few anomalous cases: in roughly one in a billion trials, the resulting string to B, which translated this input into its own language pushing the
a
button caused a green string to be emitted, or pushing the ß (which was Swedish Lisp), and tested it against its own "beliefs"— its data button caused a red string to be emitted. This tiny blemish in perfection only base. Since both data bases were composed of truths, and roughly the same whetted the scientists' appetite for an explanation of the regularity.

truths, thanks to their inference engines, whenever A sent B something A And then, one day, along came the two AI hackers who had built the

"believed," B "believed" it, too, and signaled this by flashing a red light.

boxes, and they explained it all. (Do not read on if you want to figure out the Whenever A sent B what A took to be a falsehood, B announced that it mystery for yourself.) Al, who had built box A, had been working for years judged that this was indeed a falsehood by flashing a green light.

on an "expert system"—a data base containing "true propositions" about And whenever anyone tampered with the transmission, this almost always everything under the sun, and an inference engine to deduce further resulted in a string that was not a well-formed sentence of English (B had implications from the axioms that composed the data base. There were absolutely zero tolerance for "typographical" errors). B responded to these major-league baseball statistics, meteorological records, biological taxono-by flashing the amber light. Whenever anyone chose a bit string at random, mies, histories of the world's nations, and hosts of trivia in the data base. Bo, the odds were Vast that it would not be a well-formed truth or falsehood of the Swede who had built box B, had been working during the same time on a English ASCII; hence the preponderance of amber strings.

rival "world-knowledge" data base for his own expert system. They had both So, said Al and Bo, the emergent property
red
was actually the property of stuffed their respective data bases with as many "truths" as years of work had being a true sentence of English, and
green
was the property of being a permitted.6

falsehood in English. Suddenly, the search that had eluded the scientists for But as the years progressed, they had grown bored with expert systems, years became child's play. Anyone could compose red strings
ad nauseam
—

and had both decided that the practical promise of this technology was vastly just write down the ASCII code for "Houses are bigger than peanuts" or overrated. The systems weren't actually very good at solving interesting

"Whales don't fly" or "Three times four is two less than two times seven," for problems, or "thinking," or "finding creative solutions to problems." All they instance. If you wanted a green string, try "Nine is less than eight" or "New were good at, thanks to their inference engines, was generating lots and lots York is the capital of Spain."

of true sentences (in their respective languages), and testing input sentences Philosophers soon hit upon cute tricks, such as finding strings that were (in their respective languages) for truth and falsity—relative to their red the first hundred times they were given to B but green thereafter (e.g.,

"knowledge," of course. So Al and Bo had got together and figured out how the ASCII for "This sentence has been sent to you for evaluation fewer than the fruits of their wasted effort could be put to use. They decided to make a a hundred and one times" ).

philosophical toy. They chose a
lingua franca
for translating between their But, said some philosophers, the string properties
red
and
green
are not two representational systems (it was English, actually, sent in standard really
truth in English
and
falsity in English.
After all, there are English ASCII code, the code of electronic mail), and hooked the machines together truths whose ASCII expression takes millions of bits, and besides, in spite of with a wire. Whenever you pushed A's a button, this instructed A their best efforts, Al and Bo didn't always insert
facts
in their programs.

Some of what had passed for common knowledge when they were working on their data bases had since been disproven. And so forth. There were lots of reasons why the string property—the causal property—of
redness
was not 6. For a real-world example of such a project, see Douglas Lenat's enormous CYC (short quite exactly the property of
truth in English.
So, perhaps
red
could better be for "encyclopedia" ) project at MCC ( Lenat and Guha 1990). The idea is to hand-code all defined as
relatively short expression in English ASCII of something
the millions of facts in an encyclopedia (plus all the other millions of facts that everyone

"believed" true by box B (whose "beliefs" are almost all true).
This satisfied knows, so there is no point in putting them in the encyclopedia—such as the facts that mountains are bigger than molehills, and toasters can't fly ), and then attach an inference some, but other picked nits, insisting, for various reasons, that this definition engine that can update, preserve consistency, deduce surprising implications, and in was inexact, or had counterexamples that could not be ruled out in any
non-general service the world-knowledge base. For an entirely different approach to AI,
ad hoc
way, and so forth. But as Al and Bo pointed out, there were no better consider Rodney Brooks' and Lynn Stein's humanoid-robot project (Dennett 1994c).

candidate descriptions of the property to be found, and 418 THE EVOLUTION OF MEANINGS

Blocking the Exits 4
19

hadn't the scientists been yearning for just such an explanation? Hadn't the ties anywhere in the world. Content, they said, had been
eliminated.
The mystery of red and green strings now been entirely dissolved? Moreover, debate went on for years, but the mystery with which we began was solved.

now that it was dissolved, couldn't one see that there wasn't any hope at all of explaining the
causal
regularity with which we began our tale without using
some
semantical (or mentalistic) terms?

Some philosophers argued that, though the newfound description of the 3. BLOCKING THE EXITS

regularity in the activity in the wire could be used to predict box B's behavior, it was not a
causal
regularity after all. Truth and falsehood ( or any The tale ends there. Experience teaches, however, that there is no such thing of the adjusted stand-ins just considered) are semantic properties, and as such as a thought experiment so clearly presented that no philosopher can are entirely abstract, and hence could not cause anything. Nonsense, others misinterpret it, so, in order to forestall some of the most attractive misinterpretations, I will inelegantly draw attention to a few of the critical details retorted. Pushing button a causes the red light to go on just as certainly as and explain their roles in this intuition pump.

turning the ignition key causes your car to start. If
it
had turned out that what was being sent down the wire was simply high versus low voltage, or one (l)The devices in boxes A and B are nothing but automated encyclo-pulse versus two, everybody would agree that this was a paradigm causal pedias—not even "walking encyclopedias," just "boxes of truths."

system. The fact that this system turned out to be a Rube Goldberg machine Nothing in the story presupposes or implies that these devices are didn't show that the reliability of the link between a and red flashes was any conscious, or
thinking things,
or even
agents,
except in the same less causal. On the contrary, in every single case the scientists could trace out minimal sense in which a thermostat is an agent. They are utterly the exact microcausal path that explained the result.7

boring intentional systems, rigidly fixed to fulfilling a single, simple Convinced by this line of reasoning, other philosophers began to argue that goal. They contain large numbers of true propositions and the infer-this showed that the properties red, green, and amber weren't
really
ential machinery necessary to generate more truths, and to test for semantical or mentalistic properties after all, but only imitation semantical