Darwin's Dangerous Idea (84 page)

game you'll be offered a white move. The worst you can do is a draw. (If all Godel's Theorem in particular has nothing at all to tell us about whether the opening moves for white are colored black, most improbably, your only there might be algorithms in the Library of Toshiba that could do an im-hope is to choose one at random and hope that your opponent, playing black, pressive job of "producing as true" or "detecting as true or false" candidate goofs at some later stage of play and lets you escape by getting on a gray or a sentences of arithmetic. If human mathematicians can do an impressive job white path.)

of "just seeing" with "mathematical intuition" that certain propositions are That is an algorithm, clearly. No step in the recipe requires any insight, and true, perhaps a computer can imitate this talent, the same way it can imitate I have specified it unambiguously in a finite form. The trouble is that it is not chess-playing and conversation-holding: imperfectly, but impressively. That remotely feasible or practical, because the tree it exhaustively searches is is exactly what people in AI believe: that there are risky, heuristic algorithms Vast. But I suppose it is nice to know that in principle there is an algorithm for human intelligence in general, just as there are for playing good checkers for playing perfect chess, however useless. There
might
be a
feasible
and good chess and a thousand other tasks. And here is where Penrose made algorithm for playing perfect chess. No one has ever found one, thank his big mistake: he ignored this set of possible algorithms— the only set of goodness, since it would turn chess into a game of scarcely more interest than algorithms that AI has ever concerned itself with—and concentrated on the tic-tac-toe. No one knows whether there is such a feasible algorithm, but the set of algorithms that Godel's Theorem actually tells us something about.

general consensus is that it is very unlikely. Not knowing for sure, let's choose Mathematicians, Penrose says, use "mathematical insight" to see that a the supposition that makes for the worst case for AI. Let's suppose that
there
is no
feasible algorithm for checkmate or a guaranteed draw—none at all.

440 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Library of Toshiba
441

Does it follow that no algorithm running on my Toshiba can achieve be capable of indefinitely resourceful discrimination and planning; they must checkmate? Hardly! As I have already confessed, the chess algorithms on my be good at recognizing food and shelter, telling friend from foe, learning to Toshiba are undefeated in play against one human being—me. I'm not very discriminate harbingers of spring
as
harbingers of spring, telling good argu-good, but I expect I have about as much "insight" as the next human being.

ments from bad, and even—as a sort of bonus talent thrown in—recognizing Someday I might beat my machine, if I practiced a lot and worked very hard, mathematical truths
as
mathematical truths. Of course, such "Darwinian al-but the programs on my Toshiba are trivial compared with the current gorithms" ( Cosmides and Tooby 1989) wouldn't have been designed just for champion chess programs. About them you could safely
bet your life
that this special purpose, any more than our eyes were designed for telling
italics
they would checkmate me (though not Bobby Fischer)
every time.
I don't from boldface, but that doesn't mean that they aren't superbly sensitive to recommend to anyone that you actually bet your life on the relative excel-such differences if given a chance to consider them.

lence of these algorithms—I might improve, and I wouldn't want your death Now how could Penrose have overlooked this retrospectively obvious on my conscience—but in fact, if Darwinism is right, you and your ancestors possibility? He is a mathematician, and mathematicians are primarily inter-have an unbroken string of successful gambles for similarly fatal stakes on ested in that Vanishing subset of algorithms that they
can
prove, mathemat-the algorithms embodied in your "machinery." That is what organisms have ically, to have mathematically interesting powers. I call this the God's-eye done, every day since life began: they have bet their lives that the algorithms view of algorithms. It is analogous to the God's-eye view of volumes in the that built them, and that operate within them if they are among the lucky Library of Babel. We can "prove" (for what it is worth) that there is a single organisms with brains, will keep them alive long enough to have children.

volume in the Library of Babel that lists, in perfect alphabetical order, all the Mother Nature has never aspired to absolute certainty; a good risk is enough telephone subscribers in New York City whose net worth on January 10, for her. So we would
expect
that, if mathematicians' brains are running 1994, was more than a million dollars. There has to be—there couldn't be algorithms, they will be algorithms that happen to do pretty well in the truth-that many millionaire phone-owners in New York, and so some one of the detecting department, without being foolproof.

possible volumes in the Library must list them all. But finding it—or making The chess algorithms on my Toshiba, like all algorithms, yield guaranteed it—would be a huge empirical task fraught with uncertainties and judgment results, but what they are guaranteed to do is not checkmate me, but just
play
calls, even if we just considered it to be a subset of the names already printed
legal chess.
That is all they are "for." Of the Vast number of algorithms in the actual phone book as of that date (ignoring all those with unlisted numbers ). Even though we can't put our hands on this volume, we can name guaranteed to play legal chess, some are much better than others, though none it—just the way we named Mitochondrial Eve. Call it
Megaphone.
Now, we is guaranteed a win against any other—at least this is not the sort of thing one can prove things about
Megaphone,
for instance, the first letter printed on the would hope to prove mathematically, even if, as a matter of brute first page on which there is printing is "A," but the first letter on the last page mathematical fact, the initial state of program
x
and program
y
were such that on which there is printing is not "A." (This is not quite up to the standards of
x
would win all possible games against
y.
This means that the following mathematical proof, of course, but what are the odds that
none
of the people argument is fallacious:

with phones whose names begin with "A" is a million aire, or that there's only one page of such millionaires in all New York?)
x
is excellent at achieving checkmate;

As I noted on page 52, when mathematicians think about algorithms, it is there is no (practical) algorithm for checkmate in chess; usually from the God's-eye perspective. They are interested in proving, for
therefore:
the explanation of
x'
s
talent cannot be that
x
is running an instance, that
there is
some algorithm with some interesting property, or that algorithm.

there is no
such algorithm, and in order to prove such things you needn't actually locate the algorithm you are talking about—by picking it out from a The conclusion is obviously false: the algorithm level of explanation is pile of algorithms stored on floppy disks, for instance. Our inability to locate
exactly
the right level at which to explain the power of my Toshiba to beat (the remains of) Mitochondrial Eve did not prevent us from deducing facts me at chess. It's not as if it had particularly potent electricity running through about her either. The empirical issue of identification thus doesn't often arise it, or a secret reservoir of
elan vital
inside its plastic case. What makes it for such formal deductions. Godel's Theorem tells us that not a single one of better than other chess-playing computers (I can beat the really simple ones) the algorithms that can run on my Toshiba ( or any other computer ) has a is that it has a better algorithm.

certain mathematically interesting property: being a
consistent generator of
What kind of algorithms, then, might mathematicians be running? Algo-proofs of arithmetic facts that generates them all if given enough run time.

rithms "for"
trying to stay alive.
As we saw in our consideration of the survival-machine robots in the last chapter, such algorithms would have to 442 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Library of Toshiba
443

That is interesting, but it doesn't help us much. Lots of interesting things He goes on to dismiss his second loophole (the unsound algorithm) by can be proved, mathematically, about each and every member of various sets claiming (1991): "Mathematicians require a degree of rigour that makes such of algorithms. Applying that knowledge in the real world is another matter, heuristic arguments unacceptable—so no such known procedure of this kind and that is the blind spot that led Penrose to overlook AI altogether, instead can be the way that mathematicians actually operate." This is a more of refuting it, as he hoped. This has come out quite clearly in his subsequent interesting mistake, for with it he raises the prospect that the crucial attempts at reformulation of his claim in response to his critics.

empirical test would be not to put a
single
mathematician "in the box" but the whole mathematical community! Penrose sees the theoretical importance of Given any particular algorithm, that algorithm cannot be
the
procedure the added power that human mathematicians obtain by pooling their whereby human mathematicians ascertain mathematical truth. Hence resources, communicating with each other, and hence becoming a sort of humans are not using algorithms at all to ascertain truth. [Penrose 1990, single giant mind that is hugely more reliable than any one homunculus we p. 696.]

might put in the box. It is not that mathematicians have fancier
brains
than Human mathematicians are not using a knowably sound algorithm in order the rest of us (or than chimpanzees) but that they have mind-tools—the to ascertain mathematical truth. [Penrose 1991.]

social institutions in which mathematicians present each other their proofs, check each other out, make mistakes in public, and then count on the public In the more recent of these, he goes on to consider and close various to correct those mistakes. This does indeed give the mathematics community

"loopholes," of which two in particular concern us. mathematicians might be powers to discern mathematical truth that dwarf the powers of any individual using "a horrendously complicated
unknowable
algorithm
X"
or "an
unsound
human brain (even an individual brain with paper-and-pencil peripherals, a (but presumably approximately sound) algorithm
Y."
Penrose presents these hand calculator, or a laptop!). But this does not show that human minds are loopholes as if they were
ad hoc
responses to the challenge of Godel's
not
algorithmic devices; on the contrary, it shows how the cranes of culture Theorem, instead of the standard working assumptions of AI. Of the first he can exploit human brains in distributed algorithmic processes that have no says:

discernible limits.

This seems to be totally at variance with what mathematicians seem
ac-Penrose doesn't quite see it that way. He goes on to say that "it is our
tually
to be doing when they express their arguments in terms that can ( at general (non-algorithmic) ability to
understand"
that accounts for our least in principle ) be broken down into assertions that are 'obvious', and mathematical abilities, and then he concludes: "It was not an
algorithm
x agreed by all. I would regard it as far-fetched in the extreme to believe that that was favoured, in Man ( at least) by natural selection, but this wonderful it is
really
the horrendous unknowable
X,
rather than these simple and ability to understand!" (Penrose 1991). Here he commits the fallacy I just obvious
ingredients
[emphasis added], that lies lurking behind all our exposed using the chess example. Penrose wants to argue: mathematical understanding. [Penrose 1991]

x
can understand;

These "ingredients" are indeed wielded by us all in an
apparently
nonal-there is no feasible algorithm for understanding;

gorithmic way, but this phenomenological fact is misleading. Penrose pays
therefore:
what natural selection selected, the whatever-it-is that accounts careful attention to what it is like to be a mathematician, but he overlooks a for understanding, is not an algorithm.

possibility—indeed, a likelihood—that is familiar to AI researchers: the possibility that
underlying
our general capacity to deal with such "ingre-This conclusion is a
non sequitur.
If the mind is an algorithm (contrary to dients" is a heuristic program of mind-boggling complexity. Such a compli-Penrose's claim), surely it is not an algorithm that is recognizable to, or cated algorithm would
approximate
the competence of the perfect accessible to, those whose minds it creates. It is, in his terms, unknowable.

understander, and be "invisible" to its beneficiary. Whenever we say we As a product of biological design processes ( both genetic and individual), it solved some problem "by intuition," all that really means is
we don't know
is almost certainly one of those algorithms that are somewhere or other in the
how
we solved it. The simplest way of modeling "intuition" in a computer is Vast space of interesting algorithms, full of typographical errors or "bugs,"

simply denying the computer program any access to its own inner workings.

but good enough to bet your life on—so far. Penrose sees this as a "far-Whenever it solves a problem, and you ask it how it solved the problem, it fetched" possibility, but if that is all he can say against it, he has not yet should respond: "I don't know; it just came to me by intuition" (Dennett come to grips with the best version of "strong AI."