Darwin's Dangerous Idea (86 page)

tine Equations "in general" or for checking for halting "in general"? (Re-If a Penrose-style quantum-gravity brain were truly capable of nonalgo-member: we shouldn't spend a nickel on perpetual-motion machines, even to rithmic activity, and if we have such brains, and if our brains are themselves save our lives, since it will be money wasted on an impossible task.) Penrose's answer was illuminating: if the candidates for truth-checking the products of an algorithmic evolutionary process, a curious inconsistency

"just somehow bubble up out of the ground," then we would be wise to spend emerges: an algorithmic process ( natural selection in its various levels and the money, but if some intelligent agent is the source of the candidates and incarnations) creates a nonalgorithmic subprocess or subroutine, turning the gets to examine the program in our truth-checker, then it can foil our
whole
process (evolution up to and
including
human mathematician brains) algorithmic truth-checker by constructing just the "wrong" candidate or into a nonalgorithmic process after all. This would be a cascade of cranes candidates—an equation unsolvable by it, or a program whose termination creating, eventually, a real skyhook! No wonder Penrose has his doubts prospects will confound it. To make the distinction vivid, we can imagine about the algorithmic nature of natural selection. If it were, truly, just an that a space pirate, Rumpelstiltskin by name, is holding the planet hostage, algorithmic process at all levels, all its products should be algorithmic as but will release us unharmed if we can answer a thousand true-false well. So far as I can see, this isn't an inescapable formal contradiction; questions about sentences of arithmetic. Should we put a human mathema-Penrose could just shrug and propose that the universe contains these basic tician on the witness stand, or a computer truth-checker devised by the best nuggets of nonalgorithmic power, not themselves created by natural selection programmers? According to Penrose, if we hang our fate on the computer in any of its guises, but incorporatable by algorithmic devices as found
and let Rumpelstiltskin see the computer's program,
he can devise an objects whenever they are encountered (like the oracles on the toadstools).

Achilles'-heel proposition that will foil our machine. (This would be true Those would be truly nonreducible skyhooks.

independendy of Godel's Theorem, if our program was a heuristic truth-The position is, I guess, possible, but Penrose must face an embarrassing checker, taking risks like any chess program.) But Penrose has given us no shortage of evidence for it. The physicist Hans Hansson came up with a good reason to believe that this isn't just as true of any human mathematicians we challenge in Abisko, comparing a perpetual-motion machine to a truth-might put on the witness stand. None of us is perfect, and even a team of detecting computer. Different sciences, Hansson noted, can offer different 450 THE EMPEROR'S NEW MIND, AND OTHER FABLES

The Phantom Quantum-Gravity Computer
451

experts no doubt has some weaknesses that Rumpelstiltskin could exploit, fine actuality is still accessible to us, thanks to the R-and-D work of all our given enough information about their brains. Von Neumann and Morgen-predecessors! We might as well make the most of what we have, thereby stern invented game theory to deal with the particular class of complicated leaving rather more for our descendants to work with.

problems that life throws at us when there are other agents around to compete It is time to turn the burden of proof around, the way Darwin did when
he
with us. You are always wise to shield your brain from such competitors, challenged his critics to describe some
other
way—other than natural whether you are a human being or a computer. The reason a competitive selection__ in which all
the wonders of nature could have arisen. Those who agent makes a difference in this instance is that the space of all mathematical think the human mind is nonalgorithmic should consider the hubris pre-truths is Vast, the space of Diophantine Equation solutions is a Vast but suopposed by that conviction. If Darwin's dangerous idea is right, an algo-Vanishing subspace within it, and the odds of hitting upon a truth
at random
rithmic process is powerful enough to design a nightingale and a tree.

that would "break" or "beat" our machine is truly negligible, whereas an Should it be that much harder for an algorithmic process to write an ode to a intelligent
search
through that space, guided by knowledge of the particular nightingale or a poem as lovely as a tree? Surely Orgel's Second Rule is style of the opponent and its limitations, would be likely to find the needle in correct: Evolution is cleverer than you are.

the haystack: a crushing countermove.

Rolf Wasen raised another interesting point in Abisko. The class of
interesting
algorithms no doubt includes many that are not
humanly accessible.

CHAPTER 15:
Gödel's Theorem does not cast doubt on the possibility of AI
To put it dramatically, there are programs out there in the Library of Toshiba
after all- In fact, once we appreciate how an algorithmic process can escape
that would not just run on my Toshiba, but be valued by me for the won
the clutches of Gödel's Theorem, we see more clearly than ever how Design
derful work they would do for me, but that no human programmers, or any of
Space is uniSed by Darwin's dangerous idea.

their artifacts (program-writing programs already exist), will ever be able to create! How can this be? None of these wonderful programs is more than a CHAPTER 16:
What, then, about morality? Did morality evolve, too? Socio-megabyte long, and there are plenty of actual programs much bigger than
biologists from Thomas Hobbes to the present have offered Just So Stories
that already. Once again, we must remind ourselves just how Vast the space
about the evolution of morality, but, according to some philosophers, any
of such possible programs is. Like the space of possible five-hundred-page
such attempt commits the "naturalistic fallacy": the mistake of looking to
novels, or fifty-minute symphonies, or five-thousand-line poems, the space
Acts about the way the world
is
in order to ground
—
or reduce
—
ethical
of megabyte-long programs will only ever get occupied by the slenderest
conclusions about how tilings
ought
to be. This "fallacy" is better seen as a
charge of greedy reductionism, a charge which is often justified. But then we
threads of actuality, no matter how hard we work.

shall just have to be less greedy in our reductionism.

There are short novels nobody could write that would not just be best-sellers; they would be instantly recognized as classics. The keystrokes required to type them are all available on any word-processor, and the total number of keystrokes in any such book is trivial, but they still lie beyond the horizon of human creativity. Each particular creator, each novelist or composer or computer programmer, is sped along through Design Space by a particular idiosyncratic set of habits known as a
style
( Hofstadter 1985, sec.

Ill). It is style that both constrains and enables us, giving a positive direction to our explorations but only by rendering otherwise neighboring regions off limits to us—and if off limits to us in particular, then probably off limits to everyone forever. Individual styles are truly unique, the product of untold billions of serendipitous encounters over the ages, encounters that produced first a unique genome, and then a unique upbringing, and finally a unique set of life experiences. Proust never got a chance to write any novels about the Vietnam War, and no one else could ever write
them
—the novels recounting
that
epoch in
his
manner. We are stuck, by our actuality and finitude, in a negligible corner of the total space of possibilities, but what a

CHAPTER SIXTEEN

On the Origin of Morality

1. E PLURIBUS UNUM?

Nature (the Art whereby God hath made and governes the World) is by
the
Art
of man, as in many other things, so in this also imitated, that it
can make an Artificial Animal. For seeing life is but a motion of Limbs,
the begining whereof is in some principall part within; why may we not
say, that all
Automata
(Engines that move themselves by springs and
wheeles as doth a watch) have an artificial life? For what is the
Heart,
but a
Spring;
and the
Nerves,
but so many
Strings;
and the
Joynts,
but
so many
Wheeles
giving motion to the whole Body, such as was intended by the Artificer?
Art
goes yet further, imitating that Rationall
and most excellent worke of Nature,
Man.
For by Art is created that
great LEVIATHAN called a COMMON-WEALTH or STATE (in Latine CIVITAS) which is
but an Artificiall Man; though of greater stature and strength than the
Natural!, for whose protection and defence it was intended; and in
which, the
Soveraignty
is an Artificiall
Soul,
as giving life and motion to
the whole body.

—THOMAS HOBBES 1651, p. 1

Thomas Hobbes was the first sociobiologist, two hundred years before Darwin. As the opening words of his masterpiece make clear, he saw the creation of the state as fundamentally a matter of one artifact's making another, a sort of group-survival vehicle, "intended" for the "protection and defence" of its occupants. The frontispiece of the original edition shows how seriously he took his own metaphor. Why, though, do I call Hobbes a sociobiologist? He couldn't have wanted exploit
Darwin's
ideas in an analysis of society, like today's sociobiolo-gists. But he did see, clearly and confidently, the fundamental Darwinian task: he saw that there
had
to be a story to be told about how the state first

454 ON THE ORIGIN OF MORALITY

E Pluribits Unum?
455

came to be created, and how it brought with it something altogether new on the in due course, their descendants created multicellular societies of a very face of the Earth: morality. It would be a story taking us from a time in which peculiar sort, known (until recently) as Men, capable of creating Letters (or there clearly was no right and wrong, just amoral competition, to a time in representations), which they fell to exchanging promiscuously; this made which there manifestly was right and wrong (in some parts of the biosphere) possible a second revolution.

via a process that gradually introduced the "essential" features of an ethical How Hobbes would have admired Richard Dawkins' story of the birth of perspective. Since the relevant period was prehistoric, and since he had no memes, and the creation thereby of persons, who were
not
mere survival fossil record to consult, his story would have to be a rational reconstruction, a vehicles for their genes! These tales, composed long after his, narrate major Just So Story of sorts (to commit a further anachronism). Once upon a time, he steps in evolution that antedate the step he decided to describe: the step from said, there was no morality at all. There was life; there were human beings, persons without morality to citizens. He saw this, correctly, as a major step and they even had language, so they had memes (to commit a third in the history of life on this planet, and he set out to tell the tale, as best he anachronism ). We can presume that they had words— and hence memes—for could, of the conditions under which this step could be taken and, once good and bad, but not
ethical
good and bad. "The notions of Right and Wrong, taken,
evolutionarily enforced
(to use one more anachronism). Though it was Justice and Injustice have there no place." So, although they distinguished a not a saltation but a small step, it had momentous consequences, for it was good spear from a bad spear, a good supper from a bad supper, a good hunter the birth of a hopeful monster indeed.