Monkeys and Typewriters

by Russ Roberts on May 9, 2007

in Data

In Fooled by Randomness, Nassim Nicholas Taleb gives the example of a person who wants to hire someone to type a copy of Homer’s Odyssey. Suppose an infinite number of monkeys are typing on an infinite number of typewriters. One of them comes up with a perfect copy of the Illiad. This enterprising monkey shows up at your door, applying for the job of Odyssey typist. But you’re a monkey, you reply skeptically. He takes out the perfect copy of the Illiad. You’re stunned. And you apologize and hire him on the spot, foolishly interpreting the perfect manuscript of the Illiad as an indication of the monkey’s skills.

Taleb’s point is that before you decide that monkeys are good at typing, you want to know more about the denominator—the number of monkeys trying to type the Illiad. When people claim that this stock-picker or that fund manager is a genius because he’s beaten the S&P 500 for 15 straight years, you should ask how many people are out there picking stocks and what proportion of those would be expected to beat the S&P for purely random reasons. It’s possible there are fund managers who are genuinely skilled but the fact that they do well over some limited time period is not proof.

A more mundane application is asking for references from a contractor. I always ask for two or three references. But a contractor who has been in business for any length of time will be able to find a few good ones—even a few raves—that may not be representative of the typical experience of the firm’s clients. The better strategy is to ask for references from the last three clients or clients from the last six months.

Here is a wonderful page on the infinite monkey metaphor. It will give you a healthy sense of what low-probability events really mean:

One computer program run by Dan Oliver of Scottsdale, Arizona, according to an article in The New Yorker,
came up with a result on August 4, 2004: After the group had worked for
42,162,500,000 billion billion years, one of the "monkeys" typed,
“VALENTINE. Cease toIdor:eFLP0FRjWK78aXzVOwm)-‘;8.t . . ." The first 19
letters of this sequence can be found in "The Two Gentlemen of Verona".
Other teams have reproduced 18 characters from "Timon of Athens", 17
from "Troilus and Cressida", and 16 from "Richard II".[20]

A website entitled The Monkey Shakespeare Simulator, launched on July 1, 2003, contained a Java applet
that simulates a large population of monkeys typing randomly, with the
stated intention of seeing how long it takes the virtual monkeys to
produce a complete Shakespearean play from beginning to end. For
example, it produced this partial line from Henry IV, Part 2, reporting that it took "2,737,850 million billion billion billion monkey-years" to reach 24 matching characters:

RUMOUR. Open your ears; 9r"5j5&?OWTY Z0d…

Even more entertaining is this study of actual monkeys:

In 2003, lecturers and students from the University of Plymouth MediaLab Arts course used a £2,000 grant from the Arts Council to study the literary output of real monkeys. They left a computer keyboard in the enclosure of six Sulawesi Crested Macaques in Paignton Zoo in Devon in England
for a month, with a radio link to broadcast the results on a website.
One researcher, Mike Phillips, defended the expenditure as being
cheaper than reality TV and still "very stimulating and fascinating
viewing".[24]

Not only did the monkeys produce nothing but five pages[25] consisting largely of the letter S,
the lead male began by "bashing the hell out of" the keyboard with a
stone, and the monkeys continued by urinating and defecating on it. The
zoo’s scientific officer remarked that the experiment had "little
scientific value, except to show that the ‘infinite monkey’ theory is
flawed". Phillips said that the artist-funded project was primarily
performance art, and they had learned "an awful lot" from it. He
concluded that monkeys "are not random generators. They’re more complex
than that. … They were quite interested in the screen, and they saw
that when they typed a letter, something happened. There was a level of
intention there."[24][26]

Be Sociable, Share!

Comments

comments

54 comments    Share Share    Print    Email

{ 27 comments }

Chris Meisenzahl May 9, 2007 at 10:58 am

Great point about references, I've long thought the same thing.

The Epicurean Dealmaker May 9, 2007 at 11:32 am

This is old ground, however amusing.

For my possibly definitive treatment of this meme, please see the following post.

In it, I reveal many secrets heretofore hidden.

Cheers.

Robert Book May 9, 2007 at 11:49 am

The whole monkey thing is a myth. Consider this example from Kittel &
Kroemer, Thermal Physics (1980):

It has been said that "six monkeys, set to strum unintelligently on typewriters for millions of years, would be bound to write all the book sin the British Museum." This statement is nonsense, for it gives a misleading conclusion about very, very large numbers. Could all the monkeys in the world have typed out a single specified book in the age of the universe?

Suppose that 10^10 [10 billion] monkeys have been seated at typewriters throughout the age of the universe, 10^18 s[econds]. This number of monkeys is about three times greater than the present human population of the earth [as of 1980; now it's about 1.5 times the human population]. We suppose that a monkey can hit 10 typewriter keys per second [allowing no time for eating, sleeping, etc. --RAB]. A typewriter may have 44 keys; we accept lowercase letters in place of capital letters. Assuming that Shakespeare's Hamlet has 10^5 characters, will the monkeys hit upon Hamlet?

… the probability that any given sequence of 10^5 characters typed at random will come out in the correct sequence (the sequence of Hamlet) is of the order of

(1/44)^100,000 = 10^-164,345

… the probability that a monkey-Hamlet will be typed in the age of the universe is approximately 10^-164,345. The probability of Hamlet is therefore zero in any operational sense of the event, so that the original statement at the beginning of this problem is nonsense: one book, much less a library, will never occur in the total literary production of the monkeys.

Of course, to get the probability that ANY book of that length will be written, you have to multiply that probability by the number of possible books — but even if you think there have been a billion books written in human history, the probability of any one of them actually being produced by the monkeys will be, as they say, "zero in any operational sense of the event."

The Epicurean Dealmaker May 9, 2007 at 11:59 am

To Mr. Robert Book — Sir, you are a killjoy.

Using thermal physics before lunch time: for shame. What is the internet coming to?

TED

Ironman May 9, 2007 at 12:27 pm

And besides, you're discounting the success the monkeys have had in producing television screenplays!….

Rich Berger May 9, 2007 at 12:30 pm

I don't think the whole monkey thing is really a "myth" so much as an incorrect conclusion reached by not thinking the problem through.

Russ's post hints at the analysis cited by Mr. Book. Mr. Book? Who's the real author?

george May 9, 2007 at 1:00 pm

Russ: you contradict the importance of Taleb's example after implying there's something profound there. I saw that clip of you talking at Cato, and there you mentioned the Taleb anecdote about a turkey's fallibly extrapolating his owner's affection (which of course will fail miserably on Thanksgiving). But in the Black Swan Taleb harps on the 'narrative fallacy', or the tendency of people to create theories that help them explain. So a theory is good in understanding data (eg, Thanksgiving), except when its bad (when one's theory is really just a 'narrative fallacy' to comfort one against the coldness of random luck).

Kindof like Blink, where Gladwell argues that snap judgements are good, except when they aren't.

Lee May 9, 2007 at 1:34 pm

Regarding the probability that 10 billion Monkeys, typing on 10 billion typewriters, from the beginning of the universe, would write all the books in the British museum.

I will not argue with Kromer's math, but I will point out that this is contigent blind selection criteria. If we introduce a filtering process, whereby sentences are retained depending on how closely they match the books in question, then the probability is altogether more plausable.

In fact, this latter possibility has already occured, or else we wouldn't have all the books in the British museum. Richard Dawkins pints this out in the Blind Watchmaker, since random "guesses" by unthinking "agents" have already written every book in the British museum.

Robert Book May 9, 2007 at 2:46 pm

First, a brief correction. I accidentally gave the same number, 10^-164,345, for both the probability that any given sequence of 10^5 characters would be Hamlet, and for the probability that such a sequence would occur in the age of the universe. That number is correct for any given sequence; the probability that such a sequence would occur in the age of the universe is actually 10^-164,316. This is assuming there are 10^10 monkeys, hitting 10 keys per second (on average), and that the age of the universe is 10^18 seconds. This number differs from the other number by a factor of 29, but they are both zero for all practical purposes.

TED: One of the reason's I keep Kittel and Kroemer's book close at hand is so I can refer to this example, even before lunchtime. ;-) I think I've looked at the rest of the book only once or twice since taking the course. :-(

Rich Berger: I don't want to quibble over terminology, but I just meant "myth" in the sense of "something a lot of people believe but can't possibly be true." They may — and in this case almost certainly do — believe it because they haven't really thought the problem through — which was the whole point of the example I posted.

I'm not sure who the original author is, but I'd give credit to Kittel & Kroemer, if for no other reason that they cite three other sources in that discussion, none of which offer a comparable argument. They cite the original monkey claim to J. Jeans, Mysterious Universe, Cambridge University Press (1930), p. 4; they say that book attributes the claim to Huxley. (I have not checked that book.)

Lee: If you think filtering sentences would solve the problem, I don't think you understand how small a number 10^-164,345 is. Even the probability that any given sequence of 100 characters would match any given 100-character sequence in Hamlet is only 10^-135. That's a decimal point, followed by 134 zeros, followed by a 1. And that's the chance that you'd get any given sentence in the entire age of the universe.

As for Dawkins' claim that the random monkeys idea is correct because that's how we got the books in the British Museum, the kindest thing that can be said that is that he assumes what he is trying to prove. It would be at least as valid to say that the books in the British Museum are proof that God created intelligent life — which is surely not something Dawkins would agree with.

Geoffrey Brand May 9, 2007 at 3:52 pm

This post reminds me of one of my favorite all time Dilbert comics..
It went something like this..

intern: what did you think of my report?

Dilbert: It was once said that if you had an infinate amount of monkeys on typewiters and an infinate amount of time, they would write all of the great works of Shakespeare.

intern: What about my report?

Dilbert: three monkeys, ten minutes.

Lee May 9, 2007 at 4:48 pm

"As for Dawkins' claim that the random monkeys idea is correct because that's how we got the books in the British Museum, the kindest thing that can be said that is that he assumes what he is trying to prove. It would be at least as valid to say that the books in the British Museum are proof that God created intelligent life — which is surely not something Dawkins would agree with."

First, it is probably not wise to attribute to Dawkins the argument I just made. I cannot remember precisely what Dawkins did or didn't say, so everything I say should be taken as my own argument.

Second, the chances of any random string of letters matching a paragraph of Hamlet may be inredibly small, but if adaptive variations are retained then the number becomes far more managable.

For example, take a string of 100 characters (including spaces) from Hamlet, then randomly guess 100 characters (from a set of 50 letters, numbers, symbols, etc.). There is a 1 in 50 chance that any give entry in Macbeth will match the entry on our random string.

If adaptive variations can be retained, the number of tries necessary before we have a full match dwindles considerably. In fact, the above example suggests that we should expect 2 matches on the first few attempts, though only 1 in every 50 attempts by the last.

Lee May 9, 2007 at 4:56 pm

(note: Of course, I know that I'll not match a sentence in Hamlet while looking in Macbeth. *shrugs* never very fond of Shakespeare anyway).

Peter Sidor May 9, 2007 at 5:02 pm

The problem with all those numbers was noted right at the end: "…monkeys are not random generators. … There was a level of intention there."

That still doesn't mean the results of their activities can be interpreted by human standards. :)

Lee May 9, 2007 at 5:11 pm

Oh, and as I originally pointed out. The process I am describing, with a few small alterations, has *already* written every book in the British museum, and far more.

happyjuggler0 May 9, 2007 at 5:46 pm

What are the odds that a random human could in a billion years write Hamlet without access to Hamlet to copy from?

The correct correlation is the odds of writing a tome worth reading, with "worth" of course being subjective.

T Sowell fan May 9, 2007 at 7:02 pm

Can monkeys realistically type at 10 characters/second? Great human typists — having greater manual dexterity than most (every?) monkey — type 80 words/minute. At an average word length of 5 characters — throw in another for the spaces between words if you want, great typists only achieve 8 characters/second.

Don't forget those opposable thumbs.

The Epicurean Dealmaker May 9, 2007 at 7:39 pm

Forget monkeys. If you really want to bake your noodle, read "Pierre Menard, Author of the Quixote," by Jorge Luis Borges.

It tells the story of Menard, a minor Symbolist poet and critic who undertook the heroic task of composing Don Quixote:

He did not want to compose another Quixote—which is easy—but the Quixote itself.

I cannot do the story justice in this short space, but it makes the combinatorial mathematics we have been discussing seem trivial.

Highly recommended, for those in search of spurs to thought. The rest of Borges is better than People magazine, too.

Henri Hein May 9, 2007 at 9:14 pm

Since we are on Jorges, also read "The Library of Babel." Its application to this thread could not be more spot-on.

Robert Book May 9, 2007 at 11:10 pm

Lee writes:

For example, take a string of 100 characters (including spaces) from Hamlet, then randomly guess 100 characters (from a set of 50 letters, numbers, symbols, etc.). There is a 1 in 50 chance that any give entry in Macbeth [meaning Hamlet?] will match the entry on our random string.

There is a 1/50 chance that any ONE entry will match. But the chance that they ALL match is (1/50)^100, or 1.27*10^-170. If it takes 10 seconds to type 100 characters, and the universe is 10^18 seconds old, the monkeys type about 10^17 100-character sequences each. Since there are 10^10 monkeys, that's 10^27 sequences total. Assuming they are all independent (and they are), the chance that they matching any ONE given 100-character sequence is 1.27*10^-143.

Suppose we give the monkeys a break and let them match ANY of the 100-character sequences in Hamlet,not just one given sequence.
Now, assuming Hamlet has about 10^5 100-character sequences (one starting at each character except the last 99), then the probability that AT LEAST ONE of the 10^27 sequences typed by the monkeys matches AT LEAST ONE of the 10^5 sequences in Hamlet is (1-P{none match}). P{none match}=(1-P{one match})^(10^5)). And P{one match}=1.27*10^-143. So we have (1-(1-(1.27*10^-143))^(10^5)).

In other words, this is NOT going to happen.

Lee further writes:

Oh, and as I originally pointed out. The process I am describing, with a few small alterations, has *already* written every book in the British museum, and far more.

What's the basis for claiming that? If the process you are describing happened, we wouldn't have the British Museum. We wouldn't have anything. So either some other process is happening — i.e., non-random typing — or the human race is WAY older than the universe!

Which of those do you think is more likely?

Ray G May 10, 2007 at 12:53 am

RE: References

Asking a contractor for references is an absurd notion anyway. Of course he can produce three or four decent reviews.

Word of mouth is the best way to find any service or business. I have one or two good people for everything I need. If I don't have a good recommendation on someone, I just haven't needed that kind of service yet.

An aside; I'm half way through "The Black Swan" and it is an excellent read. But does anyone know why he has such a dislike for Richard Posner? He framed his comments in what could be construed as a good natured ribbing, but one doesn't put to best selling print such remarks unless there's really is a rub present.

Lee Kelly May 10, 2007 at 4:31 pm

Robert Book,

I had written a longer response to try and explain (once more) what I am trying to say, because you have misunderstood if you believe any of what you have written contradicts my argument. In a nutshell:

(A) This sentence is probable
(B) nhypibfijb ywvspn coerg v

A is the sentence we wish to match and B is our random sequence. Here the matches are "h" from "This" and "o" from "probable." These adaptive guesses are retained, so the next trial these letters are fixed:

(A) This sentence is probable
(B) ;hbepcrn[a'anf'r844o'flni

On this trial we have matched "n" from "sentence," and this too is retained. By this method we can arrive at the desired sentence in an altogether less improbable time frame; the equivalent of rewarding or punishing the monkeys for better or worse guesses.

The broader point, is that all life evolved in this manner (with a few modifications), by random trial, error and selection. That all human knowledge is baseless; there is no solid foundation, only guesswork, but instead an indefinite selection process. The implication is that all the books in the British Museum, ever book ever written, you, me, this post, Cafe Hayek are only possible because the process I am describing works, and does not depend on such remarkable improbabilities.

Lee Kelly May 10, 2007 at 4:35 pm

Note: I lost the original message I intended to post.

The Epicurean Dealmaker May 10, 2007 at 5:51 pm

Lee Kelly — I may be missing something implicit in your argument, but I believe it breaks down (at least) when it gets to intellectual discourse, human culture, etc.

The core problem for these social constructs is this: what is the selection mechanism? Or, put another way, who decides that your random sentence iterations more closely match the (Platonic?) model you (someone?) have in mind? Who or what "punishes" or "rewards" the monkeys? And why?

One can make the argument that organic life on this planet did indeed undergo such a "selection" process, given initial conditions, physical constraints in the environment, an element of chance, etc., etc. Some call this natural selection.

But, while a blue whale may be "inevitable" in some sense on this planet, I can see nothing inevitable about Shakespeare, or even Cafe Hayek. No-one is sitting in a back room somewhere generating limitless variations on words, letters, ideas, etc. for someone to select from.

Perhaps you have some weak- or strong-form Hegelianism in mind, in which intellectual "progress" is inevitable, and proceeds along a predetermined path (determined how: by its own internal "logic," or by some exogenous entity?). For all I know, you may be right: there is no such thing as free will, and we are all plodding relentlessly and unknowingly down a path which has been set for us from the beginning.

But that is a matter of faith, not evidence. I prefer to believe something else.

Python May 10, 2007 at 7:57 pm

Lee,

Unless you believe that God is actively participating in the monkey experiment, there is no external agent which could possibly filter using the method that you suggest. How could it possibly know which "correct" letters to keep in place? There is no such thing as "correct" because Shakespeare hadn't been written yet (assuming we are using the typical time dimensionality we are accustomed to).

If there was a filtering agent, you are correct.

Your submission reminds me of these movies (Terminator 2?) where the tech geeks have these password decrypters – say for a PIN. There are 4 decimal positions to complete, and the hacking device is supposedly discovering the digits. But somehow, we see on the screen that it has figured out the first digit, then the second, and until finally it has all the digits and the safe (or whatever they are breaking into) opens. Things don't work like that, they would have to go through all 10^4 positions before finding the right one. It's like the movie is suggesting it only takes 10*4 tries. My pithy conclusion would be: "There is no filtering agent for typewriting monkeys or PINs."

I'm surprised the Unions: a) let monkeys take typewriting jobs away from able bodied humans, b) aren't upset at the working conditions that these pre-Universal monkeys must certainly have been exposed to.

cpurick May 11, 2007 at 12:19 am

"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the Internet, we know this is not true."

– Robert Wilensky, 1997

linda seebach May 11, 2007 at 9:33 am

I'd like to know what happens if researchers deliberately invoke the monkeys' "intention."
Short video loop showing monkey pressing "S" key results in "S" on screen. (Of course you wouldn't have to go all the way to the alphabet in one step.) Loop of monkey seeing a picture of monkey chow with string "chow" to match, monkey typing "chow," getting pellet. New prompt, picture of grape with string to match. New prompt, picture with YouTube button to show string if money has forgotten it. …
Parrots can do this kind of thing, Irene Pepperberg has shown.

ben May 12, 2007 at 6:26 am

I think what Lee is saying is that the books in the British library have been produced, and a mechanism analogous to the selection example he gives produced the humans that went on to produce the books. The random events combined with selection pressure was not in the writing but in the production of the species that would ultimately do the writing. If that is what he is saying, I think he is exactly correct.

Previous post:

Next post: