Book review: The Drunkard's Walk
The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow.
The human brain has a powerful capability to spot patterns. It's so good at spotting patterns, it can spot patterns that aren't even there. It's this fact that makes randomness--the topic of Mlodinow's new book--so interesting and so confusing.
Let's say you're a subject in this experiment: A scientist has rigged up a light to flash red or green, and arranged things so that it flashes red twice as often as green. You guess whether it will be red or green--and you get a reward every time you're right. What do you do?
Well, if you're a rat, you observe that the red is the most common and then always guess red--that maximizes the reward. If you're a human, though, you study the sequences and try to figure out a pattern. If there is a pattern, and it's not too complex for you to figure it out, this can be a winning strategy--you can get the reward every time. If there is no pattern, though, you'll be routinely beaten by the rat.
Mlodinow's book is a fascinating excursion into the realm of random events, and a surprisingly interesting history of the mathematical tools that people have come up with to analyze events that are or might be random. (It's also the only book that has ever made me think, "Gee, I should get a good statistics text--the math here is really interesting.")
These issues matter in the real world. Take, for example, drug testing. Let's say you have a test for (illegal or performance-enhancing) drug users that's a pretty good test--it detects 90% of the drug users while only fingering an innocent person 0.1% of the time.
Suppose you give this test to 100 student athletes at a school who are suspected of drug use. If 20 of the students actually are drug users, odds are that you'll catch 18 of them, and do so without accusing an innocent student.
Suppose, though, that you give the same test to a much larger population that includes very few drug users--perhaps a large technical facility with 20,000 scientists and engineers, only a dozen of whom are drug users. Look at what your results are now: You successful detect 10 or 11 of the drug users, but you also "detect" 20 people who aren't drug users at all--you've got more false positives than true positives! (People who don't understand randomness make this sort of mistake all the time.)
In addition to providing an interesting walk through the history of people's understanding of randomness, and an introduction to the mathematics of dealing with these problems, Mlodinow provides some useful tips on being successful in a world where random chance has a lot to do with success. (The gist of which is summed up in a quote from IBM pioneer Thomas Watson: "If you want to succeed, double your failure rate.")
Some things in life are random and other things aren't. The Drunkard's Walk is filled with interesting stories of just how far astray people can go when they don't have the tools and the insight to figure out which situation they're in.
Editor's note: Mr. Mlodinow has generously offered 7 of our lucky readers a free copy of his new book. For a chance to win, just pop over to this forum thread before 6/30/08 and leave us a comment.