Friday August 3, 2007

Thinking and Leaping

by Garrett Dimon in Design

2 comments

At last year’s Webmaster Jam Session, I went to Cameron Moll’s presentation, Nine Skills that Separate Good and Great Designers. While the presentation as a whole was incredible, there was one set of slides that really resonated with me.

He told the story of an statistician by the name of Abraham Wald who attended university in Austria. In 1938, the Germans invaded Austria and he was invited to do research in the U.S. He ended up teaching at Columbia where he took on some war work.

In one scenario, they were looking to reinforce the planes with additional armor. So, they collected data from planes as they returned from battle and visually plotted the location of bullet holes on the plane. They initially assumed that each spot on a plan was just as likely to be hit as any other and that they would see a uniform distribution of bullet holes. However, the result was a graphic that looked something like the image below.

Looking at the graphic, if you were the military and looking to reinforce your planes, where would you put the armor? At first glance, it seems like a no-brainer. We can now see that the bullet holes are not uniformly distributed and that we need to reinforce the areas of the plane that we have shown are most likely to be hit by bullets.

However, there’s a catch. What about the planes that didn’t return? It’s much more likely that the areas of the plane without any bullet holes are lacking simply because the planes that represented those spots never returned from battle. Those are the locations where the planes are most vulnerable, because the planes that are hit there generally don’t make it back.

Thinking Before We Leap

This story stuck with me. It seems at least once a week, I observe a person or team that exhibits this exact same behavior. In their exuberance to solve a problem, the data or information is taken at face value, without critical analysis. And despite having the best intentions, the wrong conclusion is made. It seems a little more critical and inductive thinking can go a long way in helping avoid this problem.

Thanks to Cameron Moll for so graciously sharing the images and the story.

Comments on “Thinking and Leaping”

1. Ralph Leonen said:

   Left unsaid is the fact that, by following the intuitive conclusion and armoring where the bullet holes are concentrated, the attendent loss of speed and manueverability would probably result in more planes lost after reinforcement than before. Thanks for the posting the beautiful example.

   ---

2. Rob Campbell said:

   Further to Ralph’s comment, another consideration is that some areas can probably withstand the bullet strikes and so could be left unarmored in order to save weight—no point putting armor where it isn’t needed even if the enemy does tend to hit the plane there. And then, when you do add armor to the other places the enemy is going to learn that and start shooting at some other vulnerable point. But that aside the initial observation that the planes that didn’t return provide the most valuable data is clever and cool. Thanks!

   ---