From Russo, JE and Schoemaker PJH, 1989, Decision Traps: Ten Barriers to Brilliant Decision-Making and How to Overcome Them:
For each of the following items, provide a low and high guess such that you are 90% sure that the correct answer falls between the two. Your challenge is to be neither too narrow (i.e. overconfident) or too wide (i.e. underconfident). If you successfully meet this challenge, you should have 10% misses -- that is, exactly one miss.
1. Martin Luther King Jr.'s age at death
2. Length of the Nile River (miles)
3. Number of countries that are members of OPEC
4. Number of books in the Old Testament
5. Diameter of the moon in miles
6. Weight of an empty Boeing 747 in pounds
7. Year in which Wolfgang Amadeus Mozart was born
8. Gestation period (in days) of an Asian elephant
9. Air distance from London to Tokyo (miles)
10. Deepest (known) point in the ocean (in feet)
Once you have your low and high guesses, scroll down...................
Answers:
1. 39 years
2. 4187 miles
3. 13 countries
4. 39 books
5. 2160 miles
6. 390,000 lb
7. 1756
8. 645 days
9. 5959 miles
10. 36,198 feet
How did you do? I missed two (books in the OT and weight of a Boeing 747), so I was a little bit overconfident, but not too terribly bad. But it was an effort even to do this well; I could easily have missed more.
"Less than 1% [of a sample of over 1,000 people] got 9 or more answers right. Most people missed 4-7 items (a surprise index of 40-70%), indicating a substantial degree of overconfidence.
...Calibration is the degree to which confidence matches accuracy. A decision-maker is perfectly calibrated when, across all judgments at a given level of confidence, the proportion of accurate judgments is identical to the expected probability of being correct. In other words, 90% of all judgments assigned a .90 probability of being correct are accurate, 80% of all judgments assigned a probability of .80 are accurate, etc.
...The most effective way to improve calibration seems to be simple: Stop to consider reasons why your judgment may be wrong."
Scott Plous, The Psychology of Judgment and Decision-Making
Subscribe to:
Post Comments (Atom)
8 comments:
Wow. I was actually concerned that I was going to get them all right partly because you put "overconfidence" in the title of the post. Instead, I missed 5 (#2, 3, 4, 6, and 8). #6 (weight of a 747) I missed by an order of magnitude, incidentally.
I was way overconfident and made my range too narrow. Several of the ones I missed was only by a small margin.
This is a lot harder than it seems like it would be, isn't it?
Well, I was overconfident. I got 3. (MLK, the plane, depth of ocean) The Nile I have no excuse on – I wasn’t even close.
The Bible is a question with a range of answers – 39 is one version (the Protestant one), but, 46 is a valid answer– that is how many books are in the Catholic version. The main Orthodox Christian sects run to about 48, some others work up to 50. (I looked all this up.) My range was low, so it makes no difference to my performance. My range – 15 to 30 –does catch the number of books in the Tanakh (Hebrew Bible) – 24 – the Tanakh combines Ezra and Nehemiah as 1, all 12 of the minor prophets as one book, and has “Samuel”, and “Kings” instead of Samuel 1 & 2 and Kings 1&2.
I calculated the moon’s diameter from the Earth’s diameter and the relative gravity. And still missed it. Apparently the moon is only 3/5ths as dense as Earth, which I didn’t account for.
I also calculated the Tokyo-London distance, but came up just long – I assumed the direct route would be almost transpolar, and got a range of 6k-9k. I then looked it up, and found that the answer provided (5959) is direct – aircraft historically have flown slightly north of the direct track – adding about 500 miles – to avoid passing over North Korea, China, and Far Eastern Russia/Soviet Union.
I thought Mozart was about 25 years earlier than he was. I then only used a 40 year confidence interval. (I didn’t realize he died after the US and French Revolutions, I thought he preceded both.)
Elephant gestation I missed because I “remembered” a 15 month gestation, not a 22 month. My range didn’t include 645 days, the range I did use was out of an abundance of caution, and still was way off.
OPEC annoyed me, because I didn’t just estimate the number, I named them off, got 7(Saudi, Venezuela, Qatar, UAE, Iran, Iraq, Kuwait) , knew there were 4 I was missing(I remembered “11” about halfway through the process - turned out to be Nigeria, Algeria, Libya and Indonesia), for 11, did 10-12 for safety, and still was wrong. Or not. I simply didn’t account for timing. I looked it up, and found there are indeed only 12 currently, and that my “11” would have been right from 1994 through 2007. 13 would have been right from 1975 until 1992 and would have included Gabon, which left in 1994, and Ecuador, which left in 1992 and rejoined in 2007. 13 was also right from 2007 through this January, when Indonesia left, to leave the current count of 12. My last ‘awareness’ of OPEC evidently dated to somewhere between 1994 and 2007 – with aided awareness I would have named Indonesia as in, with Ecuador, Gabon, and Angola as nonmembers. (I also think I would have substituted Bahrain for Algeria – I didn’t even know the latter had any oil.)
“Permanent” Members: (Iran, Iraq, Kuwait, Saudi, Venezuela, Qatar, Libya, UAE, Algeria, Nigeria) - 10
1975-1992 – with Indonesia, Ecuador, and Gabon (13)
1992-1994 – remove Ecuador (12)
1994-2007 – remove Gabon (11)
2007-2008 – add Ecuador back in, and Angola (13)
2009 – remove Indonesia (12)
RVman, I actually had the thought about OPEC that the number may have changed since 1989, but since that was the period of my life I probably knew the most about OPEC (being in school), it wouldn't have mattered.
I also put the number of books in the OT in a confidence interval that topped at 30 because I thought the actual number would be around 20. Oops.
Tam, I was way off on the Boeing one also. 200 tons? That's pretty damn heavy.
I can truthfully say that I didn't spend much time analyzing what the answers could be. I just went with gut instinct and guessed. Of couse, being overconfident,I didn't do very well.
Let's just say I fit in the category you call "most people."
The funny thing is that I told myself that I stink at these things and should pick huge ranges I was 100% sure would include the answer because then I would still have no problem being wrong at least once. But then when I chose the ranges, I didn't want to pick ranges that were too embarrassingly large, even though no one is looking at my ranges. Doh!
My bad estimates for quantities were all too low. That may be an artifact of these particular questions or it may be a good idea to keep in mind for the future.
I think my bad numbers were all too low as well. I think it's easier to pick an almost silly low number for something (the Nile? Sure, that might be only 400 miles long) than to pick a ridiculously high number (5000 miles? give me a break!)
I think my low guess on the airplane was something like 1000 pounds. Was I really thinking that an airplane only weighs as much as 5 or 6 people??
Post a Comment