All throughout the land, scientists laboured to find a way to detect when it had rained. If it rained only sometimes, it was useful: crops would grow, dams would fill with drinking water and house roofs would be clean. But, if it rained too often: floods would come, crops would rot and houses would grow mouldy and mildewy. It would be very useful to everyone to know how often it rained. But sometimes it rained at night! How were the scientists to know if it rained when everyone was asleep?
On one rainy day, Sally, one of the land’s great scientists, noticed something exciting: Every time it rained, the driveways became wet! In fact they stayed wet for hours afterward, particularly at night! A wet driveway might be a great way to tell if it had rained. Tests had to be done!
Sally applied for funding and over a period of a year research assistants stayed up all night in random geographic locations watching for rain and recording the wetness of a random selection of driveways. Pebble, concrete, bitumen, flat, steep, curved, straight…it didn’t matter. It really worked! 100% of the time that it rained, the driveway became wet. 100%! This was sensational. Even the proportion of patients with heart conditions having high cholesterol was not 100% and yet cholesterol tests were a trusted test across the globe. Driveways were available all over the world…the Wet Driveway test was marvelous!
Sally became an international sensation. She was interviewed about her great idea, invited to lecture at universities and at business functions. Soon she was being asked her opinion about all sorts of things with little relationship to either driveways or rain because polls show that she was one of the world’s most trusted “smart people”.
After a while, other scientists were able to publish useful guidelines above which various conditions occurred. On average driveways were wet about twice a week. However, if the driveway was wet more than 5 times in a week then flooding risk was increased by over 250%. Less than 0.5 times a week indicated an 80% increase in the likelihood of drought. Similar thresholds were calculated for plant rot and mildew house. Soon everyone was using the wet driveway test, and the new thresholds. Television stations started to include the wet driveway averages in their newscasts. Even governments and corporations started to use the new statistic in their budget deliberations.
Sally was very pleased that her idea was being developed in such a useful fashion. Some of the applications being developed were ones she had never considered herself. Being a trained scientist she used her frequent television appearances to educate the general public about the proper ways to use the wet driveway test:
“2 wet driveways a week is only an average of course, and a global average at that. Some driveways are located in places where rainfall averages higher or lower than the global average.”
And
“You shouldn’t get too worried if your driveway doesn’t get wet for an entire week, it doesn’t mean you’re in drought straight away. It’s when the average over the year gets below 0.5 that you can say you’re in drought …and don’t forget to adjust for your local averages”.
Despite Sally’s sterling efforts, untrained amateurs continued to use the test based on the simple guidelines only. Still, on balance, Sally felt that using the test simplistically was still an improvement over not using it at all. But the people and even other trained scientists continued to use both the tests and thresholds in an uninformed and simplistic fashion. One week of no wet driveway and people started to install water tanks and buy water. On the 6th day of rain, some people began packing up their houses ready for a flood. “Well it is 100% accurate isn’t it?” Sally tried to explain that the thresholds were based on averages and that “above 5” only indicated an “increased risk” of flood. The 100% referred to the indication of rain and then only in the direction of rain producing wet driveways, not wet driveways producing rain. The 100% did not refer to the flood risk at all. Despite her efforts, some companies started to sell flood preparation kits: “For that dreaded 6th day”.
Other companies began to sense a market opportunity. Market Research showed that people not only wanted to predict the future of floods and droughts using the Wet Driveway test, they wanted to be able to influence it. The companies set their scientists to work to:
“Find a way to keep the wet driveway average between the safe levels of 0.5 and 5 days a week,” said the CEO of one company.
Soon, a famous hairdryer company, managed to develop a “driveway dryer”. “As soon as you get to 4 Wet Driveways, you turn on the Renington Driveway Dryer and you’ll stay below 5, guaranteed!”, claimed the internationally televised advertisement. Renington sold millions of the their patented Driveway Dryers. Renington was also reportedly working on a driveway wetting device which could push the average above 0.5 and, presumably, thereby avoid a drought. Householders started using their garden hose to wet their driveways in an effort to “Ward off the drought.”
Sally immediately saw the problem. People thought that drying or wetting their driveway would influence the weather and therefore their risk of flood or drought. In fact she’d even heard some other trained scientists (possibly working for Renington) saying scientific sounding things like: “evaporation from aqueous covered non-porous surfaces contributes positively to the local humidity, which in turn increases the probability of rain.”
Sally knew she had to educate the puiblic and immediately used her guest television spot on Ellen:
“The Driveway test only tells you if rain has occurred, much like a cholesterol test just tells you how much cholesterol is in your blood. There is no evidence that wetting or drying your driveway will change your risk of flood, plant rot, drought, or whatever,” Sally explained.
Ellen asked her, “So are you saying you were wrong about your test?”. A little shocked by the question, Sally stammered a feeble “No-o, but…”. Ellen interrupted her to announce a commercial break.
Later that day the CEO of Renington offered Sally a highly paid position as Executive Vice-President of Research and a generous research budget guaranteed for the next twenty years. The Driveway Dryer is still a hot seller.
*****************************************
When I catch myself making this typically human correlation-causation error, I find it useful to “Remember the Wet Driveway Test”. Also it helps me identify when people generally or even entire industries are making the same common error. As we know much competitive advantage is found through not making the same mistakes as everyone else. Have you got any classic examples of this common error?