MythBusters,1 a popular television program on the Discovery Channel, is co-hosted by a pair of geeky, middle-aged engineer types in Adam Savage and Jamie Hyneman.2 Building contraptions and using a loose interpretation of the experimental method, Adam and Jamie have addressed topics such as if bulletproof shields are really bullet proof,3 if one’s toes can really be amputated inside of a steel-toed boot,4 and most recently, if ninjas can run on water.5
Usually, these destroyers of wives’ tales are fairly thorough in their analysis, but there seem to be some instances where the busters are themselves busted. Some say their measurement of a daddy long legs’ fang is off,6 some say they use the wrong kinds of arrows,7 and others say their paper is much too thin.8 Granted, such scrutiny is to be expected from armchair critics everywhere, but few are able (or willing?) to design and carry out counter tests.
Occasionally, however, a blunder arises which can be exposed without expensive test fixture designs. Sometimes all you need is a simple understanding of statistics. MythBusters’ investigation into the suspected contagious nature of the everyday yawn9 is one such instance.
The premise of the experiment was to invite a stranger to sit in a booth for a period of time sufficient to bore even the least ADD prone. Fifty subjects were said to be tested in total, of which two-thirds were "seeded" with a yawn by the experiment attendee. Those in the remaining third were given no yawn seed. Using a two-way mirror and a hidden camera, Kari, Scottie and Tory (Adam and Jamie’s young helpers) observed and recorded how many of the fifty yawned and how many did not.
After the experiment coverage, the show documented Kari, Scottie and Tory approaching their elder mentors with the following results:
- 25% yawned of those not given a yawn seed.
- 29% yawned of those given a yawn seed.
Faced with these numbers, the masters of determining truth from error cited the "large sample size" and the 4% difference in the results in confidently concluding the yawn seed had a significant effect on the subjects and, therefore, the yawn is decisively contagious.
Busting the Busters
EDITORIAL NOTE: See this comment thread for an important addendum to the statistical analysis.
The issue, of course, is significant results cannot be determined with one’s gut in many cases, as a cursory observation of sample size and resulting percentage can be misleading. There are, however, simple and straightforward statistical methods to determine whether or not input factors correlate to output results. When applied to the data in this particular MythBusters episode, it does not bode well for Adam and Jamie.
Sample Size Analysis
The first order of business is to extrude more details from the reported results. The available information includes:
- 25% yawned of those not given a yawn seed.
- 29% yawned of those given a yawn seed.
- Approximately 50 subjects were tested.
- Two-thirds were seeded (given a yawn seed).
- One-third was not seeded.
Next, the following variables are assigned:
- x – subjects not seeded with a yawn
- y – subjects seeded with a yawn
Transferring these variables into the above information yields:
- x + y ~ 50
- .25_x_ = close to a whole number
- .29_y_ = close to a whole number
- x ~ 50 / 3
- y ~ 50 * 2 / 3
Taking various values of x and y that might have been construed into these results, the most fitting set seems to confirm the sample size of 50, with 34 seeded subjects were given a yawn seeded.
Thus, the results are likely distributed as follows:
- 4 yawned out of 16 subjects not given a yawn seed
- 10 yawned out of 34 subjects given a yawn seed
- 14 total yawns out of 50 subjects
Correlation analysis is the perfect tool to determine if these results indicate a yawn seed would significantly alter the likelihood of a subject yawn. The indicated factor in this analysis is the correlation coefficient. Continuing with the previous denomination of those given the yawn seed as series A and those who yawned as series B, the following calculations apply:10
Covariance(A,B)is the sample covariance between A and B,
Variance(A)is the sample variance of A, and
Variance(B)is the sample variance of B.
These calculations result in a correlation coefficient of .045835 between the two series, or between those given the yawn seed and those who actually yawned. Sorry Adam and Jamie, this indicates no correlation between the two series. (A value of at least 0.1 is needed for even a weak correlation.11)
Where MythBusters Went Wrong
If 29% is not considered beyond the reach of chance with respect to 25% in a sample set of 50, what is? While it may initially seem an increase in the sample size might help, if the percentages remain the same, the correlation coefficient does not move. The only change that would make Adam and Jamie’s assessment correct, then, would be if the percentages were further apart.
Assuming the sample size remained at 50 subjects, two-thirds of whom were seeded with yawns, adding one yawner in the seeded group raises the correlation coefficient to .074848. Another needs to be added before the coefficient breaks into the significant range, at .102941. On the other hand, if only one yawner is taken out of the non-seeded group, the figure jumps over the threshold to .113385. Thus, the following two conditions satisfy the requirements for statistically significant results:
- 4/16 (25.00%) of those not seeded yawn, and 12/36 (33.33%) of those seeded yawn.
- 3/16 (18.75%) of those not seeded yawn, and 10/36 (29.41%) of those seeded yawn.
It appears a percentage difference at least in the range of 8-10% is required given MythBusters’ setup – double the 4% found in the actual experiment performed and so embarrassingly interpreted in front of millions of viewers.
MythBuster’s Reaction Since
It is difficult to see how these results could have been missed – if not at the time, then at least in hindsight. This is a big-time television program with a whole staff of editors. Surely _some_one in back raised their hand in protest _some_where along the way to point out _some_thing just did not seem right. Not that they would publish a recall or anything, but at least they could avoid the subject when in public
Quite to the contrary, calling on the results of this yawn contagiousness "proof," MythBusters launched a campaign in September 2006 to try and send a yawn "around the world."12 Not only that, they invited browsers to "catch" a yawn by viewing a video of Adam on YouTube,13 while claiming responsibility for discovering the science behind the alleged phenomenon:
The MythBusters identified that yawning is officially contagious, but we want to go one step further and involve as many people as possible in our biggest ever experiment. If only one per cent of the global population took part in the Yawn Around The World experiment then 65 million people would have yawned across the globe, which would be an amazing achievement.
The odd thing is, both may very well work. While the MythBusters should be ashamed to fall victim to such an obvious statistical blunder, it is still very possible that yawns are contagious. If they are, and if these experiments succeed, however, it most definitely will not be because MythBusters had anything to do with "officially" determining the contagiousness of a yawn – not with a correlation coefficient of .045835.
1 "MythBusters." Discovery Channel. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/mythbusters.html.
2 "MythBusters: Bios." Discovery Channel_. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/meet/meet_main.htmlmain.html.
3 "Episode 16: : Ancient Death Ray, Skunk Cleaning, What Is Bulletproof?" MythBusters_. Aired September 29, 2004. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_07.html07.html.
4 "Episode 42: Steel Toe-Cap Amputation." MythBusters_. Aired November 9, 2005. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_02.html02.html.
5 "Episode 78: Walking on Water." MythBusters. Aired April 25, 2007. Accessed April 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/episode.html.
6 "Daddy Long Legs – Great Moments in Science – The Lab." ABC.net.au. Accessed March 2007 from http://www.abc.net.au/science/k2/moments/s1721788.htm.
7 "Mythbusters are dead wrong…" Sword Forum International. Accessed March 2007 from http://forums.swordforum.com/showthread.php?threadid=70841.
8 "Paper folding." Discovery Channel Fansite. Accessed March 2007 from http://community.discovery.com/eve/forums/a/tpc/f/9801967776/m/9041918678.
9 "Episode 28: Is Yawning Contagious?" MythBusters_. Aired March 9, 2005. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_05.html05.html.
10 Weiss, Neil A. "Descriptive Methods in Regression Correlation." Elementary Statistics, 4th ed. Addison Wesley Longman, Inc., 1999. p.195-246. See also: "Correlation Coefficient." WolframMathWorld. Accessed April 2007 from http://mathworld.wolfram.com/CorrelationCoefficient.html.
11 Cohen, J. Statistical power analysis for the behavioral sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates. 1988.
12 "A new yawn for MythBusters." News.com.au. Accessed April 2007 from http://www.news.com.au/entertainment/story/0,23663,20471697-10388,00.html.
Similarly tagged OmniNerd content: