## Testing MythBusters' Yawn Contagiousness Data

MythBusters,^{1} a popular television program on the Discovery Channel, is co-hosted by a pair of geeky, middle-aged engineer types in Adam Savage and Jamie Hyneman.^{2} Building contraptions and using a loose interpretation of the experimental method, Adam and Jamie have addressed topics such as if bulletproof shields are really bullet proof,^{3} if one’s toes can really be amputated inside of a steel-toed boot,^{4} and most recently, if ninjas can run on water.^{5}

Usually, these destroyers of wives’ tales are fairly thorough in their analysis, but there seem to be some instances where the busters are themselves busted. Some say their measurement of a daddy long legs’ fang is off,^{6} some say they use the wrong kinds of arrows,^{7} and others say their paper is much too thin.^{8} Granted, such scrutiny is to be expected from armchair critics everywhere, but few are able (or willing?) to design and carry out counter tests.

Occasionally, however, a blunder arises which can be exposed without expensive test fixture designs. Sometimes all you need is a simple understanding of statistics. MythBusters’ investigation into the suspected contagious nature of the everyday yawn^{9} is one such instance.

## Experiment

The premise of the experiment was to invite a stranger to sit in a booth for a period of time sufficient to bore even the least ADD prone. Fifty subjects were said to be tested in total, of which two-thirds were "seeded" with a yawn by the experiment attendee. Those in the remaining third were given no yawn seed. Using a two-way mirror and a hidden camera, Kari, Scottie and Tory (Adam and Jamie’s young helpers) observed and recorded how many of the fifty yawned and how many did not.

## MythBusters’ Conclusion

After the experiment coverage, the show documented Kari, Scottie and Tory approaching their elder mentors with the following results:

- 25% yawned of those not given a yawn seed.
- 29% yawned of those given a yawn seed.

Faced with these numbers, the masters of determining truth from error cited the "large sample size" and the 4% difference in the results in confidently concluding the yawn seed had a significant effect on the subjects and, therefore, the yawn is decisively contagious.

## Busting the Busters

EDITORIAL NOTE: See this comment thread for an important addendum to the statistical analysis.

### Statistical Significance

The issue, of course, is significant results cannot be determined with one’s gut in many cases, as a cursory observation of sample size and resulting percentage can be misleading. There are, however, simple and straightforward statistical methods to determine whether or not input factors correlate to output results. When applied to the data in this particular MythBusters episode, it does not bode well for Adam and Jamie.

### Sample Size Analysis

The first order of business is to extrude more details from the reported results. The available information includes:

- 25% yawned of those not given a yawn seed.
- 29% yawned of those given a yawn seed.
- Approximately 50 subjects were tested.
- Two-thirds were seeded (given a yawn seed).
- One-third was not seeded.

Next, the following variables are assigned: *x*– subjects not seeded with a yawn*y*– subjects seeded with a yawn

Transferring these variables into the above information yields:

*x*+*y*~ 50- .25_x_ = close to a whole number
- .29_y_ = close to a whole number
*x*~ 50 / 3*y*~ 50 * 2 / 3

Taking various values of*x*and*y*that might have been construed into these results, the most fitting set seems to confirm the sample size of 50, with 34 seeded subjects were given a yawn seeded.

Thus, the results are likely distributed as follows:

- 4 yawned out of 16 subjects not given a yawn seed
- 10 yawned out of 34 subjects given a yawn seed
- 14 total yawns out of 50 subjects

### Correlation Analysis

Correlation analysis is the perfect tool to determine if these results indicate a yawn seed would significantly alter the likelihood of a subject yawn. The indicated factor in this analysis is the correlation coefficient. Continuing with the previous denomination of those given the yawn seed as series *A* and those who yawned as series *B*, the following calculations apply:^{10}

where

`Covariance(A,B)`

is the sample covariance between*A*and*B*,`Variance(A)`

is the sample variance of*A*, and`Variance(B)`

is the sample variance of*B*.

These calculations result in a correlation coefficient of .045835 between the two series, or between those given the yawn seed and those who actually yawned. Sorry Adam and Jamie, this indicates *no correlation* between the two series. (A value of at least 0.1 is needed for even a weak correlation.^{11})

### Where MythBusters Went Wrong

If 29% is not considered beyond the reach of chance with respect to 25% in a sample set of 50, what is? While it may initially seem an increase in the sample size might help, if the percentages remain the same, the correlation coefficient does not move. The only change that would make Adam and Jamie’s assessment correct, then, would be if the percentages were further apart.

Assuming the sample size remained at 50 subjects, two-thirds of whom were seeded with yawns, adding one yawner in the seeded group raises the correlation coefficient to .074848. Another needs to be added before the coefficient breaks into the significant range, at .102941. On the other hand, if only one yawner is taken out of the non-seeded group, the figure jumps over the threshold to .113385. Thus, the following two conditions satisfy the requirements for statistically significant results:

- 4/16 (25.00%) of those not seeded yawn, and 12/36 (33.33%) of those seeded yawn.
- 3/16 (18.75%) of those not seeded yawn, and 10/36 (29.41%) of those seeded yawn.

It appears a percentage difference at least in the range of 8-10% is required given MythBusters’ setup – double the 4% found in the actual experiment performed and so embarrassingly interpreted in front of millions of viewers.

## MythBuster’s Reaction Since

It is difficult to see how these results could have been missed – if not at the time, then at least in hindsight. This is a big-time television program with a whole staff of editors. Surely _some_one in back raised their hand in protest _some_where along the way to point out _some_thing just did not seem right. Not that they would publish a recall or anything, but at least they could avoid the subject when in public

Quite to the contrary, calling on the results of this yawn contagiousness "proof," MythBusters launched a campaign in September 2006 to try and send a yawn "around the world."^{12} Not only that, they invited browsers to "catch" a yawn by viewing a video of Adam on YouTube,^{13} while claiming responsibility for discovering the science behind the alleged phenomenon:

The MythBusters identified that yawning is officially contagious, but we want to go one step further and involve as many people as possible in our biggest ever experiment. If only one per cent of the global population took part in the Yawn Around The World experiment then 65 million people would have yawned across the globe, which would be an amazing achievement.

The odd thing is, both may very well work. While the MythBusters should be ashamed to fall victim to such an obvious statistical blunder, it is still very possible that yawnsarecontagious. If they are, and if these experiments succeed, however, it most definitely willnotbe because MythBusters had anything to do with "officially" determining the contagiousness of a yawn – not with a correlation coefficient of .045835.

## Notes

^{1} "MythBusters." *Discovery Channel*. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/mythbusters.html.

^{2} "MythBusters: Bios." *Discovery Channel_. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/meet/meet_main.html*main.html.

^{3} "Episode 16: : Ancient Death Ray, Skunk Cleaning, What Is Bulletproof?" *MythBusters_. Aired September 29, 2004. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_07.html*07.html.

^{4} "Episode 42: Steel Toe-Cap Amputation." *MythBusters_. Aired November 9, 2005. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_02.html*02.html.

^{5} "Episode 78: Walking on Water." *MythBusters*. Aired April 25, 2007. Accessed April 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/episode.html.

^{6} "Daddy Long Legs – Great Moments in Science – The Lab." *ABC.net.au*. Accessed March 2007 from http://www.abc.net.au/science/k2/moments/s1721788.htm.

^{7} "Mythbusters are dead wrong…" *Sword Forum International*. Accessed March 2007 from http://forums.swordforum.com/showthread.php?threadid=70841.

^{8} "Paper folding." *Discovery Channel Fansite*. Accessed March 2007 from http://community.discovery.com/eve/forums/a/tpc/f/9801967776/m/9041918678.

^{9} "Episode 28: Is Yawning Contagious?" *MythBusters_. Aired March 9, 2005. Accessed March 2007 from http://dsc.discovery.com/fansites/mythbusters/episode/00to49/episode_05.html*05.html.

^{10} Weiss, Neil A. "Descriptive Methods in Regression Correlation." *Elementary Statistics, 4 ^{th} ed.* Addison Wesley Longman, Inc., 1999. p.195-246. See also: "Correlation Coefficient."

*WolframMathWorld*. Accessed April 2007 from http://mathworld.wolfram.com/CorrelationCoefficient.html.

^{11} Cohen, J. *Statistical power analysis for the behavioral sciences*, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates. 1988.

^{12} "A new yawn for MythBusters." *News.com.au.* Accessed April 2007 from http://www.news.com.au/entertainment/story/0,23663,20471697-10388,00.html.

^{13} “Yawning is contagious.” *YouTube.com*. Accessed April 2007 from http://www.youtube.com/watch?v=Cy-Pf6oJNRo.

Similarly tagged OmniNerd content:

- Advanced Crypto Present in Flame, by VnutZ almost 4 years ago
- John Nash Letters Declassified, by VnutZ about 4 years ago
- Mathematics of Pancake Flipping, by VnutZ over 4 years ago
- Mythbusters: Self Propelled Sailboat, by VnutZ almost 5 years ago

## Correlation correlation by gnifyus

O.K., here’s a question about correlation coefficients in general.

Say you somehow repeated this same experiment 50 times, and 45 out of 50 times the percentages came out with a correlation coefficient of something below the .10 needed for a weak correlation, but at the same time in favor of the yawns being contagious. In other words the correlation coefficient of whether each experiment’s results were majority or minority would show a high correlation. What would this say about yawn contagiousness then?

Or is this considered statistically impossible to actually happen, given the CC’s obtained from the first experiment?

## Discussion on Slashdot by joeljkp

This article is being challenged on Slashdot.

## Completely Flawed. by Anonymous

You do NOT use descriptive statistics to study a sample, you need a completely different way of approaching things, namely, statistical analysis.

What you really need to do is Hypothesis testing (http://en.wikipedia.org/wiki/Statistical_hypothesis_testing), and test whether the hypothesis that more people yawn when seeded than when not seeded.

Your analysis is completely flawed, ask any statistician.

## hypothesis testing and dichotomous variables by Anonymous

Well, it is flawed, but not totally wrong. I coded the data set that was used on the web page and ran it through a chi-square/McNemar+Risk Estimate test (appropriate tests for dichotomous treatment variables + dichotomous outcome variables). No significant difference alpha=.744. But shame on you for using a straight up correlation. —chris

## More (and perhaps more appropriate) statistical analysis by Brandon

I received a number of emails concerning the statistical method I used (Pearson’s correlation coefficient), which provided some insight but does not sufficiently address the issue of causation in the results. Personally, I don’t understand how there can be so obviously

nota correlation between two variables and there still be a chance there is causation involved, but with the aim of statistical appropriateness, I have included a number of alternative statistical methods below.Association TestAn association test such as Fisher’s Exact Test is appropriate. This method is specifically for determining any non-random association between two

categorical(discrete) variables – which is exactly what we have in this instance. Its use, then, removes any issues there may have been in the Pearson analysis having to do with the data set not being continuous.For those interested, the calculations are described in the link above. The results are easy to come by, however, using online tools such as this calculator at Matforsk.com. Inserting the MythBuster’s data results in the following:

This corresponds to there being 4 non-seeded subjects who yawned, 10 seeded who yawned, 12 non-seeded to didn’t yawn, and 24 seeded who didn’t yawn. The resulting p-values are all well above the commonly accepted limit of .05 for significance.

Confidence Interval for the Difference in RatesThis method was recommended via email by Max Kuhn, a "Ph.D. statistician who works in industry." Max provided a very thorough and helpful analysis of the data, which I’ve included below:

Linear Regression to Show Sample Size Needed for SignificanceI received yet another very friendly and helpful email from Zinj Boisei who pointed out I was too hasty in dismissing the use of an increased sample size. By using a more appropriate analysis, linear regression in this case, Zinj confirmed there was little significance at the sample size of 50 – and even went on to find out large a sample size of the same makeup would need to be for the results to be significant:

Conclusion AddendumWhile the statistical method used in the article was sufficient to show the yawn seed was responsible for a negligible amount of the variance, methods such as association tests, confidence interval analysis and linear regression provide more appropriate insight into the causation involved. In this case, all tests lend credence to the original conclusion: the results of the MythBuster’s yawn experiment

did notsupport their conclusion.## Simple Calculations with the Chi-Square Tests by Anonymous

I preformed a Chi-Square Test on a TI-84 Silver Edition calculator, so any error in rounding or anything is not my fault, it is the fault of the statistics package. With that out of the way….

I decided to use the data the Myth Buster’s collected on yawning as follows:

I choose to take their percents and apply them to two groups out of 100, one seeded, and one not. I performed the following tests assuming their data is correct, but there is always the chance that it is not. There is also another facter that I will discuss at the bottom.

I took the expected number of people who yawned without being “seeded” to be 25 and the people in the same group who didn’t yawn to be 75.

In the group who were “seeded”, I used the 29% the Myth Buster’s percieved and ended up with 29 yawners, andn 71 not.

H0= A group of people who are seeded WILL NOT yawn more that a group who of people who aren’t seeded,

Ha= A group of people who are seeded WILL yawn more that a group who of people who aren’t seeded

I performed a Chi-Square Goodness of Fit Test on the data with one degree of freedom and if gave me the following:

Chi-Square Value: 0.853333 (three repeating)

p=0.3556110613

df=1

A p-value of 0.356 (rounded) is MUCH too high to reject the null hypothesis. I generally use an alpha-level of 0.05, (alpha levels are generally somewhere from 0.10 to 0.01), but no proper statistitian would ever reject the null hypothesis on a p-value thats as high as 0.356.

In colloquial terms, there is NOT enough evidence to conclude that when a group of people are seeded with a yawner, they are more likely to yawn than otherwise.

Now, here is that other factor I promised to get to:

If the two groups are seeded and unseeded, that means in one group, a person yawns, attempting to get others to yawn, and in another group, they are left on their own as a control group. Here is the problem with that logic.

As soon as one person yawns in the control group (or unseeded group), the rest of the people in that group are now seeded, which makes the group no longer valid as a control group. This is a difficult obstacle to overcome and I will not say that I have the answer, but I will say that it distorts the data.

In the end, my conclusion is thus:

With the data the Myth Busters collected, there is not enough evidence to say that yawns are contagious. However, since their “control” group really isn’t entirely unseeded (only the first yawn is), it makes the rest of the yawns seeded. It is entirely possible that more people yawned after the first person yawned because they were seeded, which means yawns are contagious, but the data (which I have just proven is unreliable) shows that there is no proof that yawns are contagious. I am not saying that yawns aren’t contagious, I am simply saying there is no proof that they are. A good metaphor is like a court. I am not saying they are innocent (or that yawns aren’t conagious), I am simply stating that there is not enough evidence to consider them guilty (or that yawns are contagious). To continue the analogy, the suspect may have actually committed the crime, but without the proper evidence, we cannot convict. So, to sum things up in one sentence:

The data found by the Myth Busters is flawed, but if one truely still believed it to be true, there is still not enough evidence to support their claim that yawns are contagious, so we must stay in the mindset that they are not contagious until unflawed data is produced that proves otherwise.

## Yawning Babies by VnutZ

So I’ve noticed … my kid does not yawn after seeing either of her parents yawn. But we certainly do after seeing her yawn. My scientifically sound

sample of onetherefore leads me to conclude it’s a learned behavior. :-)