Loading 6 Votes - +

Follow-up to Analysis of 2007 Football All-Americans

Last month I published an article both presenting and analyzing a set of data concerning the recruitment of the 2007 NCAA Division I All-Americans back when they were coming out of high school.1 The analysis included tables and graphs displaying things such as the distribution of star ratings by position. It did not, however, accomplish one of its aims – to see if the star ratings attributed by scouting websites Rivals.com and Scout.com accurately predicted success (or, at least how well they did compared to each other). This failure was largely due to the claimed "unavailability" information on the distribution of star rankings given out in an average class. In other words, because I didn’t know what percentage of a class was made up of 4-star recruits, I couldn’t tell if the number of 4-star recruits in the All-American class was too many, too few, or about what I’d expect.

Luckily, an anonymous commenter pointed out2 an article in Sunday Morning Quarterback (SMQ) responding to my analysis, entitled "Build Me Up, Tear Me Down, Part Three: All-Americans In Repose."3 It offers a good deal of criticism, but more importantly points out a creative way to derive the total number of 3 and 4 star prospects in a recruiting class: simply add up the totals listed on the team rankings page. Although this total does not include players not recruited (e.g., those attending junior colleges for academic reasons, deciding to pursue other sports, etc.), it’s better than nothing.

Additionally, the SMQ post contained quite a number of criticisms, so my motivation to write a follow-up article was re-doubled.

Corrections and Clarifications

As dictated by my analysis-pride, here are the corrections and clarifications I need to make in response to the SMQ article:

  • OmniNerd doesn’t write articles, it just publishes them. I (Brandon Hansen) came up with the previous analysis, and I’m sure there are many sport-hating nerds who frequent OmniNerd.com and would not appreciate being roped into my college football semi-fanaticism.
  • My assumption of a normal distribution was completely arbitrary and for the sake of making a specific point. Nowhere did I suggest it to accurately represent reality. Positioning it as some sort of fundamental premise (rather than the afterthought investigation it was) is at best wrong and at worst dishonest.
  • My comment on the bias found in Figures 54 and 65 was they may manifest a high bias. SMQ incorrectly states I concluded the data was "not biased high."
  • Perhaps most importantly, contrary to _SMQ_’s claim, my comment about the purpose of ratings being to predict college success was in no way tied to a conclusion on whether or not they actually do. Rather, it was presented in a way so as to suggest variables such as coaching and academics should be included in ratings and not serve as excuses.

Overall, it seemed SMQ missed my point. I’m not trying to show the ratings to be nonsensical in any fashion. Rather, I’m interested in collecting data and identifying interesting trends or differences between the competing websites. (Oh, and I like to throw in the occasional hypothetical, as well.)

Further Data

Let’s leave the inaccurate criticisms at that and move on to something more important: What can be done with the data gleaned using _SMQ_’s method (i.e., using team rankings to find star rating totals)?

First, I modified my previous spreadsheet6 to include the Scout.com and Rivals.com 2002-2007 team rankings.7,8 The number of prospects listed as signing on a college team is totaled, as well as the number of 5-, 4- and 3-star prospects:

College Football 2007 Div I All-Americans

Second, I investigated another SMQ notion by creating another spreadsheet leaving out all Kickers/Punters. This spreadsheet is also stored in Google Docs and available publicly.9 Note, however, that only Rivals data is presently included,10 as Scout does not provide team rankings by the kicker position:11

College Football 2007 Div I All-Americans – No Kickers

Further Analysis

I collected my analysis of this new data in three sections:

  • star rating distribution of the recruiting classes according to each website and how these compare to the distribution of the AA class
  • analysis like I provided in my first article, but this time with no kickers
  • star rating distribution again, this time without kickers

Star Rating Distributions

The data on each Rivals and Scout recruiting class (based on the reported team rankings, at least) is organized below in Tables 1 and 2. Calculated percentages representing the distributions of the recruiting class, the All-Americans, and the difference between the two are also presented. (Note rounding may cause some seeming inconsistencies. Consult the online spreadsheet for details.)

4_article_124_thumb_rivals_class_star_count

Table 1. Star rating distribution on Rivals.com by year, averaged, and compared to how Rivals.com rated the 2007 All-American class.

4_article_124_thumb_scout_class_star_count

Table 2. Star rating distribution on Scout.com by year, averaged, and compared to how Scout.com rated the 2007 All-American class.

The item of interest is how various star ratings are represented in the 2007 All-American class as compared to their representation in the average recruiting class. This comparison is made for each website in Figures 1 and 2. Figure 3 displays a head-to-head comparison of the two sites and how their ratings affect the probability a recruit will succeed.

4_article_124_thumb_rivals_rating_distribution

Figure 1. Star rating distribution of the average Rivals recruiting class as compared to the distribution of how Rivals rated the 2007 All-American class.

4_article_124_thumb_scout_rating_distribution

Figure 2. Star rating distribution of the average Scout recruiting class as compared to the distribution of how Scout rated the 2007 All-American class.

4_article_124_thumb_effect_of_star_ratings

Figure 3. Ratio of the representation of a star rating (or group of star ratings) in the 2007 All-American class as compared to the average recruiting class, by rating website.

No Kickers

As stated by SMQ, the data in my original analysis suggests kickers/punters dominate the low-rated players in the 2007 All-American class. It is valuable, then, to perform analysis without those seeming outliers. Below are the same tables and graphs used in the previous analysis, this time without including kicker data.

4_article_124_thumb_rivals_star_count_no_kickers

Table 3: Compiled counts and percentages of the Rivals.com star ratings of 2007 AAs when they were high school recruits (kickers/punters excluded).

4_article_124_thumb_scout_star_count_no_kickers

Table 4: Compiled counts and percentages of the Scout.com star ratings of 2007 AAs when they were high school recruits (kickers/punters excluded)

4_article_124_thumb_rivals_com_star_rating_count_no_kickers

Figure 4: Distribution of Rivals.com star ratings in 2007 AAs when they were high school prospects (kickers/punters excluded)

4_article_124_thumb_scout_com_star_rating_count_no_kickers

Figure 5: Distribution of Scout.com star ratings in 2007 AAs when they were high school prospects (kickers/punters excluded)

4_article_124_thumb_average_star_ranking_no_kickers_table

Table 5: Average star ranking of 2007 All-Americans when recruited, by position (kickers/punters excluded) and ranking website

4_article_124_thumb_average_star_ranking_no_kickers

Figure 6: Average star rating of 2007 All-Americans when recruited, by position (kickers/punters excluded) and ranking website (bar graph)

4_article_124_thumb_average_star_ranking_no_kickers_line

Figure 7: Average star rating of 2007 All-Americans when recruited, by position (kickers/punters excluded) and ranking website (line graph)

4_article_124_thumb_aa_star_count_no_kickers_table

Table 6: Number of 2007 AAs receiving each star rating when recruits (kickers/punters excluded) and percentage of total number of 2007 AAs

4_article_124_thumb_aa_star_count_no_kickers

Figure 8: Number of recruits (kickers/punters excluded) in each star level from the 2007 All-Americans (bar graph)

4_article_124_thumb_aa_star_count_no_kickers_line

Figure 9: Number of recruits (kickers/punters excluded) in each star level from the 2007 All-Americans (line graph)

Star Rating Distributions (without Kickers)

The star rating distributions of the no-kicker data is presented below in similar tables and graphs used in the "Star Rating Distributions" section above. Scout data is not included for the aforementioned reason. Figure 12 combines Figures 3 and 11 for comparative purposes.

4_article_124_thumb_rivals_class_star_count_no_kickers

Table 7. Star rating distribution on Rivals.com by year, averaged, and compared to how Rivals.com rated the 2007 All-American class (excluding kickers)

4_article_124_thumb_rivals_rating_distribution_no_kickers

Figure 10. Star rating distribution of the average Rivals recruiting class (kickers/punters excluded) as compared to the distribution of how Rivals rated the 2007 All-American class.

4_article_124_thumb_effect_of_rivals_star_ratings_no_kickers

Figure 11. Ratio of the representation of a star rating (or group of star ratings) in the 2007 All-American class as compared to the average Rivals recruiting class (kickers/punters excluded)

4_article_124_thumb_effect_of_star_ratings_2

Figure 12. Ratio of the representation of a star rating (or group of star ratings) in the 2007 All-American class as compared to the average recruiting class, by rating website. (Combination of Figures 3 and 11)

Figure 12 is important, as it represents the culmination of the previously presented data. It effectively shows how well star ratings seem to predict college success (as measured by All-American status). If the star ratings had no effect at all, we would expect to see all bars on the graph at "1." If the bars are higher, it shows that particular star rating predicts a greater chance at success. If lower, the opposite is true.

From the information presented in the graph, star ratings appear to have the following correlation with becoming an All-American:

  • 0-2 stars – Some negative correlation. Using the Rivals ratings and excluding kickers, there are about half as many 0-2-star prospects on the All-America teams as there should be if they were chosen randomly. If kickers are included, there is less of a negative correlation. If Scout ratings are used, the negative correlation is very slight.
  • 3 stars – No meaningful correlation. The number of 3-star prospects present on the All-America teams is about what it would be if the members were chosen at random.
  • 4 stars – Some positive correlation. While the Rivals ratings (both with and without kickers included) show there are nearly twice as many 4-star prospects on the All-America team as there would be if members were chosen at random, the Scout ratings show no meaningful correlation.
  • 5 stars – High positive correlation. All three rating systems show 5-star recruits to be greatly over-represented in the All-America teams. There are over five times as many Rivals 5-star prospects as there should be (nearly six times if you remove the kickers), and over four times as many Scout 5-stars.

Further Concluding Comments

It would be nice to have more complete (or official) data from the two websites on the average number of star ratings given, but it seems reasonable to conclude the following:

  • A 4-star rating from Rivals or a 5-star rating from either website significantly increases the chances a player will become an All-American.
  • A 4-star rating from Scout or a 3-star rating from either website has no significant effect on a player’s chances of becoming an All-American (i.e., they might as well be picked out of a hat).
  • Rivals ratings in general are more "meaningful" in predicting success (as defined herein) than Scout ratings.
    (I’ll leave the 0-2 star ratings along for now, as they don’t provide much information all grouped together.)

So, there you have it. Thanks to _SMQ_’s suggestion (and no thanks to their off-base criticism) I’ve been able to take my analysis to another level and actually accomplish one of my previous goals. Now I know what rankings to value and which ones to ignore – as well as on which site to look.

Notes

1 Hansen, Brandon. "High School Recruit Rating Analysis of 2007 NCAA Division I All-America Teams." OmniNerd_. March 9, 2008. Accessed March 2008 from http://www.omninerd.com/articles/High_School_Recruit_Rating_Analysis_of_2007_NCAA_Division_I_All_America_TeamsTeams.

2 Anonymous. "Response to this article at Sunday Morning Quarterback." OmniNerd. March 17, 2008. Accessed March 2008 from http://www.omninerd.com/comment/show/18675.

3 SMQ. "Build Me Up, Tear Me Down, Part Three: All-Americans In Repose." Sunday Morning Quarterback. March 17, 2008. Accessed March 2008 from http://www.sundaymorningqb.com/story/2008/3/17/71811/4082.

4 Figure 5. "High School Recruit Rating Analysis of 2007 NCAA Division I All-America Teams." OmniNerd_. Accessed March 2008 from http://www.omninerd.com/articles/High_School_Recruit_Rating_Analysis_of_2007_NCAA_Division_I_All_America_Teams/images/full/star_count_bar.pngbar.png.

5 Figure 6. "High School Recruit Rating Analysis of 2007 NCAA Division I All-America Teams." OmniNerd_. Accessed March 2008 from http://www.omninerd.com/articles/High_School_Recruit_Rating_Analysis_of_2007_NCAA_Division_I_All_America_Teams/images/full/star_count_line.pngline.png.

6 Hansen, Brandon. "College Football 2007 Div I All-Americans." Google Docs. Accessed March 2008 from http://spreadsheets.google.com/pub?key=pC3mpnNSv5yIeDY8bjgFPOA.

7 "Team Ranking." Rivals.com. Accessed March 2008 from http://rivals100.rivals.com/TeamRank.asp?postype=0&sort=0. Rankings from various years are accessible via the drop-down box.

8 "Team Rankings." Scout.com. Accessed March 2008 from http://recruiting.scout.com/a.z?s=73&p=9&c=14. Rankings from various years are accessible via the drop-down box.

9 Hansen, Brandon. "College Football 2007 Div I All-Americans – No Kickers." Google Docs. Accessed March 2008 from http://spreadsheets.google.com/pub?key=pC3mpnNSv5yIt-Dno7HD5OA.

10 "Team Rankings by Position." Rivals.com. Accessed March 2008 from http://rivals100.rivals.com/teamrank.asp?postype=15.

11 "Team Rankings: Position Rankings." Scout.com. Accessed March 2008 from http://recruiting.scout.com/a.z?s=73&p=9&c=14&view=3.

Similarly tagged OmniNerd content:

Information This article was edited after publication by the author on 20 Dec 2008. View changes.
Thread parent sort order:
Thread verbosity:
0 Votes  - +
Conclusions by Anonymous

I would be careful how you word your conclusions in the final bulleted list. This may be nit-picky, but the ratings themselves don’t increase the chance that the player will become an All-American. Better to simply say that the websites’ 5-star ratings correlate to AA ratings and leave it at that.

Share & Socialize

What is OmniNerd?

Omninerd_icon Welcome! OmniNerd's content is generated by nerds like you. Learn more.

Voting Booth

America's involvement with the ISIS crisis should be?

12 votes, 0 comments