Statistics tell the story III

[daj2576]

Some of you might have followed an earlier thread of mine where I discussed the recruiting and success correlation in the SEC for the 2011 and previous seasons.

Here are the predictions for the 2012 season versus the actual outcome. Below is the list of all of the teams in the SEC based off of recruiting rankings with the comparative national average in parenthesis. It is a bit misleading though as Bama averaged 2nd, that actually makes them the highest recruiting ranked school over the examined time period. All of that is really moot as I was simply wanting to examine the SEC schools in relationship to each other.

Alabama (2)
Florida (7)
LSU (8)
Georgia (9.5)
Auburn (10)
Tennessee (12.25)
South Carolina (18.25)
Texas A&M (20.25)
Ole Miss (23.75)
Arkansas (30.75)
Mississippi State (34.25)
Missouri (35)
Kentucky (54)
Vanderbilt (57.75)

Using that data, here are the talent predicted outcomes for 2012 vs. the actual outcome. To best explain "talent predicted outcomes" let us use Bama for an example. Alabama played Arkansas, Ole Miss, Missouri, Tennessee, Mississippi State, LSU, Texas A&M and Auburn. As the above list indicates, Bama out-recruited each school on their schedule and as such would be predicted to go 8-0 in SEC play. In actuality, Bama went 7-1. Now do that for each of the 14 teams in the SEC and you will have this:

Team [Predicted] (Actual)
Alabama [8-0] (7-1)
Florida [8-0] (7-1)
LSU [6-2] (6-2)
Georgia [7-1] (7-1)
Auburn [5-3] (0-8)
Tennessee [5-3] (1-7)
South Car. [4-4] (6-2)
Texas A&M [4-4] (6-2)
Ole Miss [2-6] (3-5)
Arkansas [2-6] (2-6)
MSU [1-7] (4-4)
Missouri [2-6] (2-6)
Kentucky [1-7] (0-8)
Vanderbilt [0-8] (5-3)

Does it shock you to see that 10 teams in the SEC (71.4%) actually perform within a 2 game window of their predicted outcome? If I was a betting man, I would conclude that talent is a pretty good (if not superior) indicator for success.

Only 4 teams (UT, Auburn, MSU and Vanderbilt) either exceeded or failed expectations by 3 or more games. It should be noted that Vandy and MSU (the overachievers) played both UT and Auburn (the underachievers). A logical argument could be made that both Vanderbilt's and MSU's relative success was more about other team's talent being underutilized than Franklin or Mullen being far superior coaches.

To simplify, it appears that the overachievers benefited by playing two teams who were incapable of utilizing talent. That is ultimately the reason that Dooley and Chizik were ran out of their respective towns at the wrong end of a pitch-fork wielding mob.

I still have trouble rectifying how to apply this to individual games. This "formula" does not have as much success when utilized to try to predict a specific match-up but is better statistically than a coin toss.

If you want to follow along for the bowl season, here are the picks based off of the talent differential:

1. UCF (W)
2. E. Carolina (L)
3. Washington (L)
4. Fresno State (L)
5. C. Michigan (W)
6. UCLA (L)
7. Cincinnati (W)
8. San Jose State (W)
9. Ohio (W)
10. Virginia Tech (W)
11. Texas Tech (W)
12. Rice (W)
13. W. Virginia (L)
14. Arizona State (W)
15. Texas (W)
16. Michigan St (W)
17. NCSU (L)
18. USC (L)
19. Tulsa (W)
20. LSU (L)
21. Miss State (L)
22. Oklahoma St. (W)
23. Michigan (L)
24. Georgia (W)
25. Oklahoma (L)
26. Ole Miss (W)
27. Kent St (L)
28. Stanford (W)
29. Florida St (W)
30. Florida (L)
31. Oregon (W)
32. Alabama (W)

Current Talley: Wins 19 / Losses 13: correct 59% of the time.

I only put this out here so those who might be interested can play along to see if the formula works in these one-off bowl games. It would also be interesting to debate those who have a superior system and to see what their analysis predicts. Do not use this to gamble, I doubt you would beat the spread.

Thoughts?

EDIT: My first set of bowl picks had multiple choices that I openly admitted were against the talent differential matrix that I had created. I realize that was confusing, so I changed all of the picks to simply reflect said matrix. Bottom line, the teams you see listed for the bowl picks were chosen on who had the highest four year trailing recruiting average.

 

[Volt]

For Vandy, you also need to factor in the amount of Redshirt Seniors they had on defense which Steve Martin made possible, but for which Franklin is taking credit.

 

[turambar85]

Does it shock you to see that 10 teams in the SEC (71.4%) actually perform within a 2 game window of their predicted outcome? If I was a betting man, I would conclude that talent is a pretty good (if not superior) indicator for success.

That is interesting work, and I appreciate you compiling the information. But, I don't think that performing within a 2 game window is statistically helpful really. I mean, it's not much of a measure of reliability to be within 2 games in an 8 game conference schedule.

 

[daj2576]

That is interesting work, and I appreciate you compiling the information. But, I don't think that performing within a 2 game window is statistically helpful really. I mean, it's not much of a measure of reliability to be within 2 games in an 8 game conference schedule.

So you don't think that 70+% within a deviation of 2 is meaningful?

4 teams or 29% were predicted exactly with no deviation.
8 teams or 57% were predicted within a deviation of one.
10 teams or 71% were predicted within a deviation of two.
11 teams or 79% were predicted within a deviation of 3.

To put it in another sort of perspective, imagine that we had to predict a day of a week that a certain "random" event would happen on. If over 70% of the time, someone could tell you within a 2 day window when the "random" event would happen, wouldn't that be pretty persuasive in proving that the event isn't random at all and actually can be predicted?

To restate, over 70% of the time you can narrow the potential results to less than (2/8) 25% of the total options. 100% of the time, you can narrow the potential results down to (5/8) 63% of the total options.

If you didn't follow the previous thread, I did this same evaluation back several years. The longer the evaluation went on, the higher the chance that a school would fall within a +/- 1 game window from its recruiting predicted value. I believe something like 75-80% was the actual number, but don't quote me on that.

 

[KingNick865]

You know, the interesting thing is that if you went by the 4 year class averages only (for 2009-2012), Tennessee would be ranked 11th in the nation. The team that would be ranked 12th would be...Notre Dame.

That is interesting work, and I appreciate you compiling the information. But, I don't think that performing within a 2 game window is statistically helpful really. I mean, it's not much of a measure of reliability to be within 2 games in an 8 game conference schedule.

Actually, it's pretty darn good. Most professional gamblers only hit their picks 60-65% of the time over the course of their careers. A prediction model that can hit at a 70% clip should be classified as pretty successful. The two game margin of error could conceivably be looked at as a ceiling and a floor for the team's potential for that year.

BTW OP, your work has inspired me to go a little more in-depth with this, so I gathered every FBS's recruiting class average from 2002-present and put it on an Excel speadsheet, then ran some statistical tests to find out which programs have best average, then found out which programs have the biggest variability. Tennessee has the 11th best recruiting average in FBS over the long term, with the 36th most unstable recruiting in FBS.

 

[tm3]

good stuff daj.

you did a similar kind of analysis about underperforming and overperforming coaches. i looked at it again after the BJ hire and did not find him listed. did you ever do any kind of analysis re BJ vs his competition over the years?

 

[supersmo18]

So you don't think that 70+% within a deviation of 2 is meaningful?

4 teams or 29% were predicted exactly with no deviation.
8 teams or 57% were predicted within a deviation of one.
10 teams or 71% were predicted within a deviation of two.
11 teams or 79% were predicted within a deviation of 3.

To put it in another sort of perspective, imagine that we had to predict a day of a week that a certain "random" event would happen on. If over 70% of the time, someone could tell you within a 2 day window when the "random" event would happen, wouldn't that be pretty persuasive in proving that the event isn't random at all and actually can be predicted?

To restate, over 70% of the time you can narrow the potential results to less than (2/8) 25% of the total options. 100% of the time, you can narrow the potential results down to (5/8) 63% of the total options.

If you didn't follow the previous thread, I did this same evaluation back several years. The longer the evaluation went on, the higher the chance that a school would fall within a +/- 1 game window from its recruiting predicted value. I believe something like 75-80% was the actual number, but don't quote me on that.

Good info, thanks for putting it all together.

And if you don't mind, what do you do for a living? I feel safe to conclude that you either have some sort of research-based job or you just really, REALLY love statistics.

 

[Vol_atile]

I was involved in implementing a data historian at work a few years back and it was amazing how close statistics can be to the final outcome if given enough data.

 

[VonVol]

Nobody in this thread getting any action. Nobody. Yes,I'm including myself.

 

[Vol_atile]

The odds for that are +\- 2 standard deviations.

 

[zansdad]

So you don't think that 70+% within a deviation of 2 is meaningful?

4 teams or 29% were predicted exactly with no deviation.
8 teams or 57% were predicted within a deviation of one.
10 teams or 71% were predicted within a deviation of two.
11 teams or 79% were predicted within a deviation of 3.

To put it in another sort of perspective, imagine that we had to predict a day of a week that a certain "random" event would happen on. If over 70% of the time, someone could tell you within a 2 day window when the "random" event would happen, wouldn't that be pretty persuasive in proving that the event isn't random at all and actually can be predicted?

To restate, over 70% of the time you can narrow the potential results to less than (2/8) 25% of the total options. 100% of the time, you can narrow the potential results down to (5/8) 63% of the total options.

If you didn't follow the previous thread, I did this same evaluation back several years. The longer the evaluation went on, the higher the chance that a school would fall within a +/- 1 game window from its recruiting predicted value. I believe something like 75-80% was the actual number, but don't quote me on that.

Maybe I'm missing your point but in your example using a week you said an event happening on a given day with a 2 day deviation, does this mean if I pick Wednesday, for example, that a 2 day deviation would include 2 days in either direction? That would mean anything that happened Moday through Friday would fall within that window. Which means that by just randomly guessing I would be within the 2 day deviation >70% of the time.

That being said. I agree that recruiting success is key to on the field success.

 

[huskyvol]

We need more threads like this. So according to this we have the talent, but have lacked the coaching to develop the talent. It is sad to notice some of the teams playing in a bowl this year.

 

[tim]

You cant account for poor or excellent coaching. Example...When Alabama loses to TAMU or we lose to Vandy. Recruiting does mask ineptness, but it's not a cure all.

 

[orangebloodgmc]

I'd like to see a comparison of SEC teams that looked at: of the number of starters at the end of spring practice in 2012, how many of those were still available for the first game of 2012 and how many were still around for the final game of 2012. I'd like to see it team by team and ranked. But I'm too lazy to do it.

And I am not disputing the importance of talent differential nor differences in coaching effectiveness.

 

[daj2576]

Good info, thanks for putting it all together.

And if you don't mind, what do you do for a living? I feel safe to conclude that you either have some sort of research-based job or you just really, REALLY love statistics.

I am a Land Surveyor by profession and also currently in law school.

While I don't regularly use statistical evaluations in land surveying, per se, my job is to constantly evaluate data. In other words, my whole professional world revolves around objective analysis of large quantities of sometimes conflicting data.

I just enjoy comparing the "gut feelings" of pundits and supposed experts with what quantitative data can actually prove. In this case, it doesn't surprise me that the coaches who are "the best" generally out recruit their competition by several orders of magnitude. Coaches like Spurrier and Petrino actually have a long standing track record of doing more with less.

 

[daj2576]

We need more threads like this. So according to this we have the talent, but have lacked the coaching to develop the talent. It is sad to notice some of the teams playing in a bowl this year.

You have accurately and precisely defined the key takeaway from this thread.

I think a coach who can simply coach to the latent talent level that UT has on the roster would have produced a 9-3 season, believe it or not. Truth be told, next year should be no different as far as predicted outcomes but some slack must be cut for CBJ as he is installing a new system and has to get the players to purchase his philosophy.

 

[daj2576]

good stuff daj.

you did a similar kind of analysis about underperforming and overperforming coaches. i looked at it again after the BJ hire and did not find him listed. did you ever do any kind of analysis re BJ vs his competition over the years?

That is a great idea, and thanks for the compliment. It is good to see you back in here with geeks like me.

I tell you what, I will do that exact analysis on CBJ if you can give me a few days to complete it. As it will probably be a very long and dense post, I will just make a blog entry and you can find it there. Hows that?

 

[daj2576]

Maybe I'm missing your point but in your example using a week you said an event happening on a given day with a 2 day deviation, does this mean if I pick Wednesday, for example, that a 2 day deviation would include 2 days in either direction? That would mean anything that happened Moday through Friday would fall within that window. Which means that by just randomly guessing I would be within the 2 day deviation >70% of the time.

That being said. I agree that recruiting success is key to on the field success.

If you suppose that the event has the same likelihood of happening on any day, meaning that if you guessed monday it was just as likely as happening as on a tuesday, wednesday, thursday, or friday, you would be correct. However, this data doesn't seem to suggest that. What it says is that 70% will fall within a window that could be Monday - Friday, but if we predicted a Wednesday for instance, the closer you get to Wednesday the more likely the event becomes and conversely the farther you get from Wednesday the less likely it becomes. The data also shows that if you predicted that the event would occur on a Monday (assuming you start your week on Monday) that there is a 0% chance that it occurs on either the following Saturday or Sunday. Conversely if you predict the event will happen on a Sunday, there is a 0% chance that it will happen on the preceding Monday or Tuesday.

In other words, the outcome is not random and can largely be predicted within a window that is generally one day either side of the day that the data predicts.

 

[terrypedigo]

Simple you need top 10 classes every year to compete for the SEC.

 

[orangeluvr]

Some of you might have followed an earlier thread of mine where I discussed the recruiting and success correlation in the SEC for the 2011 and previous seasons.

Here are the predictions for the 2012 season versus the actual outcome. Below is the list of all of the teams in the SEC based off of recruiting rankings with the comparative national average in parenthesis. It is a bit misleading though as Bama averaged 2nd, that actually makes them the highest recruiting ranked school over the examined time period. All of that is really moot as I was simply wanting to examine the SEC schools in relationship to each other.

Alabama (2)
Florida (7)
LSU (8)
Georgia (9.5)
Auburn (10)
Tennessee (12.25)
South Carolina (18.25)
Texas A&M (20.25)
Ole Miss (23.75)
Arkansas (30.75)
Mississippi State (34.25)
Missouri (35)
Kentucky (54)
Vanderbilt (57.75)

Using that data, here are the talent predicted outcomes for 2012 vs. the actual outcome. To best explain "talent predicted outcomes" let us use Bama for an example. Alabama played Arkansas, Ole Miss, Missouri, Tennessee, Mississippi State, LSU, Texas A&M and Auburn. As the above list indicates, Bama out-recruited each school on their schedule and as such would be predicted to go 8-0 in SEC play. In actuality, Bama went 7-1. Now do that for each of the 14 teams in the SEC and you will have this:

Team [Predicted] (Actual)
Alabama [8-0] (7-1)
Florida [8-0] (7-1)
LSU [6-2] (6-2)
Georgia [7-1] (7-1)
Auburn [5-3] (0-8)
Tennessee [5-3] (1-7)
South Car. [4-4] (6-2)
Texas A&M [4-4] (6-2)
Ole Miss [2-6] (3-5)
Arkansas [2-6] (2-6)
MSU [1-7] (4-4)
Missouri [2-6] (2-6)
Kentucky [1-7] (0-8)
Vanderbilt [0-8] (5-3)

Does it shock you to see that 10 teams in the SEC (71.4%) actually perform within a 2 game window of their predicted outcome? If I was a betting man, I would conclude that talent is a pretty good (if not superior) indicator for success.

Only 4 teams (UT, Auburn, MSU and Vanderbilt) either exceeded or failed expectations by 3 or more games. It should be noted that Vandy and MSU (the overachievers) played both UT and Auburn (the underachievers). A logical argument could be made that both Vanderbilt's and MSU's relative success was more about other team's talent being underutilized than Franklin or Mullen being far superior coaches.

To simplify, it appears that the overachievers benefited by playing two teams who were incapable of utilizing talent. That is ultimately the reason that Dooley and Chizik were ran out of their respective towns at the wrong end of a pitch-fork wielding mob.

I still have trouble rectifying how to apply this to individual games. This "formula" does not have as much success when utilized to try to predict a specific match-up but is better statistically than a coin toss.

If you want to follow along for the bowl season, here are my choices, the majority of which are based off of the talent differential:

1. UCF
2. E. Carolina
3. Washington
4. Fresno State
5. C. Michigan
6. UCLA
7. Duke
8. San Jose State
9. Ohio
10. Virginia Tech
11. Texas Tech
12. Rice
13. W. Virginia
14. Arizona State
15. Oregon St
16. Michigan St
17. Vanderbilt
18. USC
19. Tulsa
20. LSU
21. Northwestern
22. Oklahoma St.
23. S. Carolina
24. Georgia
25. Texas A&M
26. Ole Miss
27. Kent St
28. Stanford
29. Florida St
30. Florida
31. Oregon
32. Alabama

I only put this out here so those who might be interested can play along to see if the formula works in these one-off bowl games. It would also be interesting to debate those who have a superior system and to see what their analysis predicts. I admit there are about three games above that I picked against the talent differential (MSU/Northwestern, for instance). Do not use this to gamble, I doubt you would beat the spread.

Thoughts?

Very interesting keep up the good work! :worship: :clapping:

 

[FLVOL_79]

Stopped reading after I saw too many numbers.

 

[Rockytopia]

Thoughts?

Interesting, informative piece of work.

Very refreshing.

I'm thinking there one important statistic we need to correct during the 2013 season.

Instead of giving up > 500 ypg, we need to give up < 300.

 

[tim]

Interesting, informative piece of work.

Very refreshing.

I'm thinking there one important statistic we need to correct during the 2013 season.

Instead of giving up > 500 ypg, we need to give up < 300.

It might help..

 

[daj2576]

Here is something that really made me feel much better.

Per tm3 I went back and reviewed Butch Jones' numbers at Cincy.

Here is what I found:


Cincinnati's 2012 schedule ranked in order of recruiting averages

Virginia Tech = 25.25
(22) Rutgers = 39.5
Pitt = 46.25
(16) Louisville = 48.75
South Florida = 50.75
Cincinnati = 54.5
Toledo = 74.25
Syracuse = 83.75
Connecticut = 84
Temple = 92.5
Miami (OH) = 98.75
Fordham = X
Delaware St = X

Cincinnati's predicted wins = 7. ACTUAL WINS = 9 (+2)


2011

Tennessee = 16.75
West Virginia = 35.75
Pittsburgh = 41.5
Rutgers = 45
North Carolina State = 50.75
Louisville = 52
South Florida = 52
Cincinnati = 58.75
Vanderbilt = 73
Syracuse = 79.5
Connecticut = 82.5
Akron = 86
Miami (OH) = 99.5
Austin Peay = X

Predicted wins = 6. ACTUAL WINS = 10 (+4)

2010

(8)Oklahoma = 12.5
West Virginia = 23.8
Pittsburgh = 33.5
Rutgers = 37
North Carolina State = 41.75
South Florida = 50.75
Louisville = 55
Fresno State = 66.75
Cincinnati = 68.75
Syracuse = 72.75
Connecticut = 73.5
Miami (OH) = 88.5
Indiana St = X

Predicted wins = 4. ACTUAL WINS = 4 (+0).

 

[BanditVol]

Great numbers for Cincy! Hope it does mean something.

What recruiting rankings did you use? Scout, Rivals, some consensus rating?