National College FB Roster Rankings going into 2014

[KingNick865]

The summary would be thus: We have an 80% chance of finishing within 2 games of 8-4. Or, another way to look at it is that we have a 70% chance of winning every game we play against teams with a lower recruiting average, however, that means that the likelihood of us winning all 8 games we should win is only about 6%. Or to look at the inverse, of the 4 games that talent says we should lose, we only have a 24% chance of losing them all.

But, here is the kicker that I didn't discuss: Over his 7 year coaching tenure, Jones has a history of over performing, in relation to talent, by an average of 3 games a year (including the -2 we saw last year).

I actually did something similar to this a few days ago, but instead of using recruiting rankings, I used Pythagorean win-loss and WP (Pythag winning percentage (WP) takes the points you scored and the points your opponent scored and uses that to compute what your winning percentage should have been. You multiply that by how many games you played, and that gives you an idea how many games you should have won (or Pythag wins).)

I then used that data to come up with a wins above pythag stat, which is represented by (Games Won - Pythag Wins). The idea is that better coaches can get their teams to perform above their base talent level -- which is kind of what Pythag measures.

For example, Butch Jones's coaching career looks like this.



So this year, Butch actually overperformed by about 1/3 of a game. His worst performance is his 2010 Cincinnati team, which won 4 games, despite having a 5-6 win talent level. His best performance is his 2008 Central Michigan team, which won 8 games, despite having a 6-7 win team in terms of talent level. For his career, Jones has won 57 games, despite the fact that his teams should have only won 54-55. That doesn't seem like much, but remember, coaches don't have as much control over their team's performance as fans might think.

That stat at the very bottom-right is Wins Above Pythag per year. (or WAPpy as it's referred to on that spreadsheet) This gives you a good idea of who is a better coach (and remember, we're strictly talking on-field performance here) from a game-to-game basis. Here is the ranking of all SEC coaches who have at least 1 year of prior coaching experience (Mason at Vanderbilt is not included in this).



First thing you'll notice is that Nick Saban comes in pretty low on this list. I'm not going to say that Nick Saban isn't a good coach -- he clearly is -- but pretty much every team he's had since he's been at Alabama has underperformed relative to their talent level. In 2013, his team should have won 12 games. They won 11. In 2012, his team was a 13-14 win team, and he won 13, which is on the lower end of what they should have won. In 2011, his team was a 12-13 win team; they won 12, which again is on the lower end of the scale. In 2010 (which came in as the worst coaching job of his career), his team won 10 games. They should have won 12-13.

Anyways, enough about Saban. Malzahn and Stoops are at the very extreme ends of this list, and I think that has to do with the fact that they haven't coached many seasons. They need to have a bigger sample size for this to be effective measurement of their skill.

Jones is the 2nd best coach in the SEC on a per game basis, with his team winning about 1/3 of a game more per year than they should (or, to term it a different way, teams that he coaches win 1 game above their talent level about every 3 years).

tl;dr version of this post: Pythag wins measure how much talent your team has. Using this, I crafted a method to see who the best coaches were. Butch Jones is 2nd best in the SEC according to this. something something math something something the end.

 

[daj2576]

I actually did something similar to this a few days ago, but instead of using recruiting rankings, I used Pythagorean win-loss and WP (Pythag winning percentage (WP) takes the points you scored and the points your opponent scored and uses that to compute what your winning percentage should have been. You multiply that by how many games you played, and that gives you an idea how many games you should have won (or Pythag wins).)

I then used that data to come up with a wins above pythag stat, which is represented by (Games Won - Pythag Wins). The idea is that better coaches can get their teams to perform above their base talent level -- which is kind of what Pythag measures.

For example, Butch Jones's coaching career looks like this.

View attachment 75101

So this year, Butch actually overperformed by about 1/3 of a game. His worst performance is his 2010 Cincinnati team, which won 4 games, despite having a 5-6 win talent level. His best performance is his 2008 Central Michigan team, which won 8 games, despite having a 6-7 win team in terms of talent level. For his career, Jones has won 57 games, despite the fact that his teams should have only won 54-55. That doesn't seem like much, but remember, coaches don't have as much control over their team's performance as fans might think.

That stat at the very bottom-right is Wins Above Pythag per year. (or WAPpy as it's referred to on that spreadsheet) This gives you a good idea of who is a better coach (and remember, we're strictly talking on-field performance here) from a game-to-game basis. Here is the ranking of all SEC coaches who have at least 1 year of prior coaching experience (Mason at Vanderbilt is not included in this).

View attachment 75102

First thing you'll notice is that Nick Saban comes in pretty low on this list. I'm not going to say that Nick Saban isn't a good coach -- he clearly is -- but pretty much every team he's had since he's been at Alabama has underperformed relative to their talent level. In 2013, his team should have won 12 games. They won 11. In 2012, his team was a 13-14 win team, and he won 13, which is on the lower end of what they should have won. In 2011, his team was a 12-13 win team; they won 12, which again is on the lower end of the scale. In 2010 (which came in as the worst coaching job of his career), his team won 10 games. They should have won 12-13.

Anyways, enough about Saban. Malzahn and Stoops are at the very extreme ends of this list, and I think that has to do with the fact that they haven't coached many seasons. They need to have a bigger sample size for this to be effective measurement of their skill.

Jones is the 2nd best coach in the SEC on a per game basis, with his team winning about 1/3 of a game more per year than they should (or, to term it a different way, teams that he coaches win 1 game above their talent level about every 3 years).

tl;dr version of this post: Pythag wins measure how much talent your team has. Using this, I crafted a method to see who the best coaches were. Butch Jones is 2nd best in the SEC according to this. something something math something something the end.

Love it. We solved a similar problem using different methodology and ended with similar results. Actually, the reason I started looking at football through this lens is that I wanted to determine the impact of a coach.

Full disclosure: I stumbled onto this while trying to decide if Dooley deserved the criticism he was getting for the performance of the team. The results were far reaching and shocking, to me. Least importantly to my findings was that Dooley did actually under-perform at a rate well below talent.

I should note how similarly we view Nick Saban. I have long thought that his value isn't coaching, is is-in fact-recruiting. Say that too loud, though, and the barbarians come out of the wood work. The problem with judging Saban, admittedly, is that he has no chance to over-perform as he has the most talented teams in the country. If you look at Saban's history, however, his history of under-performance becomes a little more clear. In sum, Saban does an exceptional job stock-piling talent, but even when he didn't have the "best" roster, he still tends to track below a talent-predicted arc.

Interestingly enough, we view Jones through a slightly different lens. My numbers, like yours, suggest that Jones worst year was his first at Cincy, but that he performed exactly as his talent averages predicted. Here is a historical chart of all of Jones' performances in relation to talent throughout his history (using only a 4 year trailing average of rivals numbers). Jones history is actually pretty astounding, and it puts him in very elite company.



I should mention that when teams recruit outside of the top 50, the correlation of wins and losses to talent diminishes. I can't give you the exact number as I have not done a precise long term study, and have only looked at limited samples. What this tends to illustrate is that coaching matters more when teams are less talented, and/or recruiting rankings are better the more talented the individual players.
Insofar as Malzahn is concerned, I actually believe his hype is way over-blown. He performed EXACTLY as his talent predicted in 2013. Compare that to just how bad of a coach Chizik really was, and the delta is shocking. Compared that to the reality that talent is paramount of winning ball games, and you end up with a coach who is adequate but certainly not worthy of any coach of the year honors.

I really applaud your work. For the first time in a long while, I will have to re-read a post on Volnation in order to absorb it all.

email me at derek@mybloodisorange.com, we might have much to discuss.

 

[Vol8188]

Hmmm. Very interesting premise, but the actual methodology is a bit crude. Averaging the team recruiting class rankings for the last five years doesn't account for all of the subsequent transfers, dismissals or early entrees that can deplete a class. A far more accurate approach would be something like adding up the sum number of Rivals stars or ESPN number grades for every roster.

I think averaging the stars of the two deep would be best.

 

[KingNick865]

Love it. We solved a similar problem using different methodology and ended with similar results. Actually, the reason I started looking at football through this lens is that I wanted to determine the impact of a coach.

Full disclosure: I stumbled onto this while trying to decide if Dooley deserved the criticism he was getting for the performance of the team. The results were far reaching and shocking, to me. Least importantly to my findings was that Dooley did actually under-perform at a rate well below talent.

I should note how similarly we view Nick Saban. I have long thought that his value isn't coaching, is is-in fact-recruiting. Say that too loud, though, and the barbarians come out of the wood work. The problem with judging Saban, admittedly, is that he has no chance to over-perform as he has the most talented teams in the country. If you look at Saban's history, however, his history of under-performance becomes a little more clear. In sum, Saban does an exceptional job stock-piling talent, but even when he didn't have the "best" roster, he still tends to track below a talent-predicted arc.

Interestingly enough, we view Jones through a slightly different lens. My numbers, like yours, suggest that Jones worst year was his first at Cincy, but that he performed exactly as his talent averages predicted. Here is a historical chart of all of Jones' performances in relation to talent throughout his history (using only a 4 year trailing average of rivals numbers). Jones history is actually pretty astounding, and it puts him in very elite company.

View attachment 75106

I should mention that when teams recruit outside of the top 50, the correlation of wins and losses to talent diminishes. I can't give you the exact number as I have not done a precise long term study, and have only looked at limited samples. What this tends to illustrate is that coaching matters more when teams are less talented, and/or recruiting rankings are better the more talented the individual players.
Insofar as Malzahn is concerned, I actually believe his hype is way over-blown. He performed EXACTLY as his talent predicted in 2013. Compare that to just how bad of a coach Chizik really was, and the delta is shocking. Compared that to the reality that talent is paramount of winning ball games, and you end up with a coach who is adequate but certainly not worthy of any coach of the year honors.

I really applaud your work. For the first time in a long while, I will have to re-read a post on Volnation in order to absorb it all.

email me at derek@mybloodisorange.com, we might have much to discuss.

You and I started in a very similar manner. I originally got into "advanced" statistics and that sort of thing because I ultimately wanted to develop some sort of coaching rating system that measured the impact of a coach. I've kind of diversified out a little, but measuring how good a coach is still holds a great deal of my college football research.

But yeah, I've come to a similar conclusion about Saban -- actually, my next project is to develop some sort of objective method for measuring how good a coach is a recruiting. Your "stockpiling talent" hypothesis about Saban was my initial conclusion as well, but I still need to do some more research to get a more objective measure of it.

When you look at it like you're looking at it, Butch is quite a stellar coach. That +8 is incredible, as is his +5 his first year at Central Michigan.

I think wins to losses diminishes as you get outside the top 50, because at that point, the classes tend to be made up of similarly caliber recruits. At least this is what I've gotten from my peering over Rivals/247 recruits.

I've heard that Chizik's performance in 2012 was historically bad, considering all the recruits that he had stockpiled.

Yeah, I'll definitely e-mail you some of my notes, spreadsheets, and other things. I always enjoy talking with those who enjoy this type of thing.

 

[daj2576]

You and I started in a very similar manner. I originally got into "advanced" statistics and that sort of thing because I ultimately wanted to develop some sort of coaching rating system that measured the impact of a coach. I've kind of diversified out a little, but measuring how good a coach is still holds a great deal of my college football research.

But yeah, I've come to a similar conclusion about Saban -- actually, my next project is to develop some sort of objective method for measuring how good a coach is a recruiting. Your "stockpiling talent" hypothesis about Saban was my initial conclusion as well, but I still need to do some more research to get a more objective measure of it.

When you look at it like you're looking at it, Butch is quite a stellar coach. That +8 is incredible, as is his +5 his first year at Central Michigan.

I think wins to losses diminishes as you get outside the top 50, because at that point, the classes tend to be made up of similarly caliber recruits. At least this is what I've gotten from my peering over Rivals/247 recruits.

I've heard that Chizik's performance in 2012 was historically bad, considering all the recruits that he had stockpiled.

Yeah, I'll definitely e-mail you some of my notes, spreadsheets, and other things. I always enjoy talking with those who enjoy this type of thing.

My hypothesis for teams outside of the top 50 is that not only are the teams more closely situated with talent, but that recruiting services actually do a better job of differentiating 3,4 and 5 star players than they do un-ranked through 2. I think you could solve much of that problem by going to a 10 star system and creating more differentiation between the athletes.

 

[coffeecountyvol]

So we are pretty much Ole Miss record wise.