by Jerry Palm
Nov. 1, 2006
In general, the six computer ranking systems used by the BCS all measure strength of schedule and how teams did against their schedules, and all in different ways. None of them consider margin of victory (by order of the BCS poobahs) and only a couple take home and road into account, but two are unique in, well, strange ways.
Wes Colley's system rates only Division I-A teams, so it completely ignores games played against non-Division I-A opposition, win or lose. So far this season, that's 76 games, seven of which resulted in losses for the I-A team. There are still two more I-A vs I-AA games left on the schedule, including Florida vs Western Carolina.
The Billingsley and Anderson-Hester systems do not rate I-AA teams either, but treat games against such teams like games against some generic, but bad I-A team. Massey, Sagarin and Wolfe each rate I-AA teams, so those systems more precisely address those games. For example, Iowa played Montana earlier this year. Massey ranks Montana 44th, Sagarin has the Grizzlies 38th and Wolfe has them 89th.
Anderson-Hester and Billingsley rank them worse and to Colley, the game never happened.
I took the liberty of modifying Colley's formula (since it's publicly available) to treat games against I-AA teams similar to the way Billingsley and Anderson-Hester do, which is the same as playing a bad I-A team. Obviously, adding 76 games to the system changed some things.
In particular, higher rated teams that played I-AA opponents were hurt by doing so, and the higher the rating, the more it hurt. I can't be sure where the line is, but at some point down the rankings, adding the win, even against a bad team, is better than not playing the game, but for the higher rated teams, that's not true.
Under that formula, Michigan and Ohio St are still 1-2, but have significantly lower ratings. Cal drops from third to sixth, while Florida moves up to third and USC jumps Notre Dame to fifth. Rutgers drops past Louisville, Auburn and Tennessee to 10th. Rutgers and Cal are the only teams in Colley's top 10 who have played I-AA opponents so far.
Obviously, the seven teams who lost to I-AA teams were hurt the most, but teams that played them are also hurt in this system because it adds a loss to their opponents.
Duke, which is 114th to begin with, drops to 117th and Colorado falls six spots from 105th to 111th. San Diego St drops nine spots, Northwestern 14, New Mexico 15 and Indiana 17.
Wisconsin drops three spots in the rankings, from 14th to 17th. The Badgers have not only played (and beaten) a Division I-AA opponent, but also three teams that lost to one. Michigan and Purdue will also have played three teams that lost to I-AA teams by the time the season ends.
Richard Billingsley's system counts all the games, but operates much differently than the other five computers. While the others essentially recalculate their ratings every week so that each team's schedule is measured based on the current record of its opponents, Billingsley's system operates in much the same way the polls do. Like the polls, his system has a bias toward preseason expectations in the sense that teams start with the same ranking (not rating) as they finished the previous year and go from there. His system calculates each team's rating for next week based on what it was this week, the current rating of its opponent and whether it won or lost. If a team wins, its rating goes up and if it loses, its rating goes down. If a team does not play, its rating does not change. The adjustments are more extreme the first few weeks of the season to attempt to filter out the previous season. All this is probably best illustrated by some examples.
That doesn't work particularly well especially when teams have a big turnaround in record from the year before. Teams like Northwestern, Fresno St and Colorado, all bowl teams a year ago, are much higher in Billingsley's rankings than any of the others because they started relatively high (31, 61 and 44 respectively) and as they've lost, haven't fallen far enough to get close to where the other computers rank them. Northwestern is 27 spots higher in Billingsley's system than in the next closest one, Colorado is 28 spots higher and Fresno St is 16 spots higher.
That also impacts the teams they play. Last week Michigan got credit for playing the 61st rated team (Northwestern) instead of one of the worst teams in I-A. This week, Notre Dame will get credit for playing the 72nd rated team, North Carolina (1-7), which every other computer has rated at least 26 spots lower. That means playing UNC will not hurt Notre Dame's strength of schedule nearly as much in Billingsley's system nearly as much as it would in any other.
It works the other way too. Rutgers at 13 is only two spots lower than the next closest computer, but might not even be that high if they hadn't opened with North Carolina. The Scarlet Knights started 53rd, but got a big boost by beating the #33 Tar Heels. If that game were this week instead of week 1, you have to wonder if Rutgers would be as high after the win.
Indiana may be in a bit of a resurgence, but it's not showing up in Billingsley's ratings yet. IU is 65th, 14 spots lower than the next lowest rating for the Hoosiers. Kentucky is .500 after a 3-8 season a year ago, but still rates 74th in Billingsley, 22 spots below the next lowest ranking.
Billingsley's ranking is either the highest or the lowest by at least 10 spots for 31 teams. The highest rated of those in his system is #27 TCU. The next highest rating for the Horned Frogs is 40 in Anderson-Hester. The only other ranking system to be highest or lowest on one team by at least ten spots is Wolfe's. He has UL Lafayette 11 spots higher than the next highest ranking. Billingsley is the lowest on the Ragin' Cajuns by 22 spots.
Given all this, when you look at how the six computer rankings statistically correlate to each other, it is not surprising that Billingsley's system correlates relatively poorly. We now interrupt this paragraph for a Geek Alert. Statistical correlation is a measure of how similar one set of data is to another. Like the BCS, it's a number between 0 and 1, with 1 being "better," in this case identical, and 0 meaning essentially opposite.
The best correlation between Billingsley's rankings and another is with Anderson-Hester's at .9299. That's not really that far off, except when you consider that the worst correlation between two of the other five systems is between Colley and Sagarin at .9722. That's a big gap. Massey and Sagarin have the best correlation at .9962.
Because Billingsley's rankings are so different than the others and the computer average is calculated after throwing out the high and the low, his ratings have been rendered nearly irrelevant because they are rarely used. His ranking is the exclusive high or low, and therefore thrown out, on a whopping 80 of 119 teams. It is also tied for high or low on another 15.
This isn't just a mid-season thing. At the end of last season, his rankings were the exclusive high or low on 79 teams.
Different isn't necessarily bad. There's no point in having six computer ranking systems if they're all just going to be the same, but ignoring a big chunk of data because it doesn't fit cleanly with the formula or vastly overrating or underrating teams because of how they did last year don't seem like good ways to be different.
