Hoegher's Rankings! (discuss please!)

by hoegher on Nov 28, 2011 6:47 PM CST

Er... hello everyone! I'm relatively new here, and so this is my first FanPost. I was sitting at home bored (and ignoring my polymers textbook in the corner), and I decided to try and induce some discussion here on OTE (my favorite of the SBNation blogs). And what better way to do that than with a new set of computer rankings? We clearly don't have enough of those, right? Right?

Um... right. Moving on.

I started trying to come up with a ranking system last year just before bowl season, and I had great fun with it. So much fun that I've actually got 4 different rankings now, each with their own formulas and mechanisms. Not being a mathematician, these formulas are mostly based on gut instinct and evaluating the final results by "eh, that looks about right." I make no pretense of these being anything more than amateur fun, and I readily accept any and all criticism (Just remember that every time you criticize me, a puppy cries. This puppy)

Rather than try to choose one system to post here, I thought I'd post two for comparison. The first (which I'll call the Resume Rankings) is based solely on wins/losses and schedule strength. Credit is given to facing top teams, with diminishing credit margin for lower ranked teams (i.e. the difference between #60 and #80 team matters much less than the difference between a #1 and a #20 team).

The second (which I'll call the Efficiency Rankings) is based solely on the relative performance of a team. (e.g. if 20 pts is scored on an opponent giving up 10 pts/game, the relative offense for that game is 2.0). Wins/losses don't matter, just how well the team performed against a given opponent.

I chose these two because:

1) They are diametrical opposites, and therefore give a nice basis for comparison.
2) They are probably the closest I have to mathematically sound rankings ("close" being a relative term here).
3) They have the simplest formulas, which seems better somehow. Like I said, I'm not a mathematician.

Right. Let me just fire up my Excel sheet here, and on to the rankings!

Resume Rankings
NCAA Team (# Top 10 played, # Top 25 played, # Top 50 played)

LSU (3, 3, 7)
Alabama (2, 3, 5)
Oklahoma State (1, 3, 7)
Houston (0, 1, 2)
Arkansas (2, 3, 6)
Boise State (0, 3, 6)
Virginia Tech (0, 2, 5)
Stanford (1, 2, 4)
Kansas State (1, 3, 6)
Oregon (2, 3, 4)
Oklahoma (1, 3, 5)
South Carolina (1, 3, 5)
Georgia (1, 2, 5)
USC (2, 2, 4)
Michigan (0, 2, 4)
TCU (1, 2, 5)
Michigan State (0, 3, 4)
Baylor (2, 4, 6)
Wisconsin (0, 3, 4)
Arkansas State (1, 1, 4)
Nebraska (0, 4, 6)
Penn State (1, 3, 4)
Clemson (1, 2, 6)
Southern Mississippi (0, 0, 2)
Tulsa (3, 4, 4)

And the rest of the Big Ten:

54. Iowa (0, 4, 4)
61. Ohio State (0, 5, 6)
69. Ilinois (0, 4, 4)
72. Purdue (0, 3, 4)
73. Northwestern (0, 4, 4)
95. Minnesota (0, 5, 5)
119. Indiana (0, 3, 4)

Efficiency Rankings

NCAA Team (Efficiency)

LSU (0.805)
Alabama (0.780)
Wisconsin (0.725)
Boise State (0.711)
Oklahoma (0.707)
Oregon (0.701)
Michigan (0.699)
Stanford (0.695)
Michigan State (0.674)
Florida State (0.664)
Georgia (0.661)
Houston (0.660)
Virginia Tech (0.652)
Temple (0.651)
Arkansas (0.650)
TCU (0.649)
Oklahoma State (0.646)
Notre Dame (0.645)
USC (0.638)
South Carolina (0.632)
Missouri (0.620)
Texas A&M (0.619)
Penn State (0.616)
Texas (0.608)
Nebraska (0.606)

And the rest of the Big Ten:

31. Ohio State (0.588)
51. Illinois (0.543)
55. Iowa (0.535)
67. Purdue (0.506)
68. Northwestern (0.506)
99. Minnesota (0.409)
109. Indiana (0.373)

So... what can we take away from this?

Chokers: Wisconsin, Texas A&M, Florida State, Temple*, Missouri, Notre Dame
These teams have simply not played up to their potential, some more than others (looking at you Texas A&M). As always, what matters in the end is whether another mark is added to the W or L column. But these teams should have more wins, looking at the numbers. Now excuse me while I go cry in a corner over Wisconsin's two losses.
* I'm as perplexed by Temple here as you are, seeing as they're not really that good (they lost 10-13 to Bowling Green, for example). I think the reason for their high efficiency is a couple of shutouts they pitched to bad teams. Numbers lie, etc.

Praying pollsters don't look at the box score: Arkansas, Oklahoma State, Kansas State, Virginia Tech
No surprise here, as the number don't forget middling performances against mediocre competition as easily as the voters. I've been down on Arkansas this whole season, and this is why. I'm ignoring Oklahoma State here, because I really don't want an all-SEC BCS Championship.

LSU and Alabama are really good.
No matter how you look at it, they seem to be the best two teams in college football. Their near unanimous support as #1 and #2 in every poll is justified, and if the BCS wants to put the best two teams in the championship, it should be them. I happen to think the BCS has not aimed to put the best two teams in the Championship in the past, so there is no reason it should start now.

Boise State is under-rated, Stanford is over-rated.
Not that this should be a surprise, but Boise State looks to be better than the pollsters are giving them. Boise State partisans have of course been screaming this for years, but at least they can take solace in the fact that they are only slightly under-rated. Conversely, Stanford is probably not as good as what the pollsters are giving them (I believe this is known as the "Andrew Luck Factor"), though again the difference is only slight.

Indiana is not very good.
Yeesh. I hope Kevin Wilson can turn this around sometime, because those are some glaringly bad numbers. I know several of us have wanted a cripple fight between Minnesota and Indiana to determine the cripple-iest of them all, but it looks like Minnesota at least has a pair of crutches, while Indiana resembles Steven Hawking after he's fallen out of his wheelchair. (I took that metaphor too far, but I got rolling and I went for it. I regret nothing.)
Well, let me know what you think! I like this community (even the Michigan State fans!), and I enjoy the opportunity to contribute something of worth.*

- hoegher

*your mileage may vary.

Tweet Comment 6 comments | 0 recs |

Do you like this story?

Comments

Interesting.

My one problem is that the rankings you used for Bama and Oklahoma State for Resume are a bit odd. Bama has beaten 2 current BCS top 25 teams. OKST has beaten 5. OKST also has two more wins over current BCS top 50 teams. They also have 6 wins over FBS teams with winning records compared to Bama’s three. Im interested in how your rankings would change based on those numbers. I do like the concept for both formulas.

by OSUreds on Nov 28, 2011 8:33 PM CST reply actions

Oh sorry, should have clarified.

The # Top 10, # Top 25, # Top 50 refer to the teams played (both won and lost), and they also refer to the ranking within the Resume system, instead of the current BCS rankings. Make more sense?

by hoegher on Nov 28, 2011 11:01 PM CST up reply actions

Questions

1. Is that the current rank, or the rank when they played?

2. Are your resume results significantly different from CFN’s post-season ranking numbers (they only do it at the end of the season)? http://cfn.scout.com/2/1039643.html is for 2010 and explains their formula. I like their method.

3. Are your efficiency rankings for offense only, or O and D combined?

4. Are your efficiencies strictly linear or do you give diminishing returns for big blowouts?

5. Do you consider both teams’ averages for your efficiency or just one? For example, let’s say team A scores 30 ppg and team B allows 10 ppg. If A scores 20 against B, does A just get an efficiency of 2? Does B get credit for holding A below their average (like a defensive efficiency of 30/20 = 1.5)? Does A get any penalty for being held below their average?

by br27 on Nov 29, 2011 4:30 AM CST up reply actions

Answers!

1) The ranks listed are the current ranks. While I appreciate the excitement of a fan watching your team play a highly ranked team, I think you should only get credit for that if the opponent continues to prove themselves worthy of that ranking (e.g. Wisconsin in 2008 was not a Top 10 team)

2) CFN Top 10 2010 (Final 2010 Resume Rankings)
1. Auburn (1)
2. TCU (3)
3. Ohio State (7)
4. Oklahoma (10)
5. Boise State (9)
6. Stanford (8)
7. Oregon (5)
8. Oklahoma State (11)
9. Nevada (12)
10. Alabama (6)

So not too bad, but could be better (I have LSU at #2, and Arkansas #4). The CFN Rankings take into account scoring margins and home/away, however. The Resume Rankings don’t factor in the score (which may explain some of the differential), and home/away is something I don’t factor in, mostly because I don’t have the time to do it. I type in scores and press ENTER.

Oh, I forgot. I don’t factor in the FCS teams for the Resume Rankings. This is mostly because the rankings are arrived at via an iterative process, and FCS teams lay outside the system. (I didn’t explain that very well, but the point is that FCS teams are not considered in the rankings). I guess I could factor them back in, but I’m wary of how that might affect the results.

3) The Efficiency Rankings take into account Offense and Defense, I was just using an example before. To illustrate: Suppose Team A plays Team B. Team B scores 40 pts/game on average and gives up 20 pts/game on average. The game ends Team A – 40, Team B – 20. Team A then has a Relative Offense of 40/20 = 2.0. Their Relative Defense is 20/40 = 0.5.

4) There are diminishing returns, it’s just built into the formula. The Efficiency is calculated by (Rel Off)/(Rel Off + Rel Def). Originally, I calculated the Efficiency by (Rel Off)/(Rel Def). I stopped this because a) it was dumb, b) it didn’t work for teams that shut out their opponents (OH NOES I DIVIDED BY ZERO), and c) it led to runaway scores for the best teams. I got the idea for the current formula from MGoBlog.com’s short lived GopherQuest (for measuring the worst Big Ten teams of all time), and I’m satisfied with it.
Additionally, it’s actually difficult to increase a team’s efficiency that much in blowouts. Blowouts tend to come against bad teams, and if a team is giving up 30 pts/game, scoring 60 on them is still only a Rel Off of 2.0. Shutout defense is more of an issue. (see: Temple)

5) The Efficiencies (<- I don’t know why that isn’t a word) are only calculated relative to what is expected against the opponent. A team doesn’t get punished just because they set a high bar for themselves.
This actually means that a team can perform “better” than their opponent and still lose. To illustrate: consider Team A (call them the Fightin’ SEC’s if you want a name) playing Team B (call them the Fightin’ Sun Belts if you want a name)
Team A scores 40 pts/game and gives up 5 pts/game on average.
Team B scores 10 pts/game and gives up 40 pts/game on average.
Final Score: Team A – 30, Team B – 20.
Team A: Rel Off = 0.75, Rel Def = 2.0, Efficiency = 0.27
Team B: Rel Off = 4.0, Rel Def = 0.75, Efficiency = 0.84
It’s an interesting quirk (I think it’s somewhat analogous to Bill C.‘s Adj. Scores, but maybe I’m just name dropping), and it doesn’t really mean that Team B is better than Team A. Also, those things tend to even out over the course of a season.

by hoegher on Nov 29, 2011 12:13 PM CST up reply actions

OkSt has only beaten 4 so far.

The opportunity at #5 comes this week.

I've got this terrible pain in all the diodes down my left-hand side.
Bradley-Terry rankings for college football and basketball: because there aren't enough computer rankings already.

by SpartanDan on Nov 29, 2011 12:52 AM CST up reply actions

I suppose I should also explain how the Resume formula works.

I initially experimented with just the Big Ten, and their games against each other. The formula was:

Rating = Σ(Opp. Rank)/(total wins)

with the lower ratings being best (winless teams were assigned 0.5 total wins, simply to make the math work). Thus, a team that played a #1 and #10 ranked opponent (winning both) would have a rating of (1 + 10)/2 = 5.5. A team that played a #1 and #10 ranked opponent (winning one) would have a rating of (1 + 10)/1 = 11. Final rankings were arrived at via an iterative process (as the rank of each team would change after each re-ordering).

As I said, I experimented with just Big Ten play (in 2010), and it seemed to work very well. MSU, WIsconsin, and Ohio State rounded out the Top-3, and Minnesota and Indiana were at the bottom. Similar experiments with the SEC also worked well, so I expanded the formula to all FBS teams (FCS games were excluded).

I immediately ran into problems. Namely:

a) The rankings wouldn’t converge, even after several iterations.
b) The rankings were far too dependent on schedule strength (Missouri was consistently near the top for example)

which I should have seen. The Big Ten only has 11 teams (at that point), the SEC only has 12 teams, but FBS has 120 teams. I needed a way to still reward teams for playing high quality opponents, without the steep penalty for lower ranked opponents. I decided to tweak the formula to:

Rating = Σ(ln(Opp Rank + 1))/(total wins)

(the additional “+1” is so that playing the #1 ranked team actually yields a positive value)

This worked much better. It preserved the benefit of playing a top-ranked team, while also acknowledging that the difference between #1 and #20 is significant, a #100 and #120 ranked team is negligible. They both aren’t very good at football. Furthermore, the rankings now converged (and passed my “eh, that looks about right” test. I am nothing if not professional)

There are really only two issues I have with the Resume system as it stands now:

1) The rankings are not unique.
Depending on the iterative path taken, the exact rankings #1 -#120 are not the same. Fortunately, the differences are small (#19 TCU may be switched with #20 Georgia depending on the iteration) and relatively rare.

2) All wins are considered equal.
Take the example above. If a team plays the #1 and #10 teams, winning one, the rating is identical no matter if they beat the #1 team or the #10 teams. I’m not sure if this is a bad thing or not (think Oklahoma State vs. Alabama. Oklahoma State has a worse loss, but better wins). Also, this is difficult to correct for (with Excel and my programming skills) in the event of rematches.

So that’s that.

by hoegher on Nov 29, 2011 3:30 PM CST reply actions