FanPost

How Much Do Pre-Season Rankings Really Matter?

HEY GUYS HEY GUYS, GUESS WHAT?

(what?)

IT'S ALMOST TIME FOR FOOTBALL TO START! YOU KNOW WHAT THAT MEANS?

(what?)

PRE-SEASON POLLS! AND ANGER ABOUT THOSE POLLS! HUZZAH!

For the record, I have no problem with pre-season polls. I think they're a cool part to help get excited for the coming season, and they offer discussion points for us fans to use when we're laughing at Phil Steele for putting Florida State at #1 again. However, my enthusiasm for pre-season polls is largely due to believing that they have no real impact once the season has progressed beyond the first few weeks. I don't deny the existence of pre-season bias in the polls (why else does Florida State continue getting Top 10 billing for each new season), I just think that those biases become negligible with a full season of results for pollsters to look at and re-evaluate their choices.

However, people on the other side of this arguement make a good point: no one likes being wrong (this is also why I decided to do this Fan Post in the first place - I wanted to see if I was right). It follows that voters would be reluctant to deviate too much from their pre-season convictions, and this could end up having negative repercussions at the end of the season for those teams that are - ahem - less favored in the late summer consensus.

Well, I thought, I am lucky enough to have the INTERNETS at my disposal, and I am also lucky enough to have enough free time to do with as I see fit. So I went into the annals of the tubes to try and see if there was a noticeable effect that pre-season polls have on the end-of-year rankings and what that effect was. What follows is a pseudo-scientific study that aims to answer just that.

First: the information base. I decided to stick the the AP Poll, as I think it's the better regarded of the main human polls. I looked at the Pre-Season AP Poll from 2005-2011 (obviously, I would need to at least have this) as well as the Final AP Poll from those same years. As a standard for comparison, I also looked at the final F/+ Rankings for those same years. For the sake of arguement, we will assume that the F/+ Ranking of a team represents it's "true" ranking.

The difference between the Final AP Ranking and the F/+ Ranking represents the human bias for or against a team. Positive bias is assumed beneficial to a team (they are ranked better in the polls than their "true" strength). To illustrate:

If Team A has an Final AP Ranking of 10th and a F/+ Ranking of 15th, the Bias for Team A would be +5.

If Team A has an Final AP Ranking of 10th and a F/+ Ranking of 5th, the Bias for Team A would be -5.

The theory here is that - if pre-season bias has an effect through the end of season - team that are over-rated at the end of season (positive bias) will tend to be teams that were over-rated at the beginning of the season (so their Final AP Ranking is worse than their Pre-Season AP Ranking).

Similarly, teams that were over-rated to begin the season (so their Final AP Ranking is worse than their Pre-Season AP Ranking) would continue to be over-rated at the end of season (postive bias).

Follow all that? No? Too bad, we're continuing anyway.

As I said, I looked at Pre-Season AP, Final AP, and Final F/+ Rankings for every team 2005-11. I limited my focus to those teams that were ranked in either the Pre-Season AP or Final AP Poll (i.e. no Indiana's or Akron's), and I tried my best to get both a Pre-Season AP and Final AP for those teams. Most times, I was able to go through the "Others Receiving Votes" listing at the end of the poll and approximate a ranking from there, otherwise I simply assumed the team was the best team not listed there (e.g. if there were 12 teams receiving votes, the assumed ranking would be 25+12+1 = 38). It's imperfect, but just go with it.

Lastly, I filtered my teams of interest to those that had either:

Difference between Pre-Season AP Ranking and Final AP Ranking of at least 5 (either way)

OR

Difference between Final AP Ranking and F/+ Ranking (bias) of at least 5 (either way)

This was done to better draw a distinction for those teams that were over-rated or under-rated. Why five as my cut-off? It seemed like a good number. Like I said, I started this project this afternoon, so I didn't take time to write out a research proposal. Feel free to criticize if you so choose!

ANYWAY. I think I've finally explained enough to start putting up some tables and numbers. If you skipped all that explanation above: welcome! Try not to get confused.

WARNING. WARNING. Long tables of team names and numbers follow. WARNING. WARNING.

First: teams that were OVER-RATED in the Pre-Season AP Poll

Pre-Season Over-Rated (AP Rk Change is at least -5)
Year Team AP Pre Rk AP Final Rk F/+ Final Rk AP Rk Change Bias
2005 Arizona State 20 41 32 -21 -9
2005 Boise State 18 34 59 -16 25
2005 California 19 25 37 -6 12
2005 Florida State 14 23 25 -9 2
2005 Fresno State 24 32 38 -8 6
2005 Iowa 11 31 23 -20 -8
2005 Louisville 12 19 20 -7 1
2005 Miami 9 17 8 -8 -9
2005 Michigan 4 28 14 -24 -14
2005 Oklahoma 7 22 21 -15 -1
2005 Pittsburgh 23 50 60 -27 10
2005 Purdue 15 50 50 -35 0
2005 Tennessee 3 50 28 -47 -22
2005 Texas A&M 17 50 57 -33 7
2005 Virginia 25 33 54 -8 21
2006 Arizona State 24 35 43 -11 8
2006 Auburn 4 9 25 -5 16
2006 California 9 14 15 -5 1
2006 Clemson 18 35 24 -17 -11
2006 Florida State 11 35 40 -24 5
2006 Georgia 15 23 26 -8 3
2006 Iowa 16 35 48 -19 13
2006 Miami 12 35 37 -23 2
2006 Nebraska 20 27 29 -7 2
2006 Notre Dame 2 17 28 -15 11
2006 Oregon 21 35 38 -14 3
2006 Penn State 19 24 39 -5 15
2006 Texas 3 13 20 -10 7
2006 Texas Tech 25 35 36 -10 1
2006 West Virginia 5 10 8 -5 -2
2007 Arkansas 21 31 27 -10 -4
2007 Boise State 24 30 36 -6 6
2007 California 12 40 22 -28 -18
2007 Florida 6 13 6 -7 -7
2007 Florida State 19 40 45 -21 5
2007 Louisville 10 40 50 -30 10
2007 Michigan 5 18 25 -13 7
2007 Nebraska 20 40 69 -20 29
2007 Penn State 17 27 30 -10 3
2007 Rutgers 16 39 37 -23 -2
2007 TCU 22 40 41 -18 1
2007 Texas 4 10 23 -6 13
2007 Texas A&M 25 40 60 -15 20
2007 UCLA 14 40 24 -26 -16
2007 Wisconsin 7 24 40 -17 16
2008 Arizona State 15 40 74 -25 34
2008 Auburn 10 40 63 -30 23
2008 BYU 16 25 39 -9 14
2008 Clemson 9 40 29 -31 -11
2008 Georgia 1 13 17 -12 4
2008 Illinois 20 40 45 -20 5
2008 Kansas 14 47 48 -33 1
2008 LSU 7 28 33 -21 5
2008 Missouri 6 19 16 -13 -3
2008 Ohio State 2 9 11 -7 2
2008 South Florida 19 40 27 -21 -13
2008 Tennessee 18 40 46 -22 6
2008 Wake Forest 23 40 31 -17 -9
2008 West Virginia 8 23 26 -15 3
2008 Wisconsin 13 40 43 -27 3
2009 California 12 40 54 -28 14
2009 Florida State 18 34 27 -16 -7
2009 Georgia 13 33 37 -20 4
2009 Kansas 25 40 58 -15 18
2009 LSU 11 17 15 -6 -2
2009 Mississippi 8 20 26 -12 6
2009 North Carolina 21 40 30 -19 -10
2009 Notre Dame 23 40 38 -17 -2
2009 Oklahoma 3 26 16 -23 -10
2009 Oklahoma State 9 30 40 -21 10
2009 USC 4 22 29 -18 7
2010 Alabama 1 10 5 -9 -5
2010 Boise State 3 9 3 -6 -6
2010 Florida 4 31 36 -27 5
2010 Georgia 23 40 29 -17 -11
2010 Georgia Tech 16 40 64 -24 24
2010 Iowa 9 27 22 -18 -5
2010 Miami 13 40 24 -27 -16
2010 Nebraska 8 20 20 -12 0
2010 North Carolina 18 40 45 -22 5
2010 Oregon State 24 40 41 -16 1
2010 Penn State 19 40 54 -21 14
2010 Pittsburgh 15 40 23 -25 -17
2010 Texas 5 40 69 -35 29
2010 USC 14 40 31 -26 -9
2010 Virginia Tech 10 16 11 -6 -5
2010 West Virginia 25 32 18 -7 -14
2011 Florida 22 35 32 -13 -3
2011 Florida State 6 23 10 -17 -13
2011 Mississippi State 20 35 45 -15 10
2011 Missouri 21 29 27 -8 -2
2011 Nebraska 10 24 28 -14 4
2011 Notre Dame 16 35 15 -19 -20
2011 Ohio State 18 35 40 -17 5
2011 Oklahoma 1 16 9 -15 -7
2011 Texas A&M 8 35 18 -27 -17
2011 Virginia Tech 13 21 24 -8 3

Avg Bias (end of season): 2.06

What this means is that teams that are over-rated in the AP Poll to begin the season will end the season being over-ranked (relative to F/+) by about 2.06 slots on average.

So you might look at this and say, well that proves it then! Pre-Season Rankings have effect on how a team is ranked at the end of the season, clearly a team derives some benefit from being ranked highly to begin the season. To which I say: hold your horse there. It turns out that the average team in the Final AP Poll (regardless of pre-season placement) will be over-ranked by 2.11 slots (relative to F/+ Rankings). This difference is negligible right now. What we need to do is see how this compares to those teams that are under-ranked to begin the season. If those teams enjoy less of a beneficial bias than their over-ranked counter-parts, we can still say that pre-season rankings have an effect at the end of season.

Pre-Season Under-Rated (AP Rk Change = 5 or more)
Year Team AP Pre Rk AP Final Rk F/+ Final Rk AP Rk Change Bias
2005 Alabama 26 8 10 18 2
2005 Clemson 43 21 19 22 -2
2005 Nebraska 41 24 33 17 9
2005 Notre Dame 45 9 12 36 3
2005 Oregon 27 13 17 14 4
2005 Penn State 37 3 6 34 3
2005 TCU 50 11 22 39 11
2005 UCLA 33 16 34 17 18
2005 West Virginia 44 5 15 39 10
2005 Wisconsin 40 15 27 25 12
2006 Arkansas 32 15 14 17 -1
2006 Boise State 31 5 11 26 6
2006 Boston College 28 20 17 8 -3
2006 BYU 40 16 9 24 -7
2006 Florida 7 1 3 6 2
2006 Louisville 13 6 7 7 1
2006 LSU 8 3 4 5 1
2006 Michigan 14 8 10 6 2
2006 Oregon State 50 21 33 29 12
2006 Rutgers 38 12 13 26 1
2006 Wake Forest 50 18 34 32 16
2006 Wisconsin 33 7 23 26 16
2007 Arizona State 36 16 28 20 12
2007 Boston College 28 11 20 17 9
2007 BYU 37 14 14 23 0
2007 Cincinnati 50 17 19 33 2
2007 Clemson 41 21 15 20 -6
2007 Georgia 13 2 12 11 10
2007 Illinois 50 20 26 30 6
2007 Kansas 50 7 16 43 9
2007 Missouri 26 4 13 22 9
2007 Ohio State 11 5 7 6 2
2007 Oregon 29 23 8 6 -15
2007 Oregon State 33 25 18 8 -7
2007 Texas Tech 43 22 29 21 7
2008 Alabama 24 6 9 18 3
2008 Boise State 36 11 10 25 -1
2008 Cincinnati 30 17 21 13 4
2008 Florida State 31 21 20 10 -1
2008 Georgia Tech 50 22 34 28 12
2008 Iowa 50 20 14 30 -6
2008 Michigan State 35 24 42 11 18
2008 Mississippi 50 14 13 36 -1
2008 Oklahoma State 50 16 25 34 9
2008 Oregon 21 10 19 11 9
2008 Oregon State 42 18 30 24 12
2008 Penn State 22 8 7 14 -1
2008 TCU 50 7 8 43 1
2008 Texas 11 4 6 7 2
2008 Utah 29 2 12 27 10
2009 Boise State 14 4 9 10 5
2009 BYU 20 12 25 8 13
2009 Central Michigan 41 23 41 18 18
2009 Cincinnati 33 8 18 25 10
2009 Clemson 34 24 13 10 -11
2009 Iowa 22 7 12 15 5
2009 Miami 35 19 14 16 -5
2009 Nebraska 24 14 17 10 3
2009 Oregon 16 11 11 5 0
2009 TCU 17 6 5 11 -1
2009 Texas Tech 31 21 22 10 1
2009 West Virginia 32 25 33 7 8
2009 Wisconsin 55 16 21 39 5
2010 Arkansas 17 12 12 5 0
2010 Auburn 22 1 4 21 3
2010 Central Florida 46 21 33 25 12
2010 LSU 21 8 15 13 7
2010 Maryland 50 23 34 27 11
2010 Michigan State 50 14 32 36 18
2010 Mississippi State 50 15 30 35 15
2010 Missouri 33 18 19 15 1
2010 Nevada 50 11 26 39 15
2010 North Carolina State 50 25 25 25 0
2010 Oklahoma State 42 13 14 29 1
2010 Oregon 11 3 9 8 6
2010 South Carolina 29 22 16 7 -6
2010 Stanford 27 4 6 23 2
2010 Texas A&M 40 19 27 21 8
2010 Tulsa 50 24 55 26 31
2010 Wisconsin 12 7 13 5 6
2011 Arkansas 15 5 16 10 11
2011 Baylor 50 13 31 37 18
2011 Cincinnati 50 25 29 25 4
2011 Clemson 50 22 38 28 16
2011 Houston 36 18 20 18 2
2011 Kansas State 50 15 35 35 20
2011 Michigan 38 12 14 26 2
2011 Michigan State 17 11 12 6 1
2011 Oklahoma State 9 3 5 6 2
2011 Southern Miss 31 20 22 11 2
2011 USC 25 6 13 19 7
2011 West Virginia 24 17 25 7 8

Avg Bias (end of season): 5.41

Well, that is interesting! It seems that teams that are under-rated coming into a season (such that their Final AP Ranking betters their initial AP Ranking) enjoy a better AP Ranking (relative to their F/+ Ranking) than their over-rated counter-parts. This would indicate that the arguement that "it's difficult to move up the rankings if you start low in the rankings," isn't entirely accurate. A Bias of 5.41, compared to 2.06 for "over-rated" teams points to sufficient mobility for teams to move up in the poll, should their performance demand it.

But that's only one method of evaluation. We could also look at those teams that are over-rated/under-rated at the end-of-season. If it is true that it is more difficult to move up the poll if you start low, and easier to retain a top ranking if you start the season ranked high, we would expect many over-rated teams (at the end of season) to have a high Pre-Season AP Ranking, and we would expect many under-rated teams (at the end of season) to have a low Pre-Season AP Ranking. Thus, over-rated teams would be expected to have a better Pre-Season AP Ranking than under-rated teams. Two tables follow that... tabulate this (man, I need to better my vocabulary).

First, the over-rated teams to end the season:

End-of-Season Over-Rated (Bias is at least +5)
Year Team AP Pre Rk AP Final Rk F/+ Final Rk AP Change Bias
2005 Boise State 18 34 59 -16 25
2005 California 19 25 37 -6 12
2005 Florida 10 12 18 -2 6
2005 Fresno State 24 32 38 -8 6
2005 Nebraska 41 24 33 17 9
2005 Pittsburgh 23 50 60 -27 10
2005 TCU 50 11 22 39 11
2005 Texas A&M 17 50 57 -33 7
2005 UCLA 33 16 34 17 18
2005 Virginia 25 33 54 -8 21
2005 West Virginia 44 5 15 39 10
2005 Wisconsin 40 15 27 25 12
2006 Arizona State 24 35 43 -11 8
2006 Auburn 4 9 25 -5 16
2006 Boise State 31 5 11 26 6
2006 Florida State 11 35 40 -24 5
2006 Iowa 16 35 48 -19 13
2006 Notre Dame 2 17 28 -15 11
2006 Oklahoma 10 11 16 -1 5
2006 Oregon State 50 21 33 29 12
2006 Penn State 19 24 39 -5 15
2006 Texas 3 13 20 -10 7
2006 Wake Forest 50 18 34 32 16
2006 Wisconsin 33 7 23 26 16
2007 Arizona State 36 16 28 20 12
2007 Boise State 24 30 36 -6 6
2007 Boston College 28 11 20 17 9
2007 Florida State 19 40 45 -21 5
2007 Georgia 13 2 12 11 10
2007 Hawaii 23 19 44 4 25
2007 Illinois 50 20 26 30 6
2007 Kansas 50 7 16 43 9
2007 Louisville 10 40 50 -30 10
2007 Michigan 5 18 25 -13 7
2007 Missouri 26 4 13 22 9
2007 Nebraska 20 40 69 -20 29
2007 Tennessee 15 12 21 3 9
2007 Texas 4 10 23 -6 13
2007 Texas A&M 25 40 60 -15 20
2007 Texas Tech 43 22 29 21 7
2007 Wisconsin 7 24 40 -17 16
2008 Arizona State 15 40 74 -25 34
2008 Auburn 10 40 63 -30 23
2008 BYU 16 25 39 -9 14
2008 Georgia Tech 50 22 34 28 12
2008 Illinois 20 40 45 -20 5
2008 LSU 7 28 33 -21 5
2008 Michigan State 35 24 42 11 18
2008 Oklahoma State 50 16 25 34 9
2008 Oregon 21 10 19 11 9
2008 Oregon State 42 18 30 24 12
2008 Tennessee 18 40 46 -22 6
2008 Utah 29 2 12 27 10
2008 Virginia Tech 17 15 23 2 8
2009 Boise State 14 4 9 10 5
2009 BYU 20 12 25 8 13
2009 California 12 40 54 -28 14
2009 Central Michigan 41 23 41 18 18
2009 Cincinnati 33 8 18 25 10
2009 Georgia Tech 15 13 19 2 6
2009 Iowa 22 7 12 15 5
2009 Kansas 25 40 58 -15 18
2009 Mississippi 8 20 26 -12 6
2009 Oklahoma State 9 30 40 -21 10
2009 Texas 2 2 7 0 5
2009 USC 4 22 29 -18 7
2009 Utah 19 18 32 1 14
2009 West Virginia 32 25 33 7 8
2009 Wisconsin 55 16 21 39 5
2010 Central Florida 46 21 33 25 12
2010 Florida 4 31 36 -27 5
2010 Georgia Tech 16 40 64 -24 24
2010 LSU 21 8 15 13 7
2010 Maryland 50 23 34 27 11
2010 Michigan State 50 14 32 36 18
2010 Mississippi State 50 15 30 35 15
2010 Nevada 50 11 26 39 15
2010 North Carolina 18 40 45 -22 5
2010 Oregon 11 3 9 8 6
2010 Penn State 19 40 54 -21 14
2010 TCU 6 2 7 4 5
2010 Texas 5 40 69 -35 29
2010 Texas A&M 40 19 27 21 8
2010 Tulsa 50 24 55 26 31
2010 Wisconsin 12 7 13 5 6
2011 Arkansas 15 5 16 10 11
2011 Auburn 23 27 54 -4 27
2011 Baylor 50 13 31 37 18
2011 Clemson 50 22 38 28 16
2011 Kansas State 50 15 35 35 20
2011 Mississippi State 20 35 45 -15 10
2011 Ohio State 18 35 40 -17 5
2011 South Carolina 12 9 23 3 14
2011 TCU 14 14 19 0 5
2011 USC 25 6 13 19 7
2011 West Virginia 24 17 25 7 8

Avg Pre-Season AP Ranking: 24.84

Next, the under-rated teams to end a season.

End-of-Season Under-Rated (Bias is at least -5)
Year Team AP Pre Rk AP Final Rk F/+ Final Rk AP Rk Change AP over F/+
2005 Arizona State 20 41 32 -21 -9
2005 Iowa 11 31 23 -20 -8
2005 Miami 9 17 8 -8 -9
2005 Michigan 4 28 14 -24 -14
2005 Tennessee 3 50 28 -47 -22
2006 BYU 40 16 9 24 -7
2006 Clemson 18 35 24 -17 -11
2006 Tennessee 23 25 18 -2 -7
2006 Virginia Tech 17 19 12 -2 -7
2007 California 12 40 22 -28 -18
2007 Clemson 41 21 15 20 -6
2007 Florida 6 13 6 -7 -7
2007 Oregon 29 23 8 6 -15
2007 Oregon State 33 25 18 8 -7
2007 UCLA 14 40 24 -26 -16
2008 Clemson 9 40 29 -31 -11
2008 Iowa 50 20 14 30 -6
2008 Pittsburgh 25 27 18 -2 -9
2008 South Florida 19 40 27 -21 -13
2008 Wake Forest 23 40 31 -17 -9
2009 Clemson 34 24 13 10 -11
2009 Florida State 18 34 27 -16 -7
2009 Miami 35 19 14 16 -5
2009 North Carolina 21 40 30 -19 -10
2009 Oklahoma 3 26 16 -23 -10
2010 Alabama 1 10 5 -9 -5
2010 Boise State 3 9 3 -6 -6
2010 Georgia 23 40 29 -17 -11
2010 Iowa 9 27 22 -18 -5
2010 Miami 13 40 24 -27 -16
2010 Pittsburgh 15 40 23 -25 -17
2010 South Carolina 29 22 16 7 -6
2010 USC 14 40 31 -26 -9
2010 Virginia Tech 10 16 11 -6 -5
2010 West Virginia 25 32 18 -7 -14
2011 Florida State 6 23 10 -17 -13
2011 Notre Dame 16 35 15 -19 -20
2011 Oklahoma 1 16 9 -15 -7
2011 Texas A&M 8 35 18 -27 -17

Avg Pre-Season AP Ranking: 17.69

Again, if anything, we see that teams that being the season highly ranked get punished, rather than enjoy a benefit due to their pre-season warm feelings. Now, I won't pretend this is the be-all and end-all of pre-season discussions. There are certainly other possible factors at play here:

- Of more importance is probably the penultimate AP Rankings for a season (the ones that actually decide how Sparty is going to get screwed what teams go to what bowls, etc. I would've done a study on those, but I need the F/+ Rankings for a comparison standard, and those are only available for the end of season.

- My cut-off of 5 may be completely off and leaving out valuable comparison data. Unfortunately, this took a lot of work, so I am not apt to do this again.

- I am not an expert. This is probably this biggest one :)

IRREGARDLESS: I thought this was a interesting look at Pre-Season polls and their lasting (or not) effects. Hopefully, it's at least a reference for like discussions in the futre. And again: feel free to tell me how wrong I am (I warn you: I put in a lot of work on this, though. I need to be validated).

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Off Tackle Empire

You must be a member of Off Tackle Empire to participate.

We have our own Community Guidelines at Off Tackle Empire. You should read them.

Join Off Tackle Empire

You must be a member of Off Tackle Empire to participate.

We have our own Community Guidelines at Off Tackle Empire. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker