College Football Playoff Rankings: They're not as crazy as you think, except for TCU

There's been a great deal of angst over the last 14 hours, as there always is, with the release of the first CFB Playoff Rankings last night. As expected, the loudest cries are from those who think the Big 12 is getting shafted again and the committee's clearly biased in favor of Alabama.

The catch is, the numbers tell a different story altogether.

Computer systems, in and of themselves, are not the be-all, end-all. No one computer system holds all the answers to the questions of the football universe; each one, individually, has some flaw, some thing which it values just a little too much or not quite enough. So we can't just look at one computer ranking, or even just a handful, unless we're willing to endorse that one or that handful as being objectively superior -- and explain our work.

What we can do, however, is to simply use them all, including the human polls, to establish a consensus. And that's what Ken Massey does every week, alongside his own computer ranking. Massey doesn't share the details of his ranking, which bothers some folks who will insist that only an open-source ranking is valid. But he does, at least, put his own ranking up against every other for comparison's sake.

The comparison, which we know as the Massey Composite Ranking, is in and of itself a pretty good judge of quality. Any ranking which strongly deviates from the MCR is an outlier by definition, as the MCR presents a true consensus of over 100 ranking systems. It's really hard for some whackadoodle outlier to skew a team off the consensus.

Just looking at the MCR by itself, the top ten teams right now are, in order: Clemson, Ohio State, LSU, ALabama, TCU, Notre Dame, Michigan State, Florida, Iowa, and Baylor. Looks mostly pretty familiar, doesn't it? The top four, although in a different order, are the same as the top four in the CFP ranking. TCU and Baylor are effectively swapped, which presents a conundrum, but the rest of the top ten is right about where the committee has them and the top ten teams are the same in both rankings.

But that probably doesn't tell the whole story or convince anyone of anything. So what we're going to do now is compare individual teams to one another across the board. What we've done is to take all the rankings which comprise the composite for each team we're discussing and sorted them, top to bottom, and then created area graphs which overlay the "better" team over the "worse" team. In some cases, they're close enough together that we need to graphs to allow you to judge for yourself (i.e., the second graph is the same as the first, only with the teams reversed).

The reason we sorted the rankings was basically so your eyes wouldn't melt. A graph which just compares the rankings of two teams on an one-to-one basis looks like static, and doesn't tell you much more than this method.

TCU vs Baylor

This is the most thorny comparison in the sequence, because it's the one in which the committee's opinion actually diverges from the computers. The committee has -- rightly, I think most would agree -- given a great deal of weight to the eye test for Baylor, causing the Bears and Frogs to basically swap places. Here's the Baylor-TCU charts:

These two charts tell the same story, but allow you to see it quite a bit better. The first image more clearly shows you that about 60% of the ranking systems have TCU as the better team; after Baylor's top 40 or so rankings, they start falling off precipitously. Nearly half the systems don't have Baylor in the top ten, whereas only about 20% of the systems have TCU outside that group. And at the tail end, Baylor just slides off the map. (Note: one ranking, which had Baylor at 98th, was excluded for both sanity and scale.) TCU, meanwhile, never dips lower than 18th.

The second image shows you that up until the point where Baylor begins dropping off, the Bears are basically just one place ahead of the Frogs. And that is why the computer composite has TCU at #5 and Baylor at #9. But the computers don't have an eye test, and that's why Baylor is viewed far more favorably by the committee than is TCU; Baylor is just wrecking fools while TCU is often just getting by, largely thanks to the massive injury depletion in the defensive corps.

Florida vs TCU and Baylor

Our transitive rankings guru, JeffP171, wanted to know how Florida compared. Here's a look at how they stack up against TCU and Baylor.

The two Big 12 schools are both higher than Florida about halfway through the sample, but then Florida overtakes Baylor pretty decisively:

I'm not including the graph of just TCU-Florida, but do be aware that while there are individual systems which have the Gators higher, but when sorted TCU is never below Florida.

Lesson? There's a reason why Alabama and Notre Dame are in the top 5 and Florida isn't. Why? Because there's a reason Alabama and Notre Dame are in the top 5 and Baylor isn't.

Baylor and TCU vs Alabama

There's one way of looking at Baylor vs Alabama which makes things so blindingly obvious you just can't escape the conclusion the committee reached:

You can hardly even see Alabama on this graph, which means that at virtually every step along the way Alabama is ahead of or equal to Baylor. Of course, it's the equal to part that's a bit deceptive:

Basically, about 40% of systems, when sorted, consider Baylor and Alabama equals. It's the other 60% where the problem arises, and one can reasonably surmise that these are systems in which SOS is a much larger factor.

TCU and Alabama, however, are a bit clearer. There is no point on the scale where TCU is superior to the Tide, but unlike Baylor there is also no point on the scale where Alabama is vastly superior to the Frogs, except at the very tail end. Again: SOS. The big difference here, and a reasonable cause for concern, is that TCU's SOS isn't that bad, as evidenced by their remaining neck-and-neck with the Tide. The committee, however, apparently seems to think Minnesota is garbage, and we're going to have to see how that plays out.

Notre Dame

This one is pretty straightforward; Alabama is a step ahead of the Irish pretty much across the board, although in that section between 8-11 where the Irish are obscured they have a small advantage on the Tide.

TCU, on the other hand, is basically indistinguishable from Notre Dame. Again, statistically the committee is just plain wrong about the Frogs.

Baylor, again, suffers from SOS issues. They're superior to the Irish for a little over a third of the scale before being overtaken.

Ohio State

KSUEMAW! wanted to see how Ohio State compared to the Big 12 schools. There is no question that Ohio State should be ahead of TCU and Baylor, so let's just put that one to bed right now.

Stanford

JT wanted to see where Stanford stacks up, specifically against Alabama. Well, okay. We can do that.

In closing

We don't trust computers -- or, more accurately, we don't trust the people who program them. But we're looking at over 100 distinct samples here (112 to be exact, although that number fluctuates a bit week to week). To suggest that even a plurality, much less a majority, of the guys who devise these systems are in the bag for one team or conference is absolute lunacy, so let's just set that aside before anyone brings it up. They do have biases, and perhaps a few are set up to generate a desired result. But then it's also probably important to remember that the top four teams in ESPN's loathed and reviled FPI are the three undefeated Big 12 teams and Oklahoma. If anyone had any motivation to jimmy their rankings, it would be the Deathstar.

By looking at the entire spectrum of rankings, we can get a pretty good idea of who should be ranked where. And while the committee has seemingly dropped the ball on TCU, they're also somewhat over-valuing Baylor -- although both of these things can also be explained by the eye test. For the Big 12's purposes, it's actually a good thing, especially as teams start knocking one another off. TCU beating Baylor will look better to the committee than it will to the computers, thus likely bringing TCU back into line with consensus; Baylor winning that game will merely give the committee whatever impetus they require to move Baylor into the top four, even as Baylor's statistical profile drastically improves due to the backloaded schedule.

But aside from that pair of intertwined anomalies, both of which are actually explicable, the committee got things right last night. Sure, that creates a problem for people whose immediate reaction is to complain about the committee on general principle. But unless your argument is that TCU is being shafted and/or Baylor is overrated, the numbers simply don't back you up.