When it comes to NCAA Tournament selection, we seem to be having a moment.
Coaches are up in arms about metrics they don’t understand. Teams who feel they were unfairly left out are spurning NIT bids. Everyone has a suggestion for how to improve the process – change the metrics, change the composition of the selection committee, expand the tournament, stop giving automatic bids… the ideas are flying.
I don’t claim to have final and definitive answers to all that. No matter what system is adopted, a line must be drawn, and teams on the wrong side of the line are not going to be happy about it. But I do think some changes could be made that would increase transparency, increase the perception of objectivity, and thereby reduce the noise. I’m going to group my thoughts under two headings. First, the structure of the tournament as it relates to size and automatic vs. at-large qualifiers; then second, I will dive more deeply into the selection of at-large teams and how that should work.
Structure
The system of awarding automatic bids to conference champions has been in place since the very beginning. Prior to 1975, all Division I conference champions qualified for the tournament, and then there were a few spots reserved for Independent teams. There was no such thing as an at-large bid for a team in a conference; if you weren’t the conference champion, either by winning the regular season or the tournament, then you didn’t make the NCAA Tournament. At this time, there was no seeding. Instead, teams went into predetermined slots in the bracket. So, for example, the West region bracket might have specified that the Pac-8 champion received a bye, and the WCAC champion played the Big Sky champion in the first round.
Starting in 1975, the tournament went through a series of changes that resulted in the current system. The two most fundamental changes were the introduction of at-large bids in 1975 and the introduction of a seeding system in 1979.
The introduction of multiple bids for a single conference is typically associated with the 1974 ACC Tournament in which Maryland, acknowledged by everyone to be one of the best teams in the country, fell in the final to the David Thompson-led NC State team in the “greatest game ever played”. The Wolfpack went on to win the national championship while the Terps went home. The seeming injustice of that for Maryland was the impetus for an expansion of the NCAA Tournament field from 25 teams to 32 and the addition of an at-large bid for certain conferences.
The next major change was the introduction of a seeding system in 1979, along with a further expansion of the tournament field to 40 teams. The expansion to 40 teams necessitated the addition of another round, meaning that some teams would have to play six games to win the tournament, as most do today. Once this step was taken, it was inevitable that the tournament would eventually expand to 64 teams, thereby filling out that first round and ending the practice of teams getting byes. This happened in a series of steps between 1979 and 1985, which was the first 64-team tournament.
The seeding system introduced in 1979 brought an end to the practice of predetermined conference matchups in the bracket and essentially brought the structure of the tournament to its modern form. Since then, the only thing that has changed is the number of teams. From 2001 to 2010, there were 65 teams which resulted in a single play-in game. In 2011, the current number of 68 teams was adopted along with the First Four round in Dayton.
Expansion from 68 teams has been brought up innumerable times over the years, and there is a wide variety of opinions on the topic. The Bigger is Better crowd argues that March Madness is an amazing event, and therefore wider participation would bring more joy to more people. The Status Quo crowd is concerned that diluting the participant pool would diminish the value of the achievement. Expansion would also bring a number of practical challenges – extending the overall length of the event; where and when you would hold the additional games; are fans going to care about and show up for the first round games among teams that aren’t very good; who would get the additional bids; and so on.
One thing that is important to point out about the NCAA Tournament is that most everyone recognizes what an amazing event it is. It generates a tremendous amount of revenue for both the NCAA and for the participating schools, and everyone is leery of killing the goose that lays the golden eggs. This explains why the low-major conference champions continue to have a spot in the tournament – because the participation of these teams and the Cinderella narratives that arise generate intense fan interest and therefore money. Otherwise you can bet that the power conferences would cut them out in a heartbeat and have their own tournament.
Over the past several days, I’ve heard a number of half-baked ideas put forward. Eliminate automatic bids? Not going to happen. Conference tournaments are too compelling and too exciting with the “Championship Week” brand and all that. Provide a certain number of guaranteed slots for the major conferences? I don’t see that happening either. Who decides how many slots each conference gets? What happens when the 8th place team from the ACC is better than the 7th place team from the SEC, yet they don’t get in? How would teams be selected within the conferences? With imbalanced conference schedules, that might require a selection committee within each conference. There are just too many issues.
I do think the field will eventually be expanded, but my guess is that it will be done in small increments that won’t fundamentally change the structure of the event. For example, it would be easy enough to go from 68 teams to 72. The First Four would become the First Eight, and you would need a second site in addition to Dayton. Those aren’t radical changes. If you were to go to 80, or even 96, now you’re talking about a lot of additional games. Where would they be played? Would you need to add a fourth weekend to the tournament? Is that too much of a good thing? Again, the tournament is so perfect as it is that you have to be very careful about making changes.
How At-Large Teams Are Selected
Now to get on to what I really want to talk about. This is where the real controversy lies. It seems to me that it boils down to two things: 1) the committee needs to decide and communicate what matters; and 2) the analytics they are using need to measure what matters. All the disagreements can be put under one of those two headings. Either they are disagreements about what matters, or they are disagreements about how what matters is being measured. Let’s take these two in turn.
What Matters
This is where at all starts. Quad 1 wins? Road wins? Bad losses? Non-conference strength of schedule? NET? kenpom? Early-season games? Late-season games? All these are different ways of getting at what matters to the committee.
I would like to suggest that the various views on what matters can all be aligned to one of two fundamentally different views, the Resume View and the Best Teams View. The differences between these two views are quite subtle, but they are crucial nonetheless, and they lie behind a lot of the differences and debates that we hear. Let me try to describe the two views.
| Factor | Resume View | Best Teams View |
| The Essential Question | Who has earned the right to a bid as a result of their wins and losses? | Who are the teams most likely to be successful in the tournament? |
| Main Direction | Looking backward at what you have done | Looking forward at what you will do |
| Margin of victory | Doesn’t matter | Matters |
| Metrics | Results-based (e.g. Quad 1/2/3/4 record, Strength of Record) | Predictive (kenpom, BPI, etc.) |
| Timing | All games count the same | Recent games matter more |
| Eye Test | Irrelevant | Relevant |
| Injuries, Roster Changes | Irrelevant | Relevant |
Do you understand the difference? Read it again, because this is the vital point. One thing that may be confusing is my assertion that the Best Teams View is essentially forward-looking. But I stand by that. Think about it, when you say that Team A is better than Team B, what do you really mean by that? Well, I say that a statement like that is essentially a prediction. What you really mean is that if Team A and Team B played tomorrow, all other things being equal, you would expect Team A to win. Perhaps they played last week, and Team B won. There is not necessarily a contradiction there, because you are making a forward-looking statement. Of course you are looking backward in the sense that you are drawing upon what Team A has already done, but you are doing so in an attempt to predict what they will do in the future.
Do you see how this shapes the debate? Are we using kenpom, or not? Well, that depends. If you take the Resume View, kenpom becomes irrelevant. If you take the Best Teams View, it may become very relevant, depending on your ideas on how to pick the best teams.
Are recent games more important than November and December games? In the Resume View, the answer is no. It’s about who you played and who you beat, nothing more. In the Best Teams View, recent games are more important, because they are more relevant to how good a team is right now and how they will fare in the tournament.
Best player just tore his ACL? In the Resume View, that doesn’t matter at all. It doesn’t change what you’ve already done. It doesn’t change what you’ve earned by your wins and losses. In the Best Teams View, it matters a lot. You may have been one of the 36 best teams before, but you aren’t anymore.
I hope you can see where I am going with this. Our collective thinking about this topic is so muddled primarily because we have failed to be clear about this distinction. Think about the talking heads and the variety of opinions about the Eye Test, about injury status, about the predictive metrics. What I am saying is that at the root of all that lie presuppositions, maybe unconscious but no less real, that when examined will turn out to be either the Resume View or the Best Teams View.
And I go on to assert that these two views are fundamentally incompatible, and that the selection committee’s basic problem is that they are trying to have it both ways. Either you can take the Resume View or you can take the Best Teams View, but you cannot take both. If the Resume View is right, then throw out the predictive metrics; banish all talk of the Eye Test; weigh every game the same. If the Best Teams View is right, then… well, good luck with that.
Which gets to my own position. I feel strongly that the Resume View is the correct one when it comes to picking at-large teams. Why? Well, it really comes down to this. In the immortal words of Herm Edwards, you play to win the game. That’s the object. Basketball isn’t like figure skating or gymnastics or diving where judges decide who wins based on who looks better. Basketball is about one objective, inarguable thing: who has more points at the end of the game. That’s it. And that’s why we love it. Whether you won by one point or thirty, it doesn’t matter.
Maybe you aren’t that good and got lucky to win your games. Doesn’t matter. Because you play to win the game.
Not only is this the only right and fair way to approach the decision, it also has a number of side benefits. It gets us out of the business of trying to predict the future, which humans are notoriously bad at. It gets us out of the subjective business of eye tests and other such nebulous criteria. It makes the task comparatively simple: whom did you play, where did you play them, and whom did you beat.
How What Matters is Being Measured
Now that we have arrived at the conclusion to rely solely on the Resume View, it may seem that our task is done. It’s simple. Wins are good. Losses are bad. The better the opponent, the better the win; the worse the opponent, the worse the loss. Your resume becomes the sum total of your wins and losses, with every game given equal weight. Throw out the predictive metrics, we don’t need them anymore.
You could envision a simple point system. Beating the best team in the country is 100 points. Beating the worst team in the country is 0 points. Losing to the best team in the country is 0 points. Losing to the worst team in the country is -100 points. That’s overly simplistic, but you get the idea. Add up the point totals for every team, and the top 36 get the at-large bids. Done.
But there is trouble lurking in the statement “the better the opponent, the better the win; the worse the opponent, the worse the loss”. Do you see it? The perceptive reader is saying, wait a second. I thought we just concluded that the whole notion of “better” and “worse” teams is too subjective. But if we are to reward teams more for beating good teams, we need a way to decide how good or bad teams are. Aren’t we back where we started?
In a way, we are; but I think there is a way out. The solution is that rather than banishing predictive metrics completely, we use them indirectly to determine how much credit to assign to a win, and how much debit to assign to a loss. The predictive metrics become the basis for our point system.
And so my radical proposal is this. Do away with the selection committee. No smoke-filled rooms, no subjective decisions. Instead, have a point system that is clear and transparent to everyone. When you win a game, you get points. When you lose a game, you lose points. What determines how many points you get or lose is where the opponent ranks in the predictive metrics. If Auburn is #4 in the predictive metrics and Tennessee is #7, then a win over Auburn is worth more points than a win over Tennessee. And a loss to Tennessee loses more points than a loss to Auburn. Publish the predictive metrics and the game-by-game point values on a website for everyone to see. No more guessing. You know where you stand. If you want more points, win more games against better opponents, and don’t lose games against worse ones.
There is one more devil lurking in the details. Which predictive metric or metrics should we use? Does it bother anyone else that Colorado State is #70 in the ESPN BPI but #38 in kenpom? Do we have any idea why that is the case or which one is right? Is Colorado State the 70th-best team in the country, or the 38th-best? If you’re going to use a system like the one I am suggesting, that difference matters a great deal to the teams that played Colorado State. My guess is that the NCAA doesn’t really understand how the metrics they are using work. They used kenpom, Sagarin, and ESPN BPI because they were available and the sources seemed credible. They are crossing their fingers and hoping that using them together closely approximates the truth. But just throwing more metrics together doesn’t necessarily improve the quality or accuracy of what you are measuring. Instead, you should strive to have one metric that is aligned with what you want to measure.
So that brings me to my final point. The NCAA should appoint a commission to develop their own predictive metric that measures what they want to measure and values what they want to value. Ken Pomeroy is a smart guy, but I’m not going to trust his metric at face value unless I understand how it works. Does the metric weight recent games more, or not? Does it adjust for end-of-game blowouts? How does it adjust for home-court advantage? Does it begin the season with a set of prior assumptions about team quality, or does everyone begin at zero? Pomeroy himself has been reasonably open about these things, and I’m not picking on him. What I’m saying is, the NCAA hasn’t been intentional about whether the way his metric works is they way they think it should work. Have they compared the differences in algorithm between kenpom and the BPI and then decided which they prefer? They should appoint a commission consisting of, yes, analytics gurus, but also analytically-minded coaches, players, and administrators to make decisions about how these things should be valued and create a predictive metric for the NCAA that reflects their values. Publish the algorithm for the predictive metric so that no one is guessing about how it works and so that it can be improved over time.
There you have it. My point of view on how to fix the selection process. Sure to be read by few and adopted by none, but it feels good to get it off my chest.