Coming Wednesday: the rankings that matter

By Christan Shirk

On Wednesday, October 19, the NCAA Division III men's and women's soccer committees will release the first of their four weekly regional rankings—the first three leading up to and the the last one being the basis for the at-large tournament selections. These rankings are based on the same primary and secondary criteria that will be used for awarding at-large tournament berths. But before getting into that, a more general introduction to the rankings is in order for those new to D-III soccer or those still not clear on the distinction and significance of the different rankings that exist.

By this point in the season, even new fans should be aware of the NSCAA regional and national rankings and the D3soccer.com national Top 25, but some might be confused when hearing about the NCAA regional rankings and have questions such as: Are these yet another set of rankings? Why haven't I seen these rankings yet this season? Are they important? Do they matter? Adding to the potential confusion, the NCAA includes the NSCAA (note the “S”) national rankings on their website even though they are not NCAA rankings but rather rankings done by the National Soccer Coaches Association of America. You may have heard some fans chide that “the NSCAA and D3soccer.com rankings don't mean anything; it's the NCAA rankings that matter.” And they are right in that the NSCAA and D3soccer.com rankings do not play any part in the process of selecting teams for the NCAA championship tournaments; the NCAA's own rankings do as will be explained below.

However, I think it’s unfair to say the NSCAA and D3soccer.com rankings don't mean anything. They are useful for acknowledging the most successful teams and for fans to discover what schools outside their conference and their region are having great seasons. If done well, they can also give fans a feel for the relative strength of the teams they have seen when put in the national context. That is, they can be educational and informative. But equally they have entertainment value and can spark conversation and debate among fans. D-I basketball fans can banter and argue over the rankings while knowing they do not decide tournament berths, and there's no reason D-III soccer fans shouldn’t as well. Those who pooh-pooh these rankings as meaningless and chose to ignore them miss the point and miss the fun.

That said, it is the NCAA regional rankings that are a critical piece of the pre-tournament puzzle, so let’s first have a look at how they work and why they matter. Then, at the end of this column, I will share some comments and observations rooted in my experience of closely following and analyzing the regional rankings and at-large tournament selections for over a decade.

NCAA REGIONAL RANKINGS

The NCAA regional rankings are done by the same regional and national committees which will make the at-large selections for the men's and women's NCAA tournaments and the rankings are done by applying the same criteria which is used for making the at-large selections. The rankings are released following the fourth last, third last, and second last weeks prior to the tournament selections being made. Therefore, by design, these rankings are a direct foreshadowing of the at-large selections providing a certain level of transparency to the at-large selection process. It is for this reason that these rankings are so important and meaningful.

All information about the regional rankings is found in the Division III Soccer Pre-Championship Manual. Much of what follows highlights, summarizes, or quotes the manual.

Ranking Committees

The NCAA regional rankings are done by the eight-member NCAA Division III Men's and Women's Soccer Committees which are each composed of the chairs of their respective regional advisory committees. The Regional Advisory Committees assist the national committee in evaluating teams.  These are the same national and regional committees which will make the Pool B and Pool C at-large selections for the men's and women's NCAA tournaments. The members of these committees can be found on pages 10-13 of the Pre-Championship Manual.

Release Dates

As per the Pre-Championship Manual (pgs. 13, 18 and 19), the rankings will be released on the following dates which correspond to the last three Wednesdays prior to the tournament selections being made and the Monday that the at-large selections and tournament fields are announced:

Like the NSCAA and D3Soccer.com rankings, they are based on results through the Sunday prior to their release.

Where to find the Rankings

The rankings are posted by the NCAA on their Division III men's and women's soccer webpages under "Rankings" where you must select the "Regional Rankings" option from the pull-down menu.  They are also conveniently available here on our site from the “Rankings” pull-down menu above or by clicking on the following links:

Men's Regional Rankings

Women's Regional Rankings

Size of the Rankings

The number of teams ranked differs from region to region and from men to women based on the number of eligible teams in each region. The top 15 percent of eligible teams (or a minimum of four) are ranked by the committees. As per the Pre-Championship Manual (pgs. 18-19), the number of teams to be ranked in 2016 is as follows:

The only changes from last year are that one more men's team will be ranked in the East and Mid-Atlantic regions, and one less men's team will be ranked in the North region. The regional alignments, school sponsorship, and eligibility by region can be found in Appendices B and C (“Men's Sponsorship” and "Women's Sponsorship, respectively) of the Pre-Championship Manual (pgs. 28-43).

Ranking Criteria

The rankings are done by applying the same criteria which is used for making the at-large tournament selections. The at-large selection criteria are found in Section 2.4 of the Pre-Championship Manual (pgs. 21-22). The criteria is divided between primary and secondary criteria, the latter only being considered if the former does not enable a distinction to be made between schools. The criteria underwent a significant change prior to the 2013 season when the in-region/out-of-region distinction was abandoned. Previously only games versus "in-region" opponents were consider as part of the primary criteria with secondary criteria considering out-of-region and non-Division III competition.  Now, primary criteria considers all Division III opponents while results versus non-Division III opponents (NAIA, NCCAA, Division II, Division I) are considered secondary criteria.

Regular season and conference postseason matches are considered.  See the sections that follow for an explanation/clarification of the results versus ranked teams and strength of schedule.

Results versus Ranked Teams

Prior to 2013, a team's results versus ranked teams (one of the primary criteria) was based upon the official clarification in the Manual that "once a team is ranked  . . ., it is always considered ranked."  That was changed starting in 2013 with ranked teams being defined as those teams ranked "at the time of selection" (Pre-Championship Manual, pg. 22), in other words, only the teams ranked the previous week.

Obviously, for the first weekly regional rankings of the season, there is no previous ranking and thus there are no results versus ranked teams.  Consequently, the regional data sheets (see section further down) for the first weekly rankings do not include teams' records verus ranked opponents, but starting with the second weekly rankings this criteria is in play and is among the criteria listed in the regional data sheets.

"Results versus ranked teams" criteria spelled out

Strength of Schedule

The Strength of Schedule (SOS) used by the Division III soccer committees is based on Opponents' Average Winning Percentage (OWP) and Opponents' Opponents' Average Winning Percentage (OOWP) with home and away multipliers being applied for men’s soccer (since 2011) and women's soccer (starting this year). A multiplier of 0.85 is applied for home games and 1.25 for away games. Neutral site games are unfactored. An explanation with an example of these calculations is found in Appendix D (pgs. 44-45) of the Pre-Championship Manual.

Note: There is a 40% difference between the home and away multipliers, meaning an away game is worth nearly 50% more than a home game (1.25/0.85 = 1.47). Therefore, travelling to a team with a .550 winning percentage is worth more to a team's SOS than playing a home game against a team with a .800 winning percentage.

Regional Data Sheets

Along with the rankings, data sheets for each region are made available.  These data sheets, which include all teams in each region, provide some (but not all) of the data that was considered by the ranking committee.  The following data is listed: record and winning percentage against Division III opponents, results versus ranked Division III opponents, Division III SOS (all primary criteria), and non-Division III winning percentage (secondary criteria).  These sheets allow for a look at the numbers the committees had in front of them and therefore insight into why some teams are ranked and others not. The NCAA provides links to this data below the rankings. Direct links to the latest released data sheets are given below:

The data sheets can also be accessed by clicking the links on our regional rankings pages.

Definition of In-Region Competition

Starting with the 2013 season, the in-region/out-of-region distinction no longer matters for ranking and at-large selection criteria. However, the NCAA has not removed all encouragement to minimize travel and missed class time as teams are still required to play a minimum of 70 percent of their games against in-region Division III opponents to simply be eligible for at-large tournament selection. So the distinction could still be important for teams that like to fill their non-conference schedule with a good number of opponents from outside their region. But given how broadly in-region competition is defined on page 21 of the Pre-Championship Manual to include much more than just other teams from your defined region (e.g. New England, East, Mid-Atlantic, etc.), this requirement isn't difficult to meet.

Published Final Rankings

As part of the at-large tournament selection process, the committees do final rankings that include the results from the final week prior to the tournament, usually the completion of conference tournaments. After five years in which they were not released, these final rankings will once again be published following the announcement of the tournament fields, which will go a long way to answering many questions about why certain teams were at-large selections and others not.

FORESHADOWING THE AT-LARGE SELECTIONS:

SOME COMMENTS AND OBSERVATIONS

• As just mentioned above, for the first time in years we will get to see the fourth and final rankings that are the basis for the committee's at-large selections. These final rankings take into account the final week of games (usually conference tournaments). However, they will only be released after the tournament field, and thus at-large berths, are announced.  Therefore, we will still only have the first three regional rankings in order to anticipate which teams will be selected.

• Because these rankings are done by the same committees that make the at-large selections, using the same criteria as for at-large selections, they (as intended) have in the past very accurately foreshadowed the eventual at-large selections. From 2003, the year of their inception through to 2013, there was only one year (2010) in which there was more than one real head-scratcher on the men's side with many years having none.  2014 was another year in which there were two or three very hard (impossible?) to predict selections, but predictability was restored last year.

• As such, a team that is not ranked in the third weekly rankings has virtually no shot at a Pool C berth. You can get your hopes up, but it's just not going to happen if history is any guide. In the past nine years (2007-2015) no men's team that was unranked in the third weekly rankings (those released the Wednesday before the selections) got selected for the post-season tournament. Way back in 2005 and 2006 it happened once each year, and at least in one of those cases the results of the final week easily explained the unranked team having improved their standing.

• There will probably be twice as many Pool C candidates in the rankings as available berths.  For example, last year there were 39 Pool C men's teams in the third rankings but only 18 Pool C berths available.  In the four years before that, the ratio was 38/18, 44/19, 38/20 and 40/19. So it isn't good enough to simply be ranked to make the NCAA tournament as most regions will have three to five ranked teams not selected. Even the strongest regions do not have all their ranked teams selected. A team typically needs to be in the top half of their regional rankings to be selected. Depending on how little separates teams, being in the top third of the third weekly rankings may not even make a team a safe bet for an at-large berth as results in the final week could see them fall in the final rankings and land on the bubble or outside of real contention for selection.

• History indicates that the ranking of teams changes little in that last week (between the third weekly rankings and the at-large selections being made). That is, rarely does a lower ranked team in the third weekly rankings get selected ahead of a higher ranked team. Last year there were four examples of this out of eighteen men's selections, which was an unusually high number. In 2014 there were only two examples of this (both involving teams ranked one after the other) and most years there has been just one instance of this. Two of last year's instances (New England #5 Tufts over #4 Middlebury, Mid-Atlantic #5 Dickinson over #4 Elizabethtown) involved teams ranked one after the other. In the case of North #6 UW-Oshkosh being selected, the #4 and #5 teams both lost in the final week. And Central #6 Chicago being given a Pool C berth over #4 North Park can be attributed to Chicago's win over Central #1 Washington U. in the final week. It makes sense that there would be little change from the third to the final rankings as one week only represents about 10% of the total schedule. There has also been a sense that conference tournament results do not have extra weighting because they are the most recent results nor because they may be considered "big" games, which is consistent with the established selection criteria. So do not expect big jumps or falls due to the final week's results.

• The 2013 change in the definition of a ranked opponent from "once ranked, always ranked" to "ranked at the time of selection" has increased the potential for more movement from week to week than previously. In 2014 that change in the criteria made a significant impact in at least one case.  John Carroll was ranked #2 in the Great Lakes region in all three weekly regional rankings during their 14-game winning streak, however, they did not receive an at-large berth after losing in their conference final. Their record versus ranked teams was 4-0-0 at the time of the third weekly rankings which, based on their #2 ranking, obviously more than offset a mediocre SOS of 0.535. So it was a surprise when they did not get selected, but their record versus ranked teams at the time of the final rankings used for at-large selections had dropped to a more modest 2-1-0. That was due to three changes in which of their opponents were ranked in the third rankings versus the second rankings, costing them two wins and adding a loss.  It is still hard to understand their omission given two teams with lower overall winning percentages, similar or lower SOS, and just one win versus ranked opponents were selected, but that's another discussion altogether. The point to be made here is that under the current definition of a ranked opponent, at-large selections are a little less predictable than they were prior to 2013. (Note: Starting next year, the definition of ranked opponents for the purposes of the final rankings will be those ranked in either the second or third rankings, which should make the selections more predictable) 

• Comparison of the regional data sheets with the rankings (and the eventual at-large selections) has shown over the past decade that the committee highly values strength of schedule. The other criteria that can be deduced to be very important is record against ranked teams, or more precisely, wins over ranked opponents. Losses to ranked teams don't seem to be penalized so much as wins are rewarded. In other words, the committee wants teams to play challenging schedules and doesn't mind if a team drops some of their toughest games as long as they demonstrate in other games that they also can win against top opposition.  So if you do not understand why one team isn't ranked and another team is, or why one team is ranked higher than another, it probably is related to SOS and results against ranked teams.

Comments or feedback for the author?  Email Christan Shirk.