Statistically Speaking- Utah at USC

Let's Talk Talent

Analysts use a number of different metrics to evaluate a football team. A core analytical decision is whether you are going to only consider what’s actually happened on the field, or if you are going to use your eyeball test or some other measure of a team’s inherent talent, ability, or potential to adjust your analysis. This decision is what’s at the core of the disconnect between the two sets of predictions for Saturday’s game.

When it comes to what’s been done on the field, Utah is just better than USC this year. Efficiency metrics differ from the total team metrics like FPI and S&P+ by virtue of being more transparent. Total team metrics throw meaningful but obscure numbers into an opaque formula, and publish the resulting rankings. The only way to evaluate the accuracy of these numbers is to evaluate the accuracy of their predictions. Utah's S&P+ of 88.1% doesn’t actually mean anything. We evaluate it by looking at the rankings. Utah (18th) is higher ranked than Cal, ASU, and all of their other opponents except for Michigan (2nd). That ranking doesn't match our perception, so it raises some questions about the accuracy of S&P+.

Efficiency ratings, on the other hand, are a very basic formula- yards per play on offense and defense, adjusted for the quality of your competition, which is also measured in yards per play. By those metrics, Utah has excelled. FEI and ESPN’s efficiency metric both work in that way, and they each rank the Utes very highly.  That’s because they don’t make any attempt to adjust for a team’s talent level. I can't be sure that FPI and S&P+ do make those adjustments, but that would match the biases they display.

Using talent ratings to adjust your advanced metrics isn’t necessarily a bad idea: talent (typically measured by recruiting rankings) correlates with winning. To what degree talent causes winning is an open question. This is further complicated by the inherent biases in recruiting statistics. Recruiting services are, at their core, press services. They make money when fans buy memberships and click on articles. This creates an echo chamber of regional and brand bias that encourages talent evaluations which give more stars to recruits in big football areas and more stars to recruits recruited by big football teams.*

I’m not saying talent doesn’t matter. It absolutely does. If you could wave a wand and make Kenneth Scott an inch taller and two tenths of a second faster, you would. However, the tool that analysts use to measure talent is flawed, and good analysis should take that into account. These measurement errors percolate down into advanced team metrics like FPI and S&P+.

Is the talent evaluation of USC over Utah benefiting from those inherent recruiting ranking biases? Are we considering injuries and other game day factors that may be impacting the team’s talent? If these factors are present, we should give more analytical weight to team efficiency metrics, and less to overall team metrics which include talent.

USC’s recruiting rankings vastly outstrip Utah’s, which should come as no surprise. Over the last five years, the Trojans have averaged 3 five star recruits, 11.4 four star, and 5.6 three star. The Utes have gotten 0 five stars, averaged 2 four stars a year, and 16.2 three stars. USC’s average conference ranking has been 1.4, while Utah has been around 8th.

A way to measure some of the recruiting biases is by looking at NFL draft scouting. Rather than press services, these services reflect talent evaluation where money and livelihoods are on the line; NFL scouts get it right, or they are out of a job. There are still biases at play, but NFL rankings can test recruiting rankings.

2016’s draft evaluation is largely based on watching film. 2018 and 2019 draft evaluation is going to have to piggyback on recruiting rankings; there just isn’t any film to watch on redshirt freshmen tight ends. USC dramatically outstrips Utah in terms of ‘NFL talent’ in 2018 and 2019. The Trojans have 26 NFL prospects (which I’ve defined as ranked 25th or better at their position), while the Utes have 3. However, in 2016, the Utes’ NFL talent gap is narrow. Both teams have 8 NFL prospects. The Utes average ranking is 12.75, USC’s average is 9.5. There is a significant talent evaluation problem at the prep level, at least as it relates to what Utah is doing. We should still consider these numbers, but not without adjusting for that bias.

There’s also injuries to consider. The Trojans are down their starting running back and multiple wide receivers, as well as their starting center and starting left tackle. These injuries have impacted their performance and could prove to be critical against the Utes on Saturday. Meanwhile, the Utes are largely healthy, although the injuries at the tight end position will have some impact on the throw game.

USC has a talent advantage; that’s true almost every game they play. However, a closer analysis suggests that the advanced metrics which rely on recruiting rankings and other methods of talent evaluation are overstating their advantage. This week, we’ll give a bit more weight to efficiency metrics; to what these teams have actually done on the field.

Advanced Metrics Overview

Say hello to FEI! This pure efficiency metric isn’t fully published until this week. It’s always nice when it finally comes out because the Utes are, without fail, better regarded by this metric than by any other. Why? I don’t know. Darn proprietary formulas. In a week in which I’m going to be lending more weight to efficiency metrics because of the factors discussed above, it’s nice to have more than one reliable metric to lean on.

The Trojan’s profile this year is offense first. The offense has been fantastic; they rank in the top 10 in every advanced metric, as well as in yards per play and a number of statistics which measure explosiveness. The defense has been about average, although they have few glaring weaknesses. One thing the Trojans have been particularly bad at is controlling methodical (10 or more play) drives. They give up a methodical drive on about 16% of opponent’s opportunities, 96th in the nation. The Utes’ offense is well suited to exploit this weakness; their 20.6% methodical drive rate is 7th nationally. Overall, the Utes have a defense that looks like it can contain even the elite offense of the Trojans, and while the Utes offense isn’t truly elite, it is plenty good enough to put a hurt on USC.

Vegas and the polls

The Utes opened as 3 point underdogs, and the line has shifted slightly towards the Trojans to 3.5. The majority of the public money has been on the Utes, so the fact that USC’s advantage has grown slightly indicates that there are some large money bets behind the scenes for the Trojans. Massey’s aggregator has the Utes at number 1, and USC all the way down on 35. This is the only metric that shows a wide gap between the two teams; every other measurement suggests a very close game.

Stat-Head Pick

The numbers almoat universally suggest a close game on Saturday. With the injuries to the offensive line and at wide receiver, I project a game where USC will struggle more than usual to score. They’ll still get their points, but Kessler will be under pressure early and often, and the receivers will take a bit longer to get open than usual.

Adoree Jackson is an incredible player, but his move over to offense is going to put yet another hole in a USC defense that’s been susceptible to giving up plenty of points in all sorts of ways. Utah has pitted Travis Wilson against only one soft defense this year, and he absolutely lit them up. The Trojans are much more like Oregon on defense than they are Michigan or ASU. They won’t stop Wilson or Booker on Saturday.

Utah 37, USC 28.

*Utah is a great example of this bias at work. It’s unlikely that Utah’s prep football quality has improved exponentially in the last few years. A more compelling explanation for the sudden surge of high star-rated players is the continued relevance and growing fan base of the Utes. This attention has driven recruiting services to send more evaluators to Utah. Once you put eyes on players instead of stat sheets, evaluators are bound to notice more talent.