For decades, analysts had a hard time putting a number on offensive line performance. Other than using the eye test by watching game film, the one single number that was often used was the total number of sacks or the percentage of all pass plays that end in a sack.
In other words, offensive lines were held fully responsible for their quarterback getting sacked. This led to misunderstandings when evaluating historical passers, such as Joe Namath, who spent the majority of his career with the New York Jets. Compared to his peers, Namath featured very low sack rates but above-average interception rates. His overall efficiency, measured by a metric like adjusted net yards per attempt, for instance, was excellent. However, because of his low sack rate, people argued he was a mediocre passer who threw a lot of interceptions, but because his offensive line kept him clean, he was still able to produce big numbers. After people started to study sack rates in greater detail, the football community found out that, if anything, it’s the other way around: Statistical analysis suggests quarterbacks are generally more responsible for their sack rates than their interception rates, a finding that is often loosely referred to as sacks are a quarterback stat. This fundamentally changed the perception of Joe Namath and has us remember him as the Hall of Fame quarterback he was.
It follows that we can’t use sacks to evaluate an offensive line. Peyton Manning, arguably the greatest ever at avoiding sacks, would make a bad offensive line look good via sack rates, and on the flip side, a quarterback like Marcus Mariota featured very high sack rates throughout his five-year career despite playing behind an offensive line that graded out very well in PFF's grading system.
Survival analysis for offensive lines
With sacks eliminated from the discussion, we are left with basically nothing to statistically evaluate offensive line play prior to 2006, as 2006 is the year to which PFF’s thorough data collection dates back.
As well as grading every player on every play of every game, PFF has also charted quarterback pressures allowed. A pressure occurs whenever the quarterback is in danger of getting sacked and thus has to stop going through his progressions, either by leaving the pocket or getting rid of the ball in a hurry.
The pressure percentage is a much better metric to evaluate offensive line play because it is more of a measurement of how often the offensive line gets beaten. However, our own Eric Eager has recently found that, even for pressure rates, the quarterback is more in control more than one might think, so it’s still not a proper measurement for offensive line play. Eager also confirmed an intuitive result: The time a quarterback spends in the pocket is the most important variable to control for. For that reason, the analytics team from ESPN has created their pass-block/pass-rush win rate, a metric that measures how often an offensive line survives for 2.5 seconds.
In this article, we want to do a similar analysis, yet we want to consider all possible release times and account for confounding variables that help or hinder the pass protection.
PFF has charted the time a quarterback holds the ball until he throws a pass, gets sacked or takes off for a scramble drill on each play since 2012. On plays with pressure, we also chart the time taken for the defense to register pressure. The latter part is very important because there's a difference between a quarterback who is pressured two seconds into the snap, manages to escape a sack, resets his feet and finally releases the ball after four seconds and a quarterback who has a clean pocket for four seconds then finally gets pressured and throws the ball away. Both plays feature a time to throw of four seconds and a pressure, but in the first scenario, the blame goes mostly to the offensive line, while the quarterback (and potentially his receivers for not getting open) is to blame in the second scenario.
To account for this distinction, we will perform a survival analysis. Focusing on the time after the snap, we will then estimate the probability the quarterback is still kept clean, assuming he hasn’t released the ball up to that point. Naturally, we expect this probability to start at 100% just after the snap and then steadily decrease until it reaches 0% at some point, as an offensive line can’t hold forever. In practice, it turns out that seven seconds is a reasonable point to stop the analysis, as our data shows if a quarterback held the ball for seven seconds on each snap, he would be pressured 199 out of 200 times, hence the survival rate is close to zero at this point.
Without further ado, here is the league-average survival curve for 2019 (the dashed black line). To get an idea of the spread of these curves among NFL offensive lines, I added the curve of one of the worst offensive lines (Miami Dolphins) and one of the best offensive lines this season (Green Bay Packers).
© 2024 PFF - all rights reserved.