•About PFF's All-Coverage data: Since 2019, PFF has gathered comprehensive separation data for every snap played by coverage defenders and receivers in the NFL. This dataset includes information on matchups, assignments and separation. PFF has used this data to create a new metric, Advanced Coverage Grade, which has advantages over traditional coverage grades and other metrics.
• Two of the very best rank highly in the new metric: Fred Warner stands out especially in Advanced Coverage Grade, ranking within the top five linebackers in every season of data and claiming the top spot in three of the six seasons analyzed. Sauce Gardner has similarly dominated, finishing as the top-rated cornerback by Advanced Coverage Grade over the past two seasons.
• Unlock your edge with a PFF+ subscription: Get full access to all our in-season fantasy tools, including weekly rankings, WR/CB matchup charts, weekly projections, the Start-Sit Optimizer and more. Sign up now!
Estimated reading time: 7 minutes
PFF's Advanced Coverage Grade improves upon existing metrics for evaluating coverage players by considering not only their performance on plays where they are targeted but also on plays where they are not. Furthermore, it incorporates the difficulty of their coverage assignments.
Coverage Metrics
Several standard metrics are commonly used to evaluate players' effectiveness in pass coverage. These include passer rating allowed, yards allowed per coverage snap, completion percentage allowed and PFF's 0-100 coverage grade.
These metrics primarily focus on what occurs when a player is targeted in coverage. This is mainly due to the simplicity of evaluating a play at the point of a target. However, these metrics come with several limitations:
- Lack of control over targets: Players cannot fully control when they are targeted. For instance, a cornerback may be targeted more frequently if matched against the opponent’s primary wide receiver.
- Bias toward “bad” reps: Players are often targeted on their “bad” reps, as quarterbacks tend to throw to open receivers.
- Limited data: Metrics based solely on targeted plays use only a small fraction of a player’s total snaps, which limits their ability to provide a comprehensive evaluation.
All-Coverage Data Collection
Since 2019, PFF has gathered comprehensive separation data for every snap played by coverage defenders and receivers in the NFL. This dataset includes information on matchups, assignments and separation. Multiple articles have explored this dataset and its applications, including models built around it. One notable example is How PFF Route-Running Grades Teach Us More About Coverage Matchups.
In this collection process, “matchup” refers to the opposing player a defender is responsible for covering. “Assignment” refers to the type of coverage the defender plays (for example, the middle zone in Cover 3). The “separation” refers to the amount of separation the defender allows in their matchup.
A key piece of PFF's all-coverage process that gives it an advantage over tracking-based separation methods is that our data collection team measures separation when a player should have been open based on the route run. This approach provides a more accurate measure of a defender’s ability to limit separation and a receiver’s ability to generate it compared to methods like average yards of separation.
With data from 2019 through Week 13 of the 2024 NFL season, we now have a significant dataset to evaluate the stability of metrics derived from all-coverage data. For this study, separation measures are categorized into four buckets:
- The defender won the route
- Draw
- The defender lost the route
- The defender badly lost the route
Several different bucketing schemes were tested, but this proved the most effective in terms of the metrics laid out below.
The Model Explained
Two “expectation” models were developed to integrate information from both the PFF all-coverage data collection process and the base PFF data collection process.
These models aim to predict a player’s base PFF coverage grade on plays where they are targeted. For reference, the PFF coverage grade ranges from -2 to 2 in increments of 0.5.
The first model uses various information about the play, including a player’s alignment, assignment, overall coverage scheme and whether the offense used play-action.
For this purpose, “CatBoost” models were trained for a small number of iterations and used a very large minimum leaf size to avoid overfitting.
The relative feature importances for the model can be seen in the image below. The orange bars represent the features in the base model without separation allowed, while the blue bars represent the features in the model that include separation allowed.
The final advanced coverage grade on the play level is determined by subtracting the result of the base model from the result of the separation model.
Evaluating Player Metrics
When assessing the effectiveness of new or existing metrics, a fundamental question arises: How do we measure a metric’s effectiveness?
One commonly used and insightful approach is to evaluate the metric's year-over-year stability, often quantified as the correlation of the metric at the player level across seasons.
For coverage metrics, it’s crucial to analyze stability for each position independently. This avoids issues like Simpson’s paradox, where aggregate data masks or distorts relationships present within individual groups. For instance, differences in what a player is asked to do (e.g., a cornerback primarily in man coverage versus a safety in zone coverage) could skew correlations, making them reflect scheme or role rather than individual ability.
By isolating positions, we ensure that the metrics more accurately evaluate player performance rather than the context of their usage. This approach provides a clearer, more reliable understanding of a metric’s effectiveness in measuring true coverage ability.
Another approach is to look at a metric’s predictive power regarding team success. Retrodiction is used in basketball to compare all-in-one metrics of player performance. The idea is to put all metrics on a level footing by using equivalent assumptions about injuries, rookies and players with limited history in the league.
For this purpose, we calculate a snap-weighted mean (using snaps played in the current season) of players' metrics from the previous season for each team and measure the correlation to EPA allowed per pass play in the current season. We adjust rookies and low-snap players toward replacement level.
Various values were used to determine replacement level. For the final results, replacement level is the 10th-percent value for each metric among player seasons in which the player played more than 75 coverage snaps.
The results indicate that incorporating information from non-targeted plays improves coverage metrics' stability and predictive power.
The added information seems to be particularly informative for off-ball linebackers, as they see the largest stability increase.
PFF separation grade is derived from a straightforward averaging of separation results without incorporating expectation models.
Target rate demonstrates a particularly high year-over-year correlation for safeties, likely due to the significant impact of their alignment and role within the defense. Safeties who frequently line up close to the line of scrimmage tend to experience higher target rates, while those who primarily play as deep safeties are targeted less often.
This phenomenon reflects a lingering effect of Simpson’s paradox, even after dividing players by position. The observed high correlation in target rates likely arises from differences between groups within the dataset, such as role-specific alignments, rather than capturing the true stability of an individual player’s metric.
The Results
Top players
First, let's examine some expected outcomes. Sauce Gardner and Fred Warner have consistently been regarded as elite coverage players at their respective positions in recent years. Let’s analyze whether the metrics support this perception.
Both players excel across most coverage metrics. Warner stands out especially in Advanced Coverage Grade, ranking within the top five linebackers in every season of data and claiming the top spot in three of the six seasons analyzed.
Gardner has similarly dominated, finishing as the top-rated cornerback by Advanced Coverage Grade over the past two seasons. While Gardner has seen a slight dip in performance this season, being surpassed by his teammate D.J. Reed, he still ranks among the league's best.
Early indicator
Pat Surtain and Jaylon Johnson exemplify the value of incorporating separation data into coverage metrics. Despite variations in traditional target-based metrics, both players have consistently excelled at limiting separation, ranking near the top of the league in Advanced Coverage Grade throughout their careers.
Their ability to restrict separation, even from their rookie seasons, foreshadowed their subsequent improvement in target-based metrics, cementing their status as two of the league’s premier cornerbacks.
Breakout candidates
Four early-career defensive backs stand out as potential breakout candidates in 2024, starting with Christian Benford. Even as a rookie, Benford demonstrated a strong ability to limit separation, a skill he has only refined as his career progresses.
Next, Ja’Quan McMillian appears ready to establish himself alongside Trent McDuffie, Taron Johnson, and Charvarius Ward as one of the league’s premier slot corners. Like Johnson and Surtain, McMillian has earned high marks in Advanced Coverage Grade early in his career. Despite facing a high target rate—partly a result of playing in the slot and sharing the field with standout outside corners Pat Surtain and Riley Moss—McMillian has also earned a solid PFF 0-100 grade.
Brian Branch has been near-elite in both traditional and advanced metrics through his one-and-a-half years in a hybrid role with Detroit.
Finally, Garrett Williams may not have had as strong a start as the others on this list, but he appears to be on a trajectory similar to Christian Benford. His marked improvement from 2023 to 2024 is promising, and he currently ranks 10th in the league in Advanced Coverage Grade among corners.
More on Linebackers
As previously noted, incorporating separation data significantly enhances the stability of coverage metrics for linebackers. Beyond Fred Warner, several other linebackers have consistently ranked among the best in Advanced Coverage Grade since 2019.
Although Davis and Alex Anzalone have not quite reached Fred Warner’s level of production on targeted plays, both have excelled at consistently limiting separation throughout their careers.
Additionally, two second-year players have emerged as standout performers in Advanced Coverage Grade through the first half of the 2024 season.
Roquan Smith and Patrick Queen have also been near the top of the linebacker rankings over the past few years. The pipeline of elite coverage linebackers seems to be continuing in Baltimore with Trenton Simpson.
Henley and Simpson have performed well in coverage with larger roles this season; neither played enough snaps to qualify for these rankings in 2023.
Briefly on safeties
Even with the inclusion of separation data from the all-coverage process, evaluating safeties remains challenging. Safeties frequently deployed in deep zones often do not rank highly in advanced coverage grade due to limited matchup opportunities with separation data. Conversely, safeties in hybrid roles or those playing closer to the line of scrimmage exhibit more consistency in the metrics.
Notably, several safeties currently in the top 10 for 2024 have consistently been among the league's best throughout their careers.
© 2024 PFF - all rights reserved.