Does Defense Matter More in the NFL Postseason?
| 15
|
by Derekcs
In Pro Football Prospectus 2006 and 2007, it is shown that defensive performance as measured by Defense-adjusted Value Over Average correlates more with playoff success than offensive performance. Similar findings have been made in baseball. As correlation is not causation, the question is why offensive performance seems less important.
Hypothesis On my blog and this blog, we've shown that offensive performance, as measured by yards per play, correlates more highly with regular season wins than defensive performance. If better offenses get more regular season wins, then the offenses in the playoffs are going to be very good. When everybody is good, then everyone is average. So the only thing that will set a playoff team apart from the rest will be a good defense. The hypothesis, then, is that very good offenses are more common in the postseason than very good defenses, but very good offense is still important to playoff success.
Methods My data is culled from box scores from the 1996-2006 seasons. I refer to yards per play gained/allowed as offensive/defensive efficiency. Sacks are counted as pass plays and teams are penalized for sack yards lost. I then use league averages to translate each stat into a Value Over League Average (VOLA, not the most original name, I know). The league average for yards per rush was 4.006 in 2005 and 4.157 in 2006, so stats need to be adjusted year-to-year to account for moving averages. VOLA is simply the percentage above/below league average. Positive VOLA is always good, so a defense VOLA above zero means the defense allows fewer yards than league average. Findings The average efficiencies (yards per play) for playoff teams are as follows:
- Run Offense 4.1126 yards, 1.5356% VOLA (Value over League Average)
- Run Defense 4.0076 yards, 1.0906% VOLA
- Pass Offense 6.379 yards, 8.4316% VOLA
- Pass Defense 5.6086 yards, 4.6672% VOLA
So by VOLA standards, defenses of playoff teams aren't as good as the offenses. As it turns out, however, very good defenses simply aren't as common as very good offenses. In 1996-2006, 73 teams had 10%+ Pass Off. VOLA, 59 of which made the playoffs and 12 of which made the Super Bowl (6 winners, 6 losers). In the same time frame, only 47 teams had 10%+ Pass Def. VOLA, 34 of which made the playoffs and 8 of which made the Super Bowl (4 winners, 4 losers). So 80.822% of teams with 10%+ VOLA in pass offense make the playoffs, and 16.438% of which make the Super Bowl. On the other hand, 72.34% of teams with 10%+ VOLA in pass defense make the playoffs, while 17.021% of those teams make the Super Bowl. (As a sanity check, I took a quick look at FO's 2006 DVOA standings. Eleven teams had 10%+ DVOA for pass offense, while only six teams had 10%+ DVOA for pass defense.) Similarly, though to a smaller magnitude, 29 out of 58 (50%) teams with 10%+ Run Off. VOLA made the playoffs with 5 teams reaching the Super Bowl (8.6207%, 3 winners, 2 losers) . There were 55 teams with 10%+ Run Def. VOLA, 26 of which made the playoffs (47.27%) and 7 of which made the Super Bowl (12.727%, 4 winners, 3 losers). By these numbers, teams with very good offenses are more likely to make the playoffs, but teams with very good defenses are more likely to win a championship.
Let's take a look at the correlation coeffiecients with playoff seedings, specifically 7-seed # for playoff teams only. By the hypothesis, we'd expect the defensive eff. correlations to be higher because the offenses are all very good across the seeds. The correlation coefficients with 7-(Seed #) listed as Offense, Defense are as follows:
- Run VOLA .10812, .096591
- Pass VOLA .11949, .3477
- Sack Rate VOLA .1075, .23407
- Third Down Conversion Rate VOLA .11549, .24107
- Interception Rate VOLA .16219, .2013
- Fumble Rate VOLA .0189, .16761
In every category except run efficiency, defensive performance has the higher correlation with playoff seeding. So defense is what sets apart the contenders from the one-and-dones. Creating turnovers is important, but a quarterback who is poor at decision-making plays an important part as well. Stopping drives on 3rd downs and forcing punts is also important. Interestingly, punt return averages have next to nothing correlation with the seeding, but kick return averages have a .18278 correlation. And although generally, none of these correlations are particularly strong, they are strong relative to what you'd see in working with football data. I suspect ranking divisional winners 1, 2, 3 and now 4 cuts down on these coefficients, however. Of course, seedings do not equal success, but seedings equal home field advantage, which does seem to play a significant part in playoff success. From 1996-2001, 71.212% of playoff games were won by the home team. After the realignment in 2002, however, about 64% of games were won by the home team. There's usually a weaker fourth division, whose winner gets a home game despite a worse record than their wild card opponent. I'm looking at you, NFC West. So can you win a Super Bowl without a good offense? The Ravens and Bucs did. Can you win a Super Bowl without a good defense? The Colts did. But overall, it's best to have a balance.
Here are the average VOLAs for the playoff teams listed in the order of Super Bowl Winners, Super Bowl Losers, Non-Super Bowl Winners:
- Run Offense VOLA: 2.2576%, 2.8555%, 1.4699%
- Run Defense VOLA: 4.8716%, 4.8332%, 0.7469%
- Pass Offense VOLA: 10.837%, 11.239%, 8.1933%
- Pass Defense VOLA: 9.7542%, 4.5339%, 4.2048%
Pass offense still looks to be the most important factor to winning the Super Bowl, but pass defense is not much less important. Considering that very good pass defenses are less common, perhaps pass defense is more important. Overall, though, balance seems to be the key. Looking at the Colts 2006 run in this light, their succes is less surprising. To begin with, their Pass Off. VOLA was 26.051%, and their Pass Def. VOLA was actually positive at 3.29997%. They allowed 5.79 yards per pass play as opposed to the league average of 5.9743 yards. FO's DVOA puts their pass defense as slightly below average (ranked 18th). Clearly, the were an imbalanced team, but they faced a series of unbalanced teams. The Chiefs had a below average defense and a worn out running back to weigh down their normally good offense. The Ravens had a great defense (20.712% Rush VOLA, 11.405% Pass VOLA) but a so-so offense (6.26% Pass VOLA, -17.275% Rush VOLA). The Bears suffered the same problem (4.7968% Rush Def. VOLA, 15.534% Pass Def. VOLA, -8.698% Rush Off. VOLA, 2.2239% Pass Off. VOLA). The only team the Colts faced in the 2006 playoffs with some amount of balance was the New England Patriots (#7 Off. DVOA, #8 Def. DVOA), and if not for that bizarre drop by a wide-open Reche Caldwell, the Colts would likely have lost.
Conclusion The conclusion is that defense isn't necessarily more important to postseason success. It's harder to reach the Super Bowl without a very good defense, but it's harder to reach the playoffs without a very good offense. Good offenses are simply more common, however, so good defenses are what make playoff teams special. Mind you, 11 seasons is a fairly small sample size, and I do mean to expand the sample at the cost of stats like sack rate and third down conversion rates. Also, VOLA is not the most accurate measurement of quality, but it's fairly straightforward. It's certainly not the worst, though, and very good teams will still show up as such. I am also looking at re-doing the experiment using the DVOA stats that now go back to 1997. You might also interpret the stats I used differently, in which case, I'd like to hear your input. The case is still not closed, but the picture is becoming clearer. (This is an edited version of article I originally posted here.)
How about teams with ok defenses and good offense...we'll call that pretty much every colts team...
Defense wins championships...you build a good D and you'll start winning...