Bumped. Good stuff from bluebland. - BN Eds.
The short answer: almost always. Nestor asked me to do a post on the risk-reward statistics regarding 4th down attempts, after I noted my belief that CRN should've gone for it on 4th and 2 from Houston's 30 on our first drive. That wasn't close to the worst of CRN's risk-reward crimes as a coach here, as you will see. But it was another opportunity lost to win a game by being aggressive, establishing identity, and taking leadership. Oh, and playing the odds.
Much research has been done about 4th down plays in the last few years. A couple seasons ago, Bill Billichek shocked everyone by going for it on 4th and 2 from his own 28 (with the lead) with a couple minutes left in the game. The research supported his decision. Here is a great study from Advanced NFL Stats that compresses the entire question into a simple chart.
You should go for it in any situation at or below the blue line. This has been determined by a comparative "expected point" statistic for each decision, factoring in your chances at scoring and the value of field position. The blue line (and everything below) represents the point at which going for it has a superior "expected points" value to kicking. You can see the entire study here.
There are a few underlying premises to this study:
1. It's based on the game being relatively close, and time NOT being a factor. When a large lead is at stake a different statistic called "win probability" comes into play. That only means it's more urgent and obvious to go for it when you're down big late, and less urgent to do so when you're up big with little time left. In other words: this is what you should do before time has any role at all in forcing your hand. Like, say, during the 1st possession.
2. It assumes that a conversion is PRECISELY at the first-down marker. It goes without saying that most first-down conversions go further than that, and thus more likely the value of going for it is even higher.
3. It's an NFL study. And in the NFL, defense and offense is on much more even footing than in college. In college defenses are at a severe disadvantage to an offense, which makes aggression even more valuable. Especially in a game like ours against Houston, when a vast talent disparity even further increased the value of any offensive play.
Now...these are cold hard facts. And they only understate the wisdom of going for it in college. They don't take into account the confidence, leadership, and identity that can be so crucially boosted by an aggressive mentality -- which we now see in almost every top coach in this very stubborn business -- or so depressingly stunted by a passive and fearful one. Nor do they take into account a crappy kicking game.
Punting is a leftover convention from a time when the game consisted of no more than a few scores total and field position meant everything. Because it's a convention, defying it is seen as a bigger "risk" even when the stats show that the odds heavily favor defiance. As one theorist suggests, failing on 4th down is seen as a gambling "loss," while failing because you punted on 4th down is seen as merely football being football. So the more fearful coaches play it "safe."
So far, the closest CRN has come to announcing any identity for his program is "Punting is Winning." He has a few short weeks to grow 40 years.
(P.S. For any masochist who wants to review UCLA football for how much it has killed itself with these decisions in past games, one can go by the third chart on the link above and tally up how many points were sacrificed, and thus games lost.)