Individuals who engage in costly punishment do not benefit from their behavior, according to a new study published this week in the journal Nature by researchers at Harvard University and the Stockholm School of Economics.

The group, led by Martin A. Nowak of Harvard’s Program for Evolutionary Dynamics, Department of Mathematics, and Department of Organismic and Evolutionary Biology, examined cooperation among subjects playing a modified version of the Prisoner’s Dilemma. This game captures the fundamental tension between the interests of the individual and the group, and is the classic paradigm for cooperation. The study found that the use of punitive behavior correlates strongly with reduced individual payoff, and bestows no benefit on the group as a whole.

“Put simply, winners don’t punish,” says co-author David G. Rand of Harvard’s Program for Evolutionary Dynamics and Department of Systems Biology. “Punishment can lead to a downward spiral of retaliation, with destructive outcomes for everybody involved. The people with the highest total payoffs do not use costly punishment.”

“Costly punishment,” the type of punitive behavior studied by Nowak and his colleagues, refers to situations where a punisher is willing to incur a cost in order to penalize someone else. Other researchers have suggested that costly punishment can compel cooperation in one-time interactions where individuals need not worry about reputation or retaliation — a scenario Nowak and his colleagues found unrealistic, since, as they write, “most of our interactions are repeated and reputation is always at stake.”

“There’s been a lot of previous work on the use of punishment in cooperation games, but the focus has not been on situations where individuals use punishment in the context of ongoing interactions,” says co-author Anna Dreber of the Stockholm School of Economics and the Program for Evolutionary Dynamics at Harvard. “We make the setting more realistic by having subjects play repeated games and introducing costly punishment as one of several options.”

Dreber, Rand, Nowak, and Drew Fudenberg of Harvard’s Department of Economics recruited 104 Boston-area college students to participate in a computer-based Prisoner’s Dilemma game that was extended to include costly punishment alongside the usual options of cooperation and defection. Pairs of students played the game repeatedly so the interaction between costly punishment and reciprocity could be assessed.

The result: There is a strong negative correlation between individual payoff and the use of costly punishment. The five top-ranked players never used costly punishment, while players who earned the lowest payoffs tended to punish most often. Winners used a tit-for-tat-like strategy while losers used costly punishment. Furthermore, costly punishment did not increase the average payoff of the group.

The study shows that punishment is not an effective force for promoting cooperation. The unfortunate tendency of humans to engage in acts of costly punishment must have evolved for other reasons such as establishing dominance hierarchy and defending ownership, but not to promote cooperation. In cooperation games, costly punishment is a detrimental and self-destructive behavior.

“Punishment may be a tool for forcing another person to do what you want,” Dreber says. “It might have been for those kinds of dominance situations that the use of punishment has evolved.”

“Our finding has a very positive message: In an extremely competitive setting, the winners are those who resist the temptation to escalate conflicts, while the losers punish and perish,” concludes Nowak.

This study was supported by the John Templeton Foundation, the National Science Foundation, the National Institutes of Health, the Jan Wallander Foundation, and J. Epstein.