Why Strategies Depend on History
In a one-period game, a player chooses an action without having to plan a response to past conduct. In a repeated game, each period creates a history that may affect future choices. A player can reward cooperation, respond to a deviation, or continue a punishment. The strategy must therefore describe what the player will do after different possible histories, not just what the player does at the start.
This tutorial considers two standard strategies for an indefinitely repeated Prisoner’s Dilemma: grim trigger and tit-for-tat. They can both begin with cooperation, but they respond differently after a deviation. Grim trigger makes punishment permanent once triggered. Tit-for-tat responds to the opponent’s most recent action and can return to cooperation if the opponent does.
Use this illustrative Prisoner’s Dilemma payoff matrix, with the first number in each cell belonging to Player A and the second to Player B.
| Player A / Player B | Cooperate \(C\) | Defect \(D\) |
|---|---|---|
| Cooperate \(C\) | (4, 4) | (1, 7) |
| Defect \(D\) | (7, 1) | (2, 2) |
Defection is the higher-payoff action for a player regardless of the other player’s action: against \(C\), \(7>4\), and against \(D\), \(2>1\). Yet mutual cooperation pays each player 4, more than the 2 each receives from mutual defection. That tension is what makes history-dependent strategies relevant.
Two Trigger Strategies
The event that triggers punishment is any departure from the all-cooperate history. The strategy does not require that the grim-trigger player personally was the one harmed: a deviation by either player is enough. Once triggered, the strategy does not forgive a later return to cooperation.
Tit-for-tat is conditional on the most recent action, not on whether a deviation ever occurred. If an opponent defects and then cooperates, a tit-for-tat player first responds with defection, then returns to cooperation if the opponent’s latest action was cooperation. The strategy does not promise that this response will restore cooperation immediately: the other player’s strategy and the full action history determine what happens next.
Why an Infinite Payoff Stream Has a Simple Form
For a constant payoff \(g\) received in every period, the discounted total is \(g+\beta g+\beta^2g+\cdots\). Calling this total \(V\), removing the first-period payoff leaves \(\beta V\), so \(V=g+\beta V\). Rearranging gives the infinite-stream formula below.
The same reasoning applies to a constant payoff stream beginning later: multiply the formula by the discount weight for its starting period. For example, a stream starting in period 3 has weights \(\beta^2,\beta^3,\ldots\), so its value is \(\beta^2g/(1-\beta)\). The formula requires \(0<\beta<1\); if \(\beta=1\) and \(g>0\), the infinite sum does not converge to a finite value.
Tracing Grim Trigger After a Deviation
Worked Example: One Deviation Starts Permanent Punishment
Both players use grim trigger and choose \(C\) in period 1. Player A chooses \(D\) in period 2 while Player B chooses \(C\). From period 3 onward, both follow the triggered strategy. Let \(\beta=0.8\). Trace the outcomes and calculate each player’s discounted payoff from period 1. State what the calculation does and does not establish.
Method. Identify the first non-\((C,C)\) action profile. That profile is the deviation period; grim punishment begins in the following period.
Period 1: Both choose \(C\), so the payoff is \((4,4)\).
Period 2: A chooses \(D\) and B chooses \(C\), so the payoff is \((7,1)\). This history triggers grim punishment.
Period 3 onward: Both choose \(D\), giving each player 2 in every remaining period. These payoffs start in period 3, so their combined discounted value for each player is
A’s total is \(4+0.8(7)+6.4=4+5.6+6.4=16.0\). B’s total is \(4+0.8(1)+6.4=4+0.8+6.4=11.2\). As a check, if both instead cooperated forever, each would receive \(4/(1-0.8)=20\). These totals describe the two payoff paths; comparing them alone does not prove that a strategy profile is an equilibrium.
Common mistake avoided: Starting punishment in period 2 would be wrong. Period 2 is the period in which the deviation occurs; with the stated strategies, mutual defection starts in period 3.
Tracing Tit-for-Tat Through a Return to Cooperation
Worked Example: A Tit-for-Tat Response Can End
Player A uses tit-for-tat. Player B’s actions are specified as follows: both choose \(C\) in period 1; B chooses \(D\) in period 2; B returns to \(C\) in period 3; and B continues choosing \(C\) afterward. Trace the first five outcomes and calculate the period payoffs.
Method. For each period after the first, determine A’s action from B’s action in the preceding period. Do not use B’s action in the current period to decide A’s simultaneous choice.
Period 1: A starts with \(C\), and B chooses \(C\). The outcome is \((C,C)\), with payoffs \((4,4)\).
Period 2: A copies B’s period-1 \(C\) and chooses \(C\). B chooses \(D\). The outcome is \((C,D)\), with payoffs \((1,7)\).
Period 3: A copies B’s period-2 \(D\) and chooses \(D\). B returns to \(C\). The outcome is \((D,C)\), with payoffs \((7,1)\).
Period 4: A copies B’s period-3 \(C\) and chooses \(C\). B also chooses \(C\), giving \((C,C)\) and payoffs \((4,4)\).
Period 5: B chose \(C\) in period 4, so A again chooses \(C\); B continues to cooperate. The outcome is \((C,C)\), with payoffs \((4,4)\). The check is that A’s period-3 defection follows B’s period-2 defection, while A’s period-4 cooperation follows B’s period-3 cooperation.
Common mistake avoided: Tit-for-tat does not defect forever after one defection. It copies the opponent’s latest completed action, so its response changes when that action changes.
Different Histories, Different Paths
A strategy’s response must be evaluated using the history relevant to that strategy. The following comparison summarizes the distinction.
| History or event | Grim trigger | Tit-for-tat |
|---|---|---|
| No previous deviation; both have cooperated | Continue with \(C\) | Choose \(C\) if the opponent’s last action was \(C\) |
| Opponent defects in the latest period | Choose \(D\) if any deviation has triggered punishment | Choose \(D\) in the next period |
| Opponent later returns to \(C\) | Continue with \(D\) once triggered | Choose \(C\) in the next period |
| Both choose \(D\) in the latest period | Continue with \(D\) | Both choose \(D\) next, if both use tit-for-tat |
The last row illustrates why the same pair of strategy names can produce different paths after different histories. If both players use tit-for-tat and both defect in one period, both will defect in the next period because each copies the other’s latest \(D\). If they continue following tit-for-tat, mutual defection repeats. By contrast, if only one player defects while the other cooperates, their subsequent tit-for-tat actions can differ.
Worked Example: One Deviation Under Two Tit-for-Tat Strategies
Both players use tit-for-tat. They cooperate in period 1. In period 2, A chooses \(D\) while B chooses \(C\). Assume that from then on each player follows tit-for-tat. Find the outcomes and payoffs in periods 2 through 5, and describe the pattern afterward.
Method. At each step, A copies B’s preceding action and B copies A’s preceding action. The action profile in the current period therefore depends on the two separate actions in the previous period.
Period 2: By the specified deviation, the outcome is \((D,C)\), with payoffs \((7,1)\).
Period 3: A copies B’s period-2 \(C\), so A chooses \(C\). B copies A’s period-2 \(D\), so B chooses \(D\). The outcome is \((C,D)\), with payoffs \((1,7)\).
Period 4: A copies B’s period-3 \(D\), and B copies A’s period-3 \(C\). The outcome is \((D,C)\), with payoffs \((7,1)\).
Period 5: A copies B’s period-4 \(C\), and B copies A’s period-4 \(D\). The outcome is \((C,D)\), with payoffs \((1,7)\). The same two-period pattern continues: odd-numbered periods from period 3 onward have \((C,D)\), and even-numbered periods have \((D,C)\).
The check is that the action profile switches each period: after \((D,C)\), both copy the other player’s action and produce \((C,D)\); after \((C,D)\), they produce \((D,C)\). A single deviation does not necessarily lead back to mutual cooperation under tit-for-tat.
Common mistake avoided: It is incorrect to say both players choose the action they themselves took last period. Each copies the opponent’s previous action.
What the Strategy Paths Do—and Do Not—Show
Grim trigger and tit-for-tat are complete plans for responding to observed histories. Grim trigger uses the entire relevant record: once a non-cooperative action has occurred, its response remains \(D\). Tit-for-tat uses only the opponent’s immediately preceding action. As a result, the strategies can produce sharply different paths after the same initial cooperation.
A path is not, by itself, a proof that the strategies are best responses. To establish an equilibrium, one must also check whether a player can gain by changing actions, taking account of the current payoff and the effect on future play. The present value of a future punishment, and whether it is sufficient to deter a deviation, depend on the payoff data and the discounting assumptions. Keep that incentive question separate from the task of correctly tracing a strategy’s prescribed actions.
Check Your Understanding
For each problem, use the payoff matrix and strategy definitions in this tutorial unless the question supplies different information.
- Under grim trigger, both players choose \(C\) in periods 1 and 2. In period 3, B chooses \(D\) while A chooses \(C\). What is the action profile in period 4, assuming both follow grim trigger thereafter? (Answer: \((D,D)\).)
- Player A uses tit-for-tat. B chose \(D\) in the previous period, and A chose \(C\) then. What does A choose now? If B chooses \(C\) now, what does A choose in the following period? (Answer: A chooses \(D\) now and \(C\) in the following period.)
- Both players use tit-for-tat and chose \((C,C)\) in period 1. In period 2, A chooses \(D\) and B chooses \(C\). Give the period-3 and period-4 action profiles and payoffs. (Answer: Period 3 is \((C,D)\), with payoffs \((1,7)\); period 4 is \((D,C)\), with payoffs \((7,1)\).)
- Both players use grim trigger. They choose \((C,C)\) in period 1; A chooses \(D\) and B chooses \(C\) in period 2; both choose \(D\) from period 3 onward. With \(\beta=0.5\), calculate each player’s discounted total from period 1. (Answer: A receives \(4+0.5(7)+0.5^2(2)/(1-0.5)=8.5\); B receives \(4+0.5(1)+0.5^2(2)/(1-0.5)=5.5\).)
- Both players use grim trigger. A non-\((C,C)\) profile occurred three periods ago, but both players chose \(C\) in each of the last two periods. What does each player choose next? How would the answer differ if both used tit-for-tat instead? (Answer: Grim trigger prescribes \((D,D)\), because punishment is permanent; tit-for-tat prescribes \((C,C)\), because each opponent’s most recent action was \(C\).)