Finite-Horizon Choice and Terminal Values

Why the End of the Horizon Matters

A dynamic choice model predicts decisions partly by valuing what follows them. In a finite-horizon model, however, there is a final date after which the model has no further decision periods. The analyst must specify what the agent receives or values beyond that boundary. This terminal value is not merely a computational detail: if it differs across terminal states, it can affect choices before the horizon ends.

Backward induction makes the timing explicit. Start from the value assigned after the last modeled choice, calculate the value of each final-period action, and then move backward one period at a time. The resulting choices are optimal relative to the stated terminal condition. They need not be the choices implied by an infinite-horizon problem unless the terminal values appropriately represent what lies beyond the modeled horizon.

Definition: Consider dates \(t=1,\ldots,T\), state \(x_t\), action set \(A(x_t)\), discount factor \(\beta\), and terminal value \(g(x_{T+1})\). The terminal value is the value assigned to the state reached after the period-\(T\) action. It may represent a salvage value, a payoff at the boundary, or an approximation to value beyond the modeled horizon. It is distinct from the period-\(T\) flow payoff \(u_T(x_T,a_T)\).

Finite-Horizon Bellman Recursion

Assume the state contains the information needed to predict future payoffs and transitions, conditional on the current action. Let \(p(x' \mid x,a)\) be the probability of next state \(x'\) given current state \(x\) and action \(a\). The deterministic flow payoff is \(u_t(x,a)\). For the standard dynamic discrete-choice specification, the decision-maker also observes an action-specific shock \(\epsilon_t(a)\) before choosing, while the analyst does not. Assume these shocks are independent across actions and dates, with Type I extreme-value distribution normalized to scale one. The discount factor satisfies \(0\leq\beta<1\).

The choice-specific value \(q_t(x,a)\) is the payoff from taking action \(a\), including the expected value of what follows. The inclusive value \(V_t(x)\) is the expected maximum value at date \(t\), before observing that date's choice shocks. With these definitions, the finite-horizon recursion is:

V_{T+1}(x)=g(x), \qquad q_t(x,a)=u_t(x,a)+\beta\sum_{x'}p(x'\mid x,a)V_{t+1}(x'), \quad t=T,\ldots,1.

Under the stated shock normalization, integrating over the shocks gives:

V_t(x)=\gamma+\log\!\left(\sum_{a\in A(x)} \exp\{q_t(x,a)\}\right), \qquad P_t(a\mid x)= \frac{\exp\{q_t(x,a)\}} {\sum_{b\in A(x)}\exp\{q_t(x,b)\}},

where \(\gamma\approx0.5772\) is Euler's constant. These expressions apply at each decision date, including \(T\); the terminal value enters the period-\(T\) choice-specific values through the transition term. If the terminal value is zero for every terminal state, then the final action is valued only for its flow payoff, apart from any specified transition-related payoff. That is a modeling assumption, not a consequence of having a finite horizon.

Key distinction: \(u_T(x,a)\) is earned during the final modeled period. The terminal value \(g(x')\) is assigned to the state reached after that action. Counting a payoff in both places would double-count it; setting \(g\) to zero excludes all value beyond the modeled boundary.

How Backward Induction Works

At date \(T\), the continuation component of each action's value is calculated from \(g\). The inclusive value \(V_T\) then summarizes the best final-period choice, averaging over the decision-maker's shocks. At date \(T-1\), each action's expected continuation value is calculated from \(V_T\), and the same operation repeats. Thus \(V_t\) depends on the terminal condition through all later periods, and the impact can reach decisions well before \(T\).

The recursion is justified by the timing of the problem. At date \(t\), conditional on state \(x\) and action \(a\), the agent receives \(u_t(x,a)\), moves to \(x'\) according to \(p(x'\mid x,a)\), and then faces the continuation value \(V_{t+1}(x')\). Taking the conditional expectation over next states gives \(q_t(x,a)\). The agent chooses the action with the largest \(q_t(x,a)+\epsilon_t(a)\); taking the expectation of this maximum over the specified shocks yields \(V_t(x)\). Starting with \(V_{T+1}=g\) therefore determines every earlier value in sequence.

Set the boundary.
Specify \(g(x)\) for every terminal state that can be reached after the final modeled choice.

Calculate final-period choice values.
For each \(x\) and \(a\), combine \(u_T(x,a)\) with the discounted expected terminal value.

Calculate final-period values and probabilities.
Use the inclusive-value and choice-probability formulas to obtain \(V_T(x)\) and \(P_T(a\mid x)\).

Repeat toward the first date.
Use the values at \(t+1\) to calculate choice-specific values, inclusive values, and probabilities at \(t\).

Worked Examples

Worked Example: A Two-Period Model With an Exit Option

Suppose \(\beta=0.9\), the terminal value is zero, and at date 2 the agent can choose \(A\) or \(B\). Their flow payoffs are 1.0 and 0.5, respectively, and either action reaches a terminal state with value zero. At date 1 the options are to continue to this date-2 state or to exit immediately. The continuation payoff at date 1 is zero; exit gives a payoff \(c\) and ends the modeled process. Find the date-2 value and probabilities, then choose \(c\) so the date-1 continuation probability is the logistic probability associated with a value difference of 1.9492.

At date 2 the terminal continuation terms are zero, so \(q_2(A)=1.0\) and \(q_2(B)=0.5\). The inclusive value is:

V_2=\gamma+\log\{\exp(1)+\exp(0.5)\} \approx 0.5772+1.4741=2.0513.

The date-2 probability of choosing \(A\) is \(1/[1+\exp\{-(1.0-0.5)\}]\approx0.6225\); the probability of \(B\) is \(1-0.6225=0.3775\). At date 1, continuing has value \(q_1(C)=0.9V_2\approx0.9(2.0513)=1.8462\). Exit has value \(q_1(E)=c\). To make the difference \(q_1(C)-q_1(E)=1.9492\), set \(c=0.9V_2-1.9492\approx-0.1030\), using the unrounded \(V_2\) for the calculation.

The continuation probability is therefore:

P_1(C)=\frac{1}{1+\exp(-1.9492)} \approx\frac{1}{1+0.14239}=0.8754.

Thus, under the specified model and shocks, the decision-maker continues with probability about 0.8754 at date 1. This result depends on the assumed date-2 payoffs, discount factor, and exit payoff; it is not a claim that continuation is certain.

Worked Example: Changing a Terminal Value Changes an Earlier Choice

Consider a one-period model with two actions, invest and exit. Their immediate payoffs are 0.2 and 0. The discount factor is 0.9. Investing leads to terminal state \(I\), while exiting leads to terminal state \(E\). Calculate the investment probability first when both terminal values are zero, and then when \(g(I)=2\) and \(g(E)=0\).

When both terminal values are zero, the choice-specific values are \(q_1(I)=0.2+0.9(0)=0.2\) and \(q_1(E)=0+0.9(0)=0\). The investment probability is:

P_1(I)=\frac{\exp(0.2)}{\exp(0.2)+\exp(0)} =\frac{1}{1+\exp(-0.2)} \approx0.5498.

Now give the investment terminal state a value of 2. The new choice-specific values are \(q_1(I)=0.2+0.9(2)=2.0\) and \(q_1(E)=0\). Therefore:

P_1(I)=\frac{\exp(2.0)}{\exp(2.0)+\exp(0)} =\frac{1}{1+\exp(-2.0)} \approx0.8808.

The terminal value raises the relative value of investing, so the investment probability rises from about 0.5498 to 0.8808. It is the difference between terminal values, not simply the fact that terminal values are nonzero, that changes the relative choice values here.

Worked Example: State-Dependent Terminal Values at the Final Date

At the final decision date, suppose action \(A\) pays 1.0 and deterministically reaches terminal state \(L\), while action \(B\) pays 0.5 and reaches terminal state \(H\). Let \(\beta=0.9\), \(g(L)=0\), and \(g(H)=2\). Find the probability of choosing \(B\), and compare it with the probability when both terminal values are zero.

With the stated terminal values, the choice-specific values are \(q_T(A)=1.0+0.9(0)=1.0\) and \(q_T(B)=0.5+0.9(2)=2.3\). Hence:

P_T(B)=\frac{\exp(2.3)}{\exp(1.0)+\exp(2.3)} =\frac{1}{1+\exp(-1.3)} \approx\frac{1}{1+0.2725}=0.7858.

The inclusive value is \(V_T=0.5772+2.3+\log\{1+\exp(-1.3)\}\approx0.5772+2.3+0.2410=3.1182\). If both terminal values are instead zero, the choice-specific values become 1.0 and 0.5. The probability of \(B\) is then \(1/[1+\exp(0.5)]\approx0.3775\), and the inclusive value is approximately 2.0513. The higher value assigned to \(H\) makes \(B\) more attractive despite its lower immediate payoff.

What Changes, and What Does Not

A useful comparison follows directly from the probability formula. For two actions \(a\) and \(b\), the odds of choosing \(a\) rather than \(b\), conditional on state \(x\), are:

\frac{P_t(a\mid x)}{P_t(b\mid x)} =\exp\{q_t(x,a)-q_t(x,b)\}.

Thus a terminal condition affects choice probabilities when it changes differences in choice-specific values. If every action leads to the same terminal value \(c\), the terminal contribution to every choice-specific value is the same, \(\beta c\). It raises the inclusive value but cancels from pairwise differences, leaving the probabilities unchanged. If actions lead to different terminal states or distributions over terminal states, their expected terminal values can differ, changing those differences and therefore the probabilities.

More generally, if the terminal value changes from \(g\) to \(g+\Delta g\), the period-\(T\) choice-specific value changes by \(\beta\sum_{x'}p(x'\mid x,a)\Delta g(x')\). This expression makes the mechanism explicit: an action is affected according to the states it is expected to reach. Earlier in the horizon, the change can propagate through the recursion, because altered continuation values enter earlier choice-specific values.

The terminal condition matters in applications with a short observation window, such as a finite sample of replacement or participation decisions. A model that assigns zero value after the last observed date may imply that agents behave as if all opportunities and payoffs end there. That assumption can distort earlier choices if agents actually anticipate valuable post-sample opportunities. A terminal value can represent such anticipated value, but its form must be justified by the economic setting or by the model's boundary approximation.

Common Mistakes

Confusing flow payoff and terminal value. The flow payoff is earned in a modeled period; the terminal value is assigned to the state after the last modeled action. Keep their timing separate.
Assuming the terminal value is automatically zero. Zero is one possible boundary condition, not a general implication of finite-horizon modeling. State and justify it.
Adding the same terminal value to all alternatives and claiming choice probabilities must change. A common additive shift changes inclusive values but cancels from relative choice values and probabilities.
Using a terminal value without discounting it. In the recursion given here, the next-period value is multiplied by \(\beta\). Apply the same timing and discount convention consistently.
Interpreting calculated probabilities as causal effects. A probability change across terminal specifications is a model implication. It is not, by itself, evidence that a real-world terminal payoff caused a behavioral change.

Key takeaway: A finite-horizon dynamic choice model is solved by starting with a stated terminal value and applying the Bellman recursion backward. The terminal condition affects earlier choices whenever it changes relative continuation values; conclusions are conditional on that boundary specification.

Check Your Understanding

Answer each question using the timing and definitions in this tutorial.

In a finite-horizon model, how does a terminal value differ from the final-period flow payoff?
If every action leads to the same terminal value with certainty, what happens to pairwise choice probabilities when that common value increases? What happens to the inclusive value?
With \(\beta=0.8\), an action reaches a terminal state whose value increases by 3.0. By how much does that action's final-period choice-specific value increase, if its transition is deterministic?
Why can a terminal value assigned to states after date \(T\) affect decisions at date 1?
In the third worked example, what feature of the terminal values makes action \(B\) more attractive than it would be under zero terminal values?