Integrating Type I Extreme-Value Choice Shocks

Why Integrating Choice Shocks Matters

In a dynamic discrete-choice model, an agent compares alternatives using both systematic payoffs and shocks that are not observed by the analyst. Even if the deterministic value of each alternative is known conditional on the state, the agent chooses the alternative with the highest realized value. To write the Bellman equation in terms of values conditional on the state, we need to integrate over those shocks.

When the shocks are independent Type I extreme-value draws, two useful expressions are available in closed form. The expected maximum is a log-sum, up to a constant, and the probability of choosing an alternative is its exponential value divided by the sum of all alternatives’ exponential values. The derivations below show what assumptions produce these formulas and how their scale affects their interpretation.

Setup and Assumptions

Consider a finite choice set \(a=1,\ldots,J\). Conditional on the current state, alternative \(a\) has a deterministic value \(v_a\), and its realized value is \(v_a+\varepsilon_a\). The values \(v_a\) are fixed when taking expectations over the current shocks. They may include current payoffs and expected continuation values, but the integration here is over the choice shocks.

Definition: A standard Type I extreme-value shock has cumulative distribution function \(F(\varepsilon)=\exp\{-\exp(-\varepsilon)\}\), for real \(\varepsilon\). In this tutorial, shocks are independent across the \(J\) alternatives and each has this same distribution. The expected maximum is \(E[\max_a\{v_a+\varepsilon_a\}]\), conditional on the deterministic values \(v_1,\ldots,v_J\).

The distribution is also called the standard Gumbel distribution. Its location is normalized to zero and its scale to one. These normalizations matter: changing the common location shifts the expected maximum, while changing the scale affects both the expected maximum and the sensitivity of choice probabilities to differences in deterministic values. Because the shocks have continuous distributions, ties between realized alternative values have probability zero.

Worked Example: Two Alternatives with a Value Difference

Suppose two hypothetical alternatives have deterministic values \(v_1=0\) and \(v_2=\ln 3\), in utility units, and each receives an independent standard Type I extreme-value shock. The total exponential weight is

\exp(v_1)+\exp(v_2)=1+3=4.

As derived below, the expected maximum is \(\ln(4)+\gamma\), where \(\gamma\approx 0.577216\) is Euler’s constant. Numerically,

E[\max\{v_1+\varepsilon_1,v_2+\varepsilon_2\}] =\ln(4)+0.577216 =1.96351.

The same values imply choice probabilities \(1/4\) and \(3/4\). Thus alternative 2 is more likely to be chosen, but its choice probability is not one: the independent shocks can make alternative 1’s realized value larger.

Deriving the Expected-Maximum Formula

Let \(M=\max_a\{v_a+\varepsilon_a\}\), and write \(L=\ln\left(\sum_{a=1}^J\exp(v_a)\right)\). The event \(M\leq y\) requires every alternative’s realized value to be at most \(y\). Independence lets us multiply the probabilities of these events:

\begin{aligned} P(M\leq y) &=\prod_{a=1}^J P(\varepsilon_a\leq y-v_a)\\ &=\prod_{a=1}^J \exp\{-\exp[-(y-v_a)]\}\\ &=\exp\left\{-\exp(-y)\sum_{a=1}^J\exp(v_a)\right\}\\ &=\exp\{-\exp[-(y-L)]\}. \end{aligned}

The last expression is the cumulative distribution function of a standard Type I extreme-value variable shifted by \(L\). In distribution, \(M=L+G\), where \(G\) is standard Type I extreme value. Therefore \(E[M]=L+E[G]\). The standard Gumbel density is \(f_G(g)=\exp(-g)\exp\{-\exp(-g)\}\). With the substitution \(t=\exp(-g)\), its mean is

E[G]=\int_{-\infty}^{\infty}g\exp(-g)\exp\{-\exp(-g)\}\,dg =-\int_0^\infty \ln(t)\exp(-t)\,dt =\gamma.

Here the integral is the standard integral representation of Euler’s constant. Combining these steps gives the expected-maximum result.

Theorem: If \(v_1,\ldots,v_J\) are fixed and the shocks \(\varepsilon_1,\ldots,\varepsilon_J\) are independent standard Type I extreme-value draws, then

E\left[\max_{a=1,\ldots,J}\{v_a+\varepsilon_a\}\right] =\gamma+\ln\left(\sum_{a=1}^J\exp(v_a)\right).

The expectation is conditional on the deterministic values. The constant \(\gamma\) is common to all states and choice sets with the same shock normalization.

Deriving the Choice Probabilities

The expected maximum does not by itself identify which alternative wins. To derive that probability, transform each shock by setting \(Z_a=\exp(-\varepsilon_a)\). For \(z>0\), the standard shock distribution implies

P(Z_a>z)=P(\varepsilon_a<-\ln z) =\exp(-z).

Thus \(Z_a\) has an exponential distribution with rate one. The alternative with the highest \(v_a+\varepsilon_a\) is equivalently the one with the smallest \(\exp[-(v_a+\varepsilon_a)]=\exp(-v_a)Z_a\). Multiplying an exponential random variable with rate one by \(\exp(-v_a)\) gives an exponential random variable with rate \(\lambda_a=\exp(v_a)\). Independence is preserved by these separate transformations.

For independent exponential random variables, the probability that alternative \(a\)'s transformed value is the smallest can be calculated directly. Its density at \(t>0\) is \(\lambda_a\exp(-\lambda_a t)\), and the probability that every other transformed value exceeds \(t\) is \(\prod_{b\ne a}\exp(-\lambda_b t)\). Integrating over possible minimum values gives

\begin{aligned} P(a\text{ is chosen}) &=\int_0^\infty \lambda_a\exp(-\lambda_a t) \prod_{b\ne a}\exp(-\lambda_b t)\,dt\\ &=\int_0^\infty \lambda_a \exp\left\{-t\sum_{b=1}^J\lambda_b\right\}\,dt\\ &=\frac{\lambda_a}{\sum_{b=1}^J\lambda_b} =\frac{\exp(v_a)}{\sum_{b=1}^J\exp(v_b)}. \end{aligned}

Theorem: Under the same independent, standard Type I extreme-value shock assumptions, the probability that alternative \(a\) has the highest realized value is

P(a\text{ is chosen}\mid v_1,\ldots,v_J) =\frac{\exp(v_a)}{\sum_{b=1}^J\exp(v_b)}.

These probabilities are conditional on the state-dependent deterministic values. They sum to one because the alternatives are mutually exclusive and one alternative is chosen.

Worked Example: Three Alternatives and Relative Choice Probabilities

Consider a hypothetical choice among three alternatives with deterministic values \(v_1=0\), \(v_2=\ln 2\), and \(v_3=\ln 3\). All three have independent standard Type I extreme-value shocks. Their exponential weights are \(1\), \(2\), and \(3\), respectively, so the denominator is \(1+2+3=6\). The probabilities are

P(1\text{ is chosen})=\frac{1}{6},\qquad P(2\text{ is chosen})=\frac{2}{6}=\frac{1}{3},\qquad P(3\text{ is chosen})=\frac{3}{6}=\frac{1}{2}.

They sum to \(1/6+1/3+1/2=1\). The expected maximum is

\gamma+\ln(1+2+3) =0.577216+\ln(6) =2.36898

utility units. In this example, alternative 3 has three times the choice probability of alternative 1 because its exponential weight is three times as large. The probability ratio follows from the formula; it is not a claim that the utility difference is three times as large.

General Shock Scale

A common extension allows the shocks to have scale \(\sigma>0\) rather than one. Specifically, let \(\varepsilon_a=\sigma\eta_a\), where the \(\eta_a\) are independent standard Type I extreme-value draws. Then

Result: With independent Type I extreme-value shocks of scale \(\sigma>0\), and deterministic values measured in the same utility units,

E\left[\max_a\{v_a+\varepsilon_a\}\right] =\sigma\gamma+\sigma\ln\left(\sum_{a=1}^J\exp(v_a/\sigma)\right), \qquad P(a\text{ is chosen}) =\frac{\exp(v_a/\sigma)}{\sum_{b=1}^J\exp(v_b/\sigma)}.

These expressions follow by factoring out \(\sigma\) from the maximum: \(\max_a\{v_a+\sigma\eta_a\}=\sigma\max_a\{v_a/\sigma+\eta_a\}\). Applying the standard-scale results to values \(v_a/\sigma\) gives both formulas. A larger \(\sigma\) makes the choice probabilities less concentrated on alternatives with higher deterministic values. The scale is therefore not just a numerical detail; it determines the units in which deterministic utility differences affect choices.

Worked Example: A Nonunit Shock Scale

Suppose two hypothetical alternatives have values \(v_1=2\) and \(v_2=0\), measured in utility units, and independent Type I extreme-value shocks with scale \(\sigma=2\). The probabilities use values divided by the scale:

P(1\text{ is chosen}) =\frac{\exp(2/2)}{\exp(2/2)+\exp(0/2)} =\frac{e}{e+1} =0.7311, \qquad P(2\text{ is chosen})=\frac{1}{e+1}=0.2689.

The expected maximum is

\begin{aligned} E[\max\{2+\varepsilon_1,0+\varepsilon_2\}] &=2\gamma+2\ln\{\exp(2/2)+\exp(0/2)\}\\ &=2(0.577216)+2\ln(e+1)\\ &=1.15443+2(1.31326)\\ &=3.78095 \end{aligned}

utility units. The scale of two enters both the constant and the log-sum, while the choice probabilities depend on the values relative to that scale.

Use in a Dynamic Choice Model

To connect the integration to dynamic programming, suppose a state \(x\) and action \(a\) imply deterministic choice-specific value \(v_a(x)\). For example, this value may be the current payoff plus the discounted expected value of the next state. If a fresh vector of independent standard Type I extreme-value shocks is realized at the current choice, the value after integrating over those shocks is

V(x)=\gamma+\ln\left(\sum_{a=1}^J\exp(v_a(x))\right).

This is the log-sum expression inside the dynamic Bellman equation. It integrates the current choice shocks conditional on the state and deterministic values; it does not remove the need to calculate expected continuation values or specify the state transitions. The Bellman operator’s contraction properties concern how the value function is determined across states, while the formula here supplies the closed-form integration over the current shocks.

If the shocks instead have scale \(\sigma\), the corresponding expression is \(\sigma\gamma+\sigma\ln\left(\sum_a\exp(v_a(x)/\sigma)\right)\). Any units and normalization used for \(v_a(x)\) must agree with the chosen shock scale.

Common Mistakes and Interpretation

Leaving out the Euler constant. The log-sum alone is not the expected maximum under the standard zero-location shock normalization. The expectation includes \(+\gamma\). A different common shock location would change this additive constant.
Using values rather than scaled values when \(\sigma\ne1\). The probability formula uses \(\exp(v_a/\sigma)\), not \(\exp(v_a)\). Omitting the scale changes the implied sensitivity of choices to deterministic values.
Assuming the log-sum formula holds for arbitrary shocks. The derivation uses both the Type I extreme-value distribution and independence across alternatives. With other distributions or dependence structures, these closed forms need not apply.
Calling a choice probability a deterministic ranking. A higher \(v_a\) raises the probability of choosing \(a\), but does not guarantee that \(a\) is chosen in a particular realization. The shocks remain part of realized utility.
Treating the integrated value as a realized payoff. The log-sum expression is an expectation over current shocks conditional on deterministic values, not the value observed in one individual choice realization.

A useful check is invariance of the choice probabilities to a common additive shift. If the same constant \(c\) is added to every \(v_a\), then each numerator and the denominator are multiplied by \(\exp(c)\), so the probabilities do not change. The expected maximum does change by \(c\), as it should: adding \(c\) to every alternative raises every realized utility by \(c\).

Check Your Understanding

Use the formulas and assumptions developed above to answer these questions.

Why is independence across alternatives needed when deriving the distribution of the maximum?
For values \(v_1=\ln 4\) and \(v_2=0\) with standard shocks, calculate both choice probabilities and the expected maximum.
What changes in the choice-probability formula when the Type I extreme-value shocks have scale \(\sigma=3\)?
Explain why adding the same constant to every deterministic value changes the expected maximum but leaves choice probabilities unchanged.
Which feature of the model would make the displayed log-sum and choice-probability formulas generally inapplicable: a different common value in every state, or dependence between the shocks across alternatives? Explain.