Deriving Multinomial Logit Choice Probabilities

From Random Utilities to Choice Probabilities

A decision-maker chooses the alternative with the highest utility, but an analyst typically observes only some of the factors that determine that utility. The unobserved part can be represented by a random shock. A useful and widely applied specification assumes that these shocks are independent and type-I extreme-value distributed. This assumption has a striking consequence: the probability of choosing an alternative is a simple ratio of exponentiated utility indices.

The derivation matters because it shows precisely what the multinomial logit formula assumes. It also clarifies how observed characteristics enter the model: they first determine each alternative’s systematic utility, and those utilities then determine probabilities across the alternatives in the choice set.

Definition: For decision-maker $i$ and available alternative $j$, write utility as $U_{ij}=V_{ij}+\varepsilon_{ij}$. Here $V_{ij}$ is the systematic utility, a known function of observed characteristics conditional on model parameters, and $\varepsilon_{ij}$ is an unobserved utility shock. The decision-maker chooses the alternative with the greatest $U_{ij}$.

Suppose the choice set is finite and contains $J$ alternatives. For the standard multinomial logit derivation, assume that, conditional on observed characteristics, the shocks $\varepsilon_{i1},\ldots,\varepsilon_{iJ}$ are independent and each has the standard type-I extreme-value cumulative distribution function

F(e)=\exp\{-\exp(-e)\}, \qquad -\infty<e<\infty.

Its density is $f(e)=\exp(-e)\exp\{-\exp(-e)\}$. The shocks are continuous, so the probability of an exact tie between two alternatives is zero. The choice rule therefore assigns the decision-maker to the unique alternative with the greatest utility, with probability one.

Deriving the Multinomial Logit Formula

Fix a decision-maker and suppress the index $i$. For alternative $j$ to be chosen, its utility must exceed the utility of every other alternative $k$. Conditional on the shock for $j$ taking value $e$, that requirement is

V_j+e > V_k+\varepsilon_k \quad\Longleftrightarrow\quad \varepsilon_k < e+V_j-V_k \qquad (k\ne j).

Independence lets us multiply the conditional probabilities for the other alternatives. Integrating over all possible values of $j$'s shock gives the choice probability. The following derivation evaluates that integral.

Condition on the chosen alternative’s shock.
At a particular value $e$, each other alternative’s shock must fall below $e+V_j-V_k$. Its probability is $F(e+V_j-V_k)$.

Use independence and integrate.
The joint probability at shock value $e$, integrated over the density of $\varepsilon_j$, is $\int_{-\infty}^{\infty} f(e)\prod_{k\ne j}F(e+V_j-V_k)\,de$.

Simplify the integrand.
The type-I extreme-value density and distribution turn the product into a single exponential involving the sum of exponentiated utility differences.

Evaluate the integral.
Substitution $t=\exp(-e)$ reduces the expression to an exponential integral, yielding the ratio of exponentiated systematic utilities.

In detail, substituting the density and distribution functions gives

\begin{aligned} P(j\text{ chosen}) &=\int_{-\infty}^{\infty} f(e)\prod_{k\ne j}F(e+V_j-V_k)\,de\\ &=\int_{-\infty}^{\infty}\exp(-e) \exp\left\{-\exp(-e)\left[1+\sum_{k\ne j}\exp(V_k-V_j)\right]\right\}\,de. \end{aligned}

Define $A=\sum_{k=1}^{J}\exp(V_k-V_j)$. The bracketed quantity above is exactly $A$, because the $k=j$ term in that sum is $\exp(0)=1$. With $t=\exp(-e)$, we have $dt=-\exp(-e)\,de$. As $e$ increases from $-\infty$ to $+\infty$, $t$ decreases from $+\infty$ to $0$. Thus

\begin{aligned} P(j\text{ chosen}) &=\int_{-\infty}^{\infty}\exp(-e)\exp\{-A\exp(-e)\}\,de\\ &=\int_0^\infty \exp(-At)\,dt\\ &=\left[-\frac{1}{A}\exp(-At)\right]_{0}^{\infty} =\frac{1}{A} =\frac{\exp(V_j)}{\sum_{k=1}^{J}\exp(V_k)}. \end{aligned}

Theorem: If a decision-maker chooses the alternative with the highest utility $U_j=V_j+\varepsilon_j$, the choice set is finite, and the shocks are independent standard type-I extreme-value variables, then $$ P(j\text{ chosen}\mid V_1,\ldots,V_J) =\frac{\exp(V_j)}{\sum_{k=1}^{J}\exp(V_k)}. $$ The probabilities are nonnegative and sum to one over the available alternatives.

The last property follows directly by summing the formula over $j$: the sum of the numerators is the denominator. Each $V_j$ is measured on the utility scale, and exponentiation converts its level into a positive weight. A larger $V_j$, relative to the other systematic utilities, gives alternative $j$ a larger probability. The formula is sometimes called the softmax form of the multinomial logit probabilities.

Worked Probabilities from Utility Indices

Worked Example: Three Alternatives with Given Utilities

Suppose a hypothetical decision-maker faces alternatives A, B, and C, with systematic utilities $V_A=0.8$, $V_B=0.2$, and $V_C=-0.4$. Assume independent standard type-I extreme-value shocks. Find the probability of choosing each alternative.

The assumptions needed for the derived formula are explicit: all three alternatives are available, the decision-maker chooses the highest total utility, and the three shocks are independent with the stated distribution. Their exponentiated utility weights are

\exp(0.8)=2.2255,\qquad \exp(0.2)=1.2214,\qquad \exp(-0.4)=0.6703.

Using the unrounded weights, the denominator is $\exp(0.8)+\exp(0.2)+\exp(-0.4)\approx4.1173$. Therefore

\begin{aligned} P(A)&=\frac{\exp(0.8)}{\exp(0.8)+\exp(0.2)+\exp(-0.4)} \approx\frac{2.2255}{4.1173}=0.5405,\\ P(B)&\approx\frac{1.2214}{4.1173}=0.2967,\\ P(C)&\approx\frac{0.6703}{4.1173}=0.1628. \end{aligned}

The probabilities sum to 1 up to rounding. Under this model, A has the greatest choice probability because its systematic utility is highest. It is not certain to be chosen: the random shocks can alter the utility ranking.

How Covariates Enter Systematic Utility

The probability formula does not prescribe a single form for $V_{ij}$. Researchers specify that part of the model using observed attributes of the decision-maker, attributes of each alternative, or both. A common general representation is

V_{ij}=\alpha_j+x_i^\top\gamma_j+w_{ij}^\top\beta.

Here $\alpha_j$ is an alternative-specific constant, $x_i$ is a vector of characteristics of decision-maker $i$, $w_{ij}$ is a vector of characteristics of alternative $j$ as faced by decision-maker $i$, and the coefficient vectors describe how these observed quantities contribute to systematic utility. The coefficients on $x_i$ may vary across alternatives, as indicated by $\gamma_j$; the vector $w_{ij}$ has a common coefficient vector $\beta$ in this particular specification. Other systematic-utility specifications are possible, but the probability formula uses the resulting $V_{ij}$ in the same way.

Worked Example: Building Utilities from Attributes

Consider a hypothetical commute choice among A, B, and C. Let $x_i=2$ be a decision-maker characteristic, and let the alternatives’ costs be $c_{iA}=1$, $c_{iB}=2$, and $c_{iC}=3$, in stated cost units. Suppose the specified systematic utilities are

\begin{aligned} V_{iA}&=0.2+0.3x_i-0.1c_{iA},\\ V_{iB}&=-0.1+0.1x_i-0.2c_{iB},\\ V_{iC}&=0.4-0.1c_{iC}. \end{aligned}

The constants and coefficients are hypothetical model inputs. Substitution gives

V_{iA}=0.2+0.3(2)-0.1(1)=0.7,\quad V_{iB}=-0.1+0.1(2)-0.2(2)=-0.3,\quad V_{iC}=0.4-0.1(3)=0.1.

Assume all three commute alternatives are available and their utility shocks are independent standard type-I extreme-value variables. The weights are $\exp(0.7)\approx2.0138$, $\exp(-0.3)\approx0.7408$, and $\exp(0.1)\approx1.1052$. Their exact sum is approximately $3.8597$. Hence

P(A)\approx0.5217,\qquad P(B)\approx0.1919,\qquad P(C)\approx0.2863.

These are conditional choice probabilities for the stated attributes and utility specification. For example, the model assigns A a probability of about 0.5217 for a decision-maker with $x_i=2$ and the listed costs. This is a model-based probability, not a claim that the observed characteristics cause a particular choice.

Available Alternatives and the Denominator

The denominator contains only alternatives that are genuinely available to the decision-maker. If an alternative cannot be chosen in a particular situation, it should not be included as a possible outcome in that decision’s choice set. Probabilities then sum to one across the alternatives that remain available. This is a direct consequence of applying the formula to the actual set over which the maximum-utility choice is made.

Worked Example: Recomputing Probabilities for a Smaller Choice Set

Suppose a hypothetical choice set contains three alternatives with systematic utilities $V_A=0$, $V_B=\log(2)$, and $V_C=\log(4)$. The shocks are independent standard type-I extreme-value variables, and all three alternatives are initially available. Their exponentiated weights are $1$, $2$, and $4$, so the total is $7$:

P(A)=\frac{1}{7},\qquad P(B)=\frac{2}{7},\qquad P(C)=\frac{4}{7}.

Now suppose C is unavailable for a particular decision-maker. Under the same utility and shock assumptions, the relevant choice set contains only A and B. The denominator is then $1+2=3$, giving

P(A\mid A,B\text{ available})=\frac{1}{3},\qquad P(B\mid A,B\text{ available})=\frac{2}{3}.

The probabilities are recalculated over the available alternatives and again sum to one. Keeping C in the denominator would assign probability to an impossible choice and would not describe the stated decision situation.

What the Shock Assumption Does—and Does Not—Say

The extreme-value assumption is a distributional model for unobserved utility differences. It is not merely a convenient algebraic trick: independence across alternatives is used when multiplying the probabilities that all competitors fall below a given utility threshold. The particular extreme-value cumulative distribution function then makes the integral simplify to the exponential ratio.

If shocks are correlated across alternatives, the product of their conditional distribution functions generally does not apply, and the derivation above does not establish the multinomial logit formula. Likewise, using a different shock distribution generally produces a different choice-probability formula. One should therefore state the shock assumptions before applying the softmax rule, rather than treating it as a universal formula for all random-utility models.

The systematic utility can include alternative-specific constants, decision-maker characteristics, and attributes that differ across alternatives. The probabilities depend on those observed inputs through the resulting utility indices. A covariate does not enter a probability automatically: its role is determined by the specified utility function.

Common Pitfall

Common mistake: Writing $F(e)=\exp\{-\exp(e)\}$ for the standard type-I extreme-value distribution and then using the multinomial logit formula. That expression has the wrong sign for the standard maximum-type distribution used here. The correct CDF is $F(e)=\exp\{-\exp(-e)\}$, with density $f(e)=\exp(-e)\exp\{-\exp(-e)\}$. Using the correct CDF and the independence assumption yields the integral evaluated above. A separate mistake is to include unavailable alternatives in the denominator; the sum must run over the alternatives in the decision-maker’s actual choice set.

Key takeaway: With $U_{ij}=V_{ij}+\varepsilon_{ij}$, highest-utility choice, and independent standard type-I extreme-value shocks, the probability of choosing $j$ is $\exp(V_{ij})/\sum_k\exp(V_{ik})$, where the sum covers the available alternatives. Observed covariates determine the systematic utilities before this probability calculation.

Check Your Understanding

Use the utility and shock assumptions carefully when answering.

State the distribution function and independence condition used to derive multinomial logit probabilities.
For two alternatives with systematic utilities $V_1=0$ and $V_2=\log(3)$, calculate each choice probability.
Explain why the probability denominator changes when one alternative is unavailable.
In a systematic utility specification, distinguish an alternative-specific constant from an alternative attribute such as cost.
Why does the derivation not directly establish the same formula when shocks are correlated across alternatives?