Identification And Normalization In Multinomial Logit

Why Normalization Is Necessary

A multinomial logit model assigns probabilities to a set of mutually exclusive alternatives by comparing their utilities. Those probabilities depend on differences in utility, not on the absolute level used to measure utility. As a result, several different parameter values can describe exactly the same choice probabilities. Without a normalization, the model’s parameters are not uniquely determined by the probabilities it implies.

Two normalizations are central. A base-category normalization chooses a reference alternative and expresses other alternatives’ utility components relative to it. A scale normalization fixes the scale of the random utility errors, which otherwise cannot be separated from the scale of the utility coefficients. These conventions make the parameterization interpretable and estimable; they do not add behavioral content to the model.

The Multinomial Logit Setup

For decision maker \(i\), let \(j=1,\ldots,J\) index the available alternatives. Random utility is written as \(U_{ij}=V_{ij}+\varepsilon_{ij}\), where \(V_{ij}\) is systematic utility and \(\varepsilon_{ij}\) is an unobserved component. The decision maker chooses the alternative with the greatest utility. The multinomial logit probability formula applies when the errors are independent and identically distributed type-I extreme-value random variables, with their scale normalized to one.

Definition: Under the unit-scale multinomial logit convention, the probability that decision maker \(i\) chooses alternative \(j\), conditional on the available alternatives and observed systematic utilities, is

P_{ij} = \frac{\exp(V_{ij})}{\sum_{k=1}^{J}\exp(V_{ik})}.

Here \(P_{ij}\) is the choice probability and \(V_{ij}\) is the systematic utility of alternative \(j\). Each probability is positive, and the probabilities over the \(J\) alternatives sum to one.

The formula makes clear that a probability depends on the systematic utility of an alternative compared with the utilities of the other alternatives. It does not depend on a utility origin or unit that is common to all alternatives. The following normalizations make these two facts explicit.

Common Utility Shifts Leave Probabilities Unchanged

Suppose the same value \(c_i\) is added to every alternative’s systematic utility for decision maker \(i\). The value may differ across decision makers, but it must be common across that person’s alternatives. The transformed probability is

\frac{\exp(V_{ij}+c_i)}{\sum_{k=1}^{J}\exp(V_{ik}+c_i)} = \frac{\exp(c_i)\exp(V_{ij})} {\exp(c_i)\sum_{k=1}^{J}\exp(V_{ik})} = \frac{\exp(V_{ij})}{\sum_{k=1}^{J}\exp(V_{ik})} = P_{ij}.

The common factor \(\exp(c_i)\) cancels. Thus the utility levels themselves are not identified by choice probabilities: adding a common utility amount changes none of the predicted probabilities. The same conclusion follows from utility comparisons, since \((V_{ij}+c_i)-(V_{ik}+c_i)=V_{ij}-V_{ik}\).

Worked Example: Adding the Same Utility to Every Alternative

Consider three hypothetical alternatives with systematic utilities \(V_{i1}=1.2\), \(V_{i2}=0.4\), and \(V_{i3}=-0.3\). Compute their probabilities, then add 2 to every utility and check the result.

Original probabilities. The exponentiated utilities are approximately \(3.3201\), \(1.4918\), and \(0.7408\), with denominator \(3.3201+1.4918+0.7408=5.5527\). Therefore,

(P_{i1},P_{i2},P_{i3}) = \left( \frac{3.3201}{5.5527}, \frac{1.4918}{5.5527}, \frac{0.7408}{5.5527} \right) \approx (0.5979,\,0.2687,\,0.1334).

After the shift. The utilities become \(3.2\), \(2.4\), and \(1.7\). Their exponentials are approximately \(24.5325\), \(11.0232\), and \(5.4739\), with sum \(41.0296\). The probabilities are approximately \(0.5979\), \(0.2687\), and \(0.1334\), just as before. The arithmetic verifies the cancellation: each exponentiated utility was multiplied by \(\exp(2)\).

Base-Category Normalization

Taking the ratio of the probabilities of two alternatives cancels their shared denominator. In particular, relative to an alternative \(b\), the log probability ratio is

\log\!\left(\frac{P_{ij}}{P_{ib}}\right)=V_{ij}-V_{ib}.

This equation states exactly what the choice probabilities reveal about systematic utility: differences relative to a reference alternative. For example, if the systematic utilities contain alternative-specific constants \(a_j\), only the contrasts \(a_j-a_b\) matter. Adding the same constant to all \(a_j\) leaves every contrast unchanged.

Definition: A base-category normalization selects a reference alternative \(b\) and sets its alternative-specific constant to zero. When coefficients on a common set of covariates are specified separately for each alternative, the corresponding coefficients for the base alternative are also set to zero. The remaining alternative-specific parameters describe utility differences relative to \(b\).

For example, suppose \(V_{ij}=a_j+x_i\gamma_j\), where \(x_i\) is a covariate common to all alternatives and \(\gamma_j\) is its alternative-specific coefficient. Setting \(a_b=0\) and \(\gamma_b=0\) gives

V_{ij}-V_{ib} = (a_j-a_b)+x_i(\gamma_j-\gamma_b).

The normalized parameters are therefore contrasts, not absolute utility levels. A zero base-category constant does not assert that the reference option has no utility. It sets the origin for comparing utilities. Another alternative could be selected as the base; the numerical parameter values would change, but the implied probabilities would not, provided the model is otherwise reparameterized consistently.

Worked Example: Rewriting Utilities Relative to a Base Alternative

Suppose three hypothetical travel alternatives, \(A\), \(B\), and \(C\), have systematic utilities

V_{iA}=0.5+0.8x_i,\qquad V_{iB}=1.1+0.3x_i,\qquad V_{iC}=-0.2+1.0x_i,

where \(x_i\) is a traveler characteristic common to the alternatives. Choose \(A\) as the base and write the utility contrasts. Then calculate probabilities at \(x_i=2\).

Subtract \(V_{iA}\) from the other utilities. The contrast for \(B\) is \((1.1-0.5)+(0.3-0.8)x_i=0.6-0.5x_i\). The contrast for \(C\) is \((-0.2-0.5)+(1.0-0.8)x_i=-0.7+0.2x_i\). With the base utility set to zero, the normalized utilities are \(V_{iA}^{*}=0\), \(V_{iB}^{*}=0.6-0.5x_i\), and \(V_{iC}^{*}=-0.7+0.2x_i\).

At \(x_i=2\), the original utilities are \(2.1\), \(1.7\), and \(1.8\). Subtracting the base utility \(2.1\) gives the normalized utilities \(0\), \(-0.4\), and \(-0.3\). Using those values, the denominator is \(\exp(0)+\exp(-0.4)+\exp(-0.3)=1+0.6703+0.7408=2.4111\). Thus,

(P_{iA},P_{iB},P_{iC}) = \left(\frac{1}{2.4111},\frac{0.6703}{2.4111},\frac{0.7408}{2.4111}\right) \approx (0.4147,\,0.2780,\,0.3072).

The original utilities produce the same probabilities: subtracting \(2.1\) from all three is a common utility shift, which leaves probabilities unchanged. The normalized coefficients therefore reproduce the original model’s predictions while describing \(B\) and \(C\) relative to \(A\).

This example uses a covariate that is common across alternatives but has alternative-specific coefficients. If a covariate enters every utility with exactly the same coefficient, its contribution cancels from every utility difference and cannot be identified from the choices. Variables that vary by alternative can contribute to differences, but their coefficients still require enough variation in the data to distinguish them. A normalization removes a known redundancy; it does not guarantee that every remaining coefficient is identified.

Scale Normalization

The second indeterminacy concerns the unit in which utility is measured. To make it explicit, let the errors be independent type-I extreme-value variables with common scale \(\tau>0\). Their choice probabilities have the form

P_{ij} = \frac{\exp(V_{ij}/\tau)} {\sum_{k=1}^{J}\exp(V_{ik}/\tau)}.

If all systematic utilities and the error scale are multiplied by the same positive constant \(q\), then \((qV_{ij})/(q\tau)=V_{ij}/\tau\). The probabilities are unchanged. Consequently, the data cannot separately determine the utility scale and the error scale. The standard multinomial logit normalization fixes \(\tau=1\), so the estimated coefficients are measured relative to the unobserved error scale.

In the standard extreme-value convention with scale \(\tau\), the error variance is \(\pi^2\tau^2/6\). Fixing \(\tau=1\) therefore also fixes that variance at \(\pi^2/6\). This is a scale convention, not an empirical finding that the unobserved component has a particular variance in some absolute utility unit.

Worked Example: Rescaling Utilities and the Error Scale

Suppose a hypothetical three-alternative model uses utilities \((2,0,-2)\) and error scale \(\tau=2\). Compare its probabilities with those from utilities multiplied by 3 and scale multiplied by 3.

With the original values, the scaled utilities in the probability formula are \((2/2,0/2,-2/2)=(1,0,-1)\). The denominator is \(\exp(1)+\exp(0)+\exp(-1)=2.7183+1+0.3679=4.0862\). Hence the probabilities are approximately \(0.6652\), \(0.2447\), and \(0.0900\).

After multiplying both utilities and the scale by 3, the utilities are \((6,0,-6)\) and the scale is \(6\). The scaled utilities remain \((6/6,0/6,-6/6)=(1,0,-1)\), so the denominator and all three probabilities are exactly the same. Only the ratios of systematic utility to error scale are identified by these probabilities.

In the unit-scale parameterization, the effective utilities in this example are \((1,0,-1)\). Reporting \((2,0,-2)\) as if its absolute scale were identified would overstate what the choice probabilities determine. A different convention could fix the scale another way, but the convention must be stated and applied consistently.

What Normalization Does—and Does Not—Establish

Identification means that distinct parameter values cannot produce the same observable model implications, after the parameterization and its restrictions have been specified. Base-category and scale normalizations remove two particular sources of nonuniqueness in multinomial logit. They do not prove identification for every design. For example, a covariate that never varies cannot support estimation of its slope, and exact collinearity among regressors can leave parameter combinations indistinguishable.

The distinction is useful in interpreting model output. A base-category coefficient is a relative utility effect for that alternative versus the reference, on the normalized error scale. It is not an absolute utility level. Changing the base changes the reported contrasts but not the underlying choice probabilities. Likewise, changing the error-scale convention changes the numerical scale of the coefficients, not the information about choice probabilities.

Normalization is also separate from estimation. The model must first be parameterized with appropriate normalizations; an estimation procedure then uses observed choices and covariates to obtain parameter estimates. Normalizing does not by itself calculate estimates, establish that the data are informative, or guarantee that a chosen model is substantively appropriate.

Common Pitfalls

Treating a base constant of zero as a claim about preference. The incorrect reasoning is that the base alternative has zero utility because its constant is fixed at zero. The correction is that zero defines the utility origin; the base alternative’s probability depends on its utility relative to the others.
Trying to estimate every alternative’s utility level. This retains a common shift that has no effect on probabilities. Express alternative-specific terms as contrasts against a base, or impose an equivalent location normalization.
Estimating coefficients and the common error scale freely. The same probabilities result when both are multiplied by the same positive number. Fix the scale, conventionally to one, and interpret coefficients relative to that scale.
Assuming the normalization solves every identification problem. It removes location and scale indeterminacies, but insufficient covariate variation or other redundancy can still prevent unique identification.

Key takeaway: Multinomial logit probabilities depend on utility contrasts measured relative to the error scale. Fix a reference alternative to express location contrasts and fix the error scale to define the units of the coefficients; interpret the resulting parameters as normalized relative utilities.

Check Your Understanding

Answer each question using the distinction between utility contrasts and absolute utility levels.

Why does adding the same constant to every alternative’s systematic utility leave multinomial logit probabilities unchanged?
For a base alternative \(b\), write the log ratio \( \log(P_{ij}/P_{ib}) \) in terms of systematic utilities. What does it show is identified by the probabilities?
If a base-category constant is normalized to zero, does this mean that the base alternative has no utility? Explain.
Why can a common coefficient on a covariate that enters every alternative’s utility identically not be identified from choice probabilities?
If all systematic utilities and the error scale are multiplied by the same positive number, what happens to the probabilities? What is the conventional scale normalization?