Worked Example With A Three-Category Logit

From Fitted Logits to Category Probabilities

A baseline-category logit model describes a response with three nominal categories by comparing each nonbaseline category with a selected reference category. Its coefficients are log-odds coefficients, but a fitted model can also answer practical questions about the probability of each outcome at specified predictor values. This worked example connects those views: we calculate probabilities, interpret category-specific and pairwise odds ratios, and test a predictor’s effects jointly.

Let the response \(Y\) take values \(1,2,3\), with category 1 as the baseline. Let \(x\) be a quantitative predictor and \(z\) a binary indicator. The model has one linear predictor for each nonbaseline category:

Suppose model fitting produces the following coefficient estimates:

Here \(x\) increases by one unit when the quantitative predictor increases by one unit, and \(z=1\) denotes the group coded as present. The intercepts are the fitted baseline-category log-odds when \(x=z=0\). These estimates specify two log-odds relative to category 1; neither is itself a probability.

The denominator is essential: exponentiating each linear predictor gives odds relative to the baseline, not the category probabilities themselves. The normalization makes the three probabilities add to one.

Category-Specific and Pairwise Odds Ratios

A one-unit increase in \(x\), with \(z\) held fixed, changes the category-2-versus-category-1 log odds by \(\beta_{2x}\). Exponentiating that coefficient gives the corresponding multiplicative change in the odds. The same calculation applies to category 3 versus category 1, using its own coefficient.

The same fitted model also gives odds for comparing two nonbaseline categories directly. The pairwise comparison follows by subtracting their baseline log odds: the baseline terms cancel. This is useful when the scientific question concerns category 2 versus category 3 rather than either category versus category 1.

To verify the result, write \(P(Y=2\mid x,z)=\exp(\eta_2)/D\) and \(P(Y=3\mid x,z)=\exp(\eta_3)/D\), where \(D=1+\exp(\eta_2)+\exp(\eta_3)\). Dividing these probabilities cancels the common denominator, so their ratio is \(\exp(\eta_2-\eta_3)\). Taking logarithms gives the stated log-odds identity. In the linear predictors, subtracting \(\eta_3\) from \(\eta_2\) subtracts the corresponding coefficients, including the slopes. Exponentiating the slope difference gives the odds ratio for a one-unit increase. \(\square\)

Testing a Predictor Jointly Across Categories

A predictor with category-specific slopes contributes more than one coefficient. To test whether \(x\) has any linear-predictor effect in the model, test both of its nonbaseline slopes together:

This is a joint Wald test. It uses the covariance between coefficient estimates as well as their individual estimated variances. In general, treating the two coefficient tests as unrelated does not give the same test.

For the justification, the asymptotic normality assumption gives the null limit of the scaled contrast as a mean-zero normal vector with covariance matrix \(\Sigma=CVC^\mathsf{T}\), where \(V\) is the relevant asymptotic covariance matrix. The rank and nonsingularity assumptions make \(\Sigma\) positive definite. Consistency of the covariance estimator and Slutsky’s theorem allow its estimated version in the quadratic form without changing the limiting distribution. If \(Z\) is a mean-zero normal vector with covariance \(\Sigma\), then \(\Sigma^{-1/2}Z\) is a \(q\)-dimensional standard normal vector. Its squared Euclidean length is the sum of squares of \(q\) independent standard normal variables, which has a \(\chi^2_q\) distribution. Hence the quadratic form has the stated asymptotic null distribution. \(\square\)

Common Pitfall: A Coefficient Is Not a Probability Effect

A frequent error is to interpret \(0.8\), the category-2 slope, as an increase of \(0.8\) in the probability of category 2 for each unit increase in \(x\). That coefficient is a change in log odds relative to category 1. Its exponent, \(\exp(0.8)\), is a multiplicative odds ratio. The probability effect depends on the full set of fitted probabilities, including both nonbaseline linear predictors, and therefore on the starting values of the predictors.

For example, at \(x=0,z=0\), the linear predictors are \(-0.4\) and \(0.2\). The weights are approximately \(0.6703\) and \(1.2214\), giving denominator \(1+0.6703+1.2214=2.8917\). The probabilities are approximately \(0.3458, 0.2318,\) and \(0.4224\). These are not obtained by adding \(0.8\) to any probability. To describe a probability change, calculate the probabilities at both predictor profiles and compare them.

A second important distinction is between a pairwise contrast and a joint test. The pairwise slope contrast \(\beta_{2x}-\beta_{3x}\) describes how category-2-versus-category-3 odds change. The joint test of \(\beta_{2x}=\beta_{3x}=0\) asks whether \(x\) contributes to either baseline logit. They answer different questions, so the contrast should be chosen to match the scientific interpretation of interest.