The Log-Odds Versus Probability Interpretation Pitfall

Why a Positive Slope Can Mislead

In a multinomial logit model, a positive slope for one category has a direct interpretation: as a predictor increases, that category’s odds relative to the reference category increase. It is tempting to conclude that the category’s predicted probability must also increase. That conclusion is not generally valid. Predicted probabilities are shares of a common total, so their changes depend on how the linear predictors for all categories move.

The distinction matters when interpreting fitted models. A statement about an odds ratio is not automatically a statement about a probability difference, and a positive coefficient alone does not settle the direction of a probability change. We will make the distinction precise and calculate both quantities.

Definition: In a baseline-category multinomial logit model with reference category $0$, let $\eta_0(x)=0$ and let $\eta_j(x)=\alpha_j+\beta_j x$ for each nonreference category $j=1,\ldots,J$. The predicted category probabilities are $$ p_j(x)=\frac{\exp\{\eta_j(x)\}}{\sum_{r=0}^{J}\exp\{\eta_r(x)\}}, \qquad j=0,\ldots,J. $$ Here $\exp\{\eta_0(x)\}=1$. The slope $\beta_j$ describes the change in the log odds of category $j$ versus category $0$ per one-unit increase in $x$; it is not, by itself, the slope of $p_j(x)$.

What the Slope Says About Odds

Because $p_j(x)/p_0(x)=\exp\{\eta_j(x)\}$, the log odds of category $j$ versus the reference are $\log\{p_j(x)/p_0(x)\}=\alpha_j+\beta_jx$. For a change from $x$ to $x+\Delta$, where the other model terms are held fixed, the odds ratio is therefore $\exp(\beta_j\Delta)$. If $\beta_j>0$ and $\Delta>0$, this odds ratio exceeds one. This is a precise claim about the odds of $j$ relative to category $0$.

It is not a claim that $p_j(x+\Delta)>p_j(x)$. The probability $p_j$ has the sum of all category weights in its denominator. A different category’s weight may increase faster and take a larger share of the total, even as category $j$’s odds against the reference increase.

Result: Suppose there are finitely many categories, $\eta_0(x)=0$, and $\eta_j(x)=\alpha_j+\beta_jx$ for $j=1,\ldots,J$. Then $$ \frac{d p_j(x)}{dx} = p_j(x)\left(\beta_j-\sum_{r=0}^{J}p_r(x)\beta_r\right), $$ where $\beta_0=0$. Thus, for $p_j(x)>0$, the probability of category $j$ is increasing at $x$ exactly when its slope exceeds the probability-weighted average slope across all categories.

To derive the formula, write $w_r(x)=\exp\{\eta_r(x)\}$ and $S(x)=\sum_{r=0}^{J}w_r(x)$. For the reference category, $w_0(x)=1$, so $w_0'(x)=0=\beta_0w_0(x)$. For every category, $w_r'(x)=\beta_rw_r(x)$. Since the sum is finite, differentiating $S$ term by term gives $S'(x)=\sum_{r=0}^{J}\beta_rw_r(x)$. Applying the quotient rule,

\begin{aligned} p_j'(x) &=\frac{\beta_j w_j(x)S(x)-w_j(x)S'(x)}{S(x)^2}\\ &=\frac{w_j(x)}{S(x)} \left(\beta_j-\frac{\sum_{r=0}^{J}\beta_rw_r(x)}{S(x)}\right)\\ &=p_j(x)\left(\beta_j-\sum_{r=0}^{J}p_r(x)\beta_r\right). \end{aligned}

The last equality follows from $p_r(x)=w_r(x)/S(x)$. Since $p_j(x)>0$, the derivative has the sign of the difference between $\beta_j$ and the weighted average slope. A positive $\beta_j$ can therefore coexist with a negative derivative. $\square$

Worked Numerical Examples

Worked Example: Positive Slope but Falling Probability

Consider three categories: reference $R$, category $A$, and category $B$. Let their weights be $1$, $\exp(x)$, and $4\exp(2x)$, respectively. Thus $\beta_A=1>0$, $\beta_B=2$, and the reference slope is $0$.

At $x=0$, the weights are $1,1,4$, their sum is $6$, and $p_A(0)=1/6\approx0.1667$. The weighted average slope at zero is $(1/6)(0)+(1/6)(1)+(4/6)(2)=9/6=1.5$. The derivative formula gives $p_A'(0)=(1/6)(1-1.5)=-1/12\approx-0.0833$. So $A$'s predicted probability is falling at $x=0$, despite its positive slope.

Check a finite increase of $0.1$. The new weights are $1$, $\exp(0.1)\approx1.1052$, and $4\exp(0.2)\approx4.8856$. Their total is approximately $6.9908$, so $p_A(0.1)\approx1.1052/6.9908\approx0.1580$, below $0.1667$. Nevertheless, the odds of $A$ versus $R$ increase from $1$ to $\exp(0.1)\approx1.1052$. There is no contradiction: the odds against $R$ rise while the faster-growing weight for $B$ reduces $A$'s share of the total.

Worked Example: A Positive Slope and a Rising Probability

Now let the reference weight be $1$, the weight for $A$ be $(1/3)\exp(x)$, and the weight for $B$ be $1/3$. The slope for $A$ is again positive, $\beta_A=1$, while $\beta_B=0$.

At $x=0$, the total weight is $1+1/3+1/3=5/3$, giving $p_A(0)=(1/3)/(5/3)=0.2000$. At $x=0.1$, the weights are $1$, $(1/3)\exp(0.1)\approx0.3684$, and $1/3\approx0.3333$. Their total is approximately $1.7017$, and $p_A(0.1)\approx0.3684/1.7017\approx0.2165$. Here the probability rises.

The odds of $A$ versus the reference move from $1/3\approx0.3333$ to $0.3684$, a multiplication by $\exp(0.1)\approx1.1052$, as the model specifies. This example shows that a positive slope is consistent with a rising probability; the other categories’ slopes and current probabilities matter in determining the direction.

Worked Example: An Odds Ratio Is Not a Probability Difference

Take three categories $R,A,B$ with weights $1,\exp\{(\log 2)x\},1$. At $x=0$, all weights are $1$, so $p_A(0)=1/3$. At $x=1$, the weights are $1,2,1$, so $p_A(1)=2/4=1/2$.

The odds of $A$ versus $R$ are $1$ at $x=0$ and $2$ at $x=1$: the odds ratio is $2$, matching $\exp(\beta_A)=\exp(\log 2)=2$. The probability change, however, is $1/2-1/3=1/6$, or about $0.1667$ (16.67 percentage points). It is not a doubling of the probability. The probability has risen by a factor of $(1/2)/(1/3)=1.5$, while the category odds against the reference have doubled.

Why the Probabilities Compete

The derivative formula describes a local change: it gives the instantaneous rate at a particular value of $x$, not necessarily the total change over a large interval. The condition for category $j$ to be increasing can also be written $\beta_j>\sum_{r\ne j}p_r(x)\beta_r/(1-p_j(x))$, when $p_j(x)<1$. The right-hand side is the probability-weighted average slope among the other categories. A category’s probability rises when its slope exceeds that average, and falls when its slope is smaller.

The probabilities must sum to one. Differentiating that identity gives $\sum_{j=0}^{J}p_j'(x)=0$: increases in some category probabilities must be balanced by decreases in others. A positive logit slope does not make the probability sum grow; it changes the balance of shares.

For a finite predictor change, use the fitted probabilities at both predictor values if the question concerns a probability difference. A derivative at the starting value only describes the local direction; the weighted average slopes can change as the probabilities change. By contrast, under the linear logit specification here, the odds ratio against the reference over a finite change $\Delta$ is exactly $\exp(\beta_j\Delta)$.

Common Mistake

Common mistake: “The coefficient for category $A$ is positive, so increasing the predictor must increase $P(Y=A)$.” This reasoning confuses a log-odds slope with a probability slope. The coefficient says the odds of $A$ versus the reference increase. To determine the direction of $P(Y=A)$, compare $\beta_A$ with the probability-weighted average slope across all categories, or calculate the predicted probabilities at the values of interest.

Keep the claim matched to the quantity being interpreted. “The odds of $A$ versus $R$ are multiplied by 1.5” is an odds-ratio statement. “The probability of $A$ rises by 0.04” is a probability-difference statement. Neither can be substituted for the other without a calculation.

Key takeaway: A positive category-specific slope guarantees increasing odds against the reference as the predictor rises, under the stated linear logit model. It does not guarantee an increasing category probability. The probability’s direction depends on how that category’s slope compares with the probability-weighted slopes of all categories.

Check Your Understanding

Answer each question by distinguishing odds claims from probability claims.

In a baseline-category logit, what odds ratio corresponds to increasing $x$ by two units when a category’s slope is $0.3$?
For three categories with slopes $0,1,2$ and probabilities $0.2,0.5,0.3$, respectively, is the probability for the category with slope $1$ increasing at that predictor value? Use the derivative condition.
Can a category’s probability decrease while its odds against the reference increase? Explain how the denominator permits this.
Why does an odds ratio of $2$ not mean that a category’s predicted probability doubles?
What additional calculation is needed to report a finite probability change between two predictor values?