Average Marginal Effects in Logistic Regression

From a Logistic Model to a Probability Derivative

A logistic-regression coefficient for a continuous predictor describes a change in log odds, not a constant change in probability. The probability effect depends on the covariate profile at which it is evaluated. An average marginal effect summarizes those profile-specific derivatives by averaging them over a specified set of observations.

Write the fitted conditional probability at covariate profile $x_i$ as $p_i=P(Y_i=1\mid x_i)$, with linear predictor $\eta_i=x_i^\mathsf{T}\beta$. The logistic mean function is $p_i=\exp(\eta_i)/(1+\exp(\eta_i))$. For a continuous predictor $x_j$, its probability derivative is found by the chain rule.

Definition: For a differentiable logistic-regression model, the marginal effect of a continuous predictor $x_j$ at profile $x_i$ is $$ \frac{\partial p_i}{\partial x_{ij}}. $$ The average marginal effect over $n$ specified profiles is the arithmetic mean of these derivatives: $$ \operatorname{AME}_j=\frac{1}{n}\sum_{i=1}^{n}\frac{\partial p_i}{\partial x_{ij}}. $$ When model parameters are estimated, the reported plug-in estimate replaces $\beta$ by $\hat{\beta}$ in each derivative.

The profiles being averaged over matter: the empirical AME averages over the covariate values in the chosen sample. It is a summary of model-implied probability derivatives for those profiles, holding the other modeled covariates fixed. It is not automatically a causal effect; causal interpretation requires additional design and identification assumptions.

The Logistic Derivative and Its Interpretation

Let $\eta_i$ depend differentiably on $x_{ij}$, and define $p_i=\exp(\eta_i)/(1+\exp(\eta_i))$. Differentiating the logistic function gives the key identity.

Theorem: At any profile where $\eta_i$ is differentiable with respect to $x_{ij}$, $$ \frac{\partial p_i}{\partial x_{ij}} =p_i(1-p_i)\frac{\partial \eta_i}{\partial x_{ij}}. $$ In an additive logistic model with no interactions or nonlinear terms involving $x_j$, $\eta_i=\beta_0+\beta_jx_{ij}+\sum_{k\ne j}\beta_kx_{ik}$, so $$ \frac{\partial p_i}{\partial x_{ij}}=\beta_jp_i(1-p_i). $$

Proof. Write $p_i=e^{\eta_i}/(1+e^{\eta_i})$. Differentiating first with respect to $\eta_i$ gives $$ \frac{d p_i}{d\eta_i} =\frac{e^{\eta_i}(1+e^{\eta_i})-e^{2\eta_i}}{(1+e^{\eta_i})^2} =\frac{e^{\eta_i}}{(1+e^{\eta_i})^2} =p_i(1-p_i). $$ Applying the chain rule to $\eta_i(x_{ij})$ proves the general identity. In the additive model, $\partial\eta_i/\partial x_{ij}=\beta_j$, which gives the stated special case. $\square$

The factor $p_i(1-p_i)$ varies with the profile. It is largest at $p_i=1/2$ and approaches zero as $p_i$ approaches either zero or one. Thus, even when the predictor’s log-odds slope is constant, its probability derivative is not generally constant. More generally, when the linear predictor contains an interaction or a transformed predictor, $\partial\eta_i/\partial x_{ij}$ can also vary across profiles.

Worked Example: Averaging Derivatives in an Additive Model

Suppose $P(Y=1\mid x)=\operatorname{expit}(-2+0.8x)$, where $\operatorname{expit}(u)=e^u/(1+e^u)$. Calculate the marginal effects at $x=0,1,2$, their average, and the effect at the mean profile.

For this model the derivative is $0.8p(1-p)$. At $x=0$, $\eta=-2$, so $p=0.119203$, and the derivative is $0.8(0.119203)(0.880797)=0.083995$.

At $x=1$, $\eta=-1.2$, so $p=0.231475$, and the derivative is $0.8(0.231475)(0.768525)=0.142316$.

At $x=2$, $\eta=-0.4$, so $p=0.401312$, and the derivative is $0.8(0.401312)(0.598688)=0.192209$.

The sample average is $(0.083995+0.142316+0.192209)/3=0.139507$, approximately. The mean predictor value is $\bar{x}=1$, so the derivative at the mean profile is $0.142316$. These are different summaries: the AME averages the three derivatives, while the derivative at the mean evaluates the model once at $x=1$.

Interactions and Other Predictor Terms

The general derivative formula remains valid when the predictor’s effect depends on other covariates. The necessary step is to differentiate the whole linear predictor with respect to the continuous variable, including every term that contains it.

Worked Example: A Continuous Predictor with an Interaction

Consider $\eta=-1+0.5x+0.8z+0.4xz$, where $x$ is continuous and $z$ is a binary covariate. Calculate the marginal effect of $x$ at profiles $(x,z)=(0,0)$ and $(1,1)$, then average the two effects.

Differentiating the linear predictor with respect to $x$ gives $\partial\eta/\partial x=0.5+0.4z$. At $(0,0)$, $\eta=-1$, $p=\operatorname{expit}(-1)=0.268941$, and $\partial\eta/\partial x=0.5$. Therefore the probability derivative is $0.268941(1-0.268941)(0.5)=0.098306$.

At $(1,1)$, $\eta=-1+0.5+0.8+0.4=0.7$, so $p=\operatorname{expit}(0.7)=0.668188$, and $\partial\eta/\partial x=0.5+0.4=0.9$. The probability derivative is $0.668188(1-0.668188)(0.9)=0.199542$.

The average of these two profile-specific effects is $(0.098306+0.199542)/2=0.148924$. The interaction changes the log-odds slope for $x$ across values of $z$, and the logistic factor also changes across profiles. Both features contribute to the difference between the two probability derivatives.

The same method applies to smooth transformations. For example, if the linear predictor includes $\gamma x^2$, then its derivative with respect to $x$ includes $2\gamma x$. If a model includes both $x$ and $xz$, the derivative includes the coefficient of $x$ plus $z$ times the interaction coefficient. Differentiation is with respect to the named continuous predictor while the other covariates are held fixed.

Why the Average Is Not the Effect at the Mean

For a finite set of profiles, the AME and the marginal effect at their mean profile answer different questions. The AME is the mean of the observation-specific derivatives. The effect at the mean profile first constructs a profile from mean covariate values and then evaluates one derivative there. Because the logistic probability function is nonlinear, these operations generally do not produce the same number.

Theorem: Suppose each $p_i(t)$ is differentiable in a common shift $t$ to a continuous predictor, and define the average fitted probability $A(t)=n^{-1}\sum_{i=1}^{n}p_i(t)$. Then $$ A'(t)=\frac{1}{n}\sum_{i=1}^{n}p_i'(t). $$ In particular, when the same small shift is applied to that predictor in every profile, the derivative of the average fitted probability is the AME for those profiles.

Proof. The sum has finitely many terms, each differentiable by assumption. Differentiation is therefore linear over this finite sum: $$ A'(t)=\frac{d}{dt}\left(\frac{1}{n}\sum_{i=1}^{n}p_i(t)\right) =\frac{1}{n}\sum_{i=1}^{n}\frac{d p_i(t)}{dt}. $$ Each term on the right is the probability derivative at profile $i$ under the common shift. Their arithmetic mean is precisely the AME. $\square$

This result concerns the derivative of the average fitted probability under a common shift. It does not say that the AME equals the derivative evaluated at average covariate values. With interactions, a mean profile may also fail to represent the average of the interaction terms: the mean of $xz$ need not equal the product of the means of $x$ and $z$.

Worked Example: An Odds Ratio Is Not a Probability Effect

For the model $P(Y=1\mid x)=\operatorname{expit}(-2+0.8x)$, the odds ratio for a one-unit increase in $x$ is $\exp(0.8)=2.226$, approximately. Compute the corresponding one-unit probability changes at $x=0,1,2$.

At $x=0$, the probability is $\operatorname{expit}(-2)=0.119203$. At $x=1$, it is $\operatorname{expit}(-1.2)=0.231475$. The probability change is $0.231475-0.119203=0.112272$.

From $x=1$ to $x=2$, the probabilities are $0.231475$ and $\operatorname{expit}(-0.4)=0.401312$, giving a change of $0.169837$. From $x=2$ to $x=3$, the probabilities are $0.401312$ and $\operatorname{expit}(0.4)=0.598688$, giving a change of $0.197376$.

The odds are multiplied by the same factor, about $2.226$, for each one-unit increase. The probability changes differ across starting values of $x$. These finite one-unit changes are not derivatives, either: a derivative describes a local rate of change, whereas these differences compare probabilities at two distinct predictor values.

Estimation, Units, and Common Pitfalls

In an analysis, the model parameters are usually estimated. For an additive model, the estimated marginal effect at observation $i$ is $\hat{\beta}_j\hat{p}_i(1-\hat{p}_i)$, where $\hat{p}_i=\operatorname{expit}(x_i^\mathsf{T}\hat{\beta})$. The estimated AME is the arithmetic mean of these $n$ quantities. With interactions or nonlinear terms, first calculate $\partial\eta_i/\partial x_{ij}$ using $\hat{\beta}$, and multiply it by $\hat{p}_i(1-\hat{p}_i)$ at each profile before averaging.

Units require care. If $x_j$ is measured in years, its derivative is the local change in probability per year. Re-expressing it in months changes the numerical derivative by the corresponding scale factor, though the underlying model predictions need not change when the coefficient is rescaled consistently. It is therefore good practice to state the predictor’s units and the profiles or sample over which the average was taken.

Common mistake: Reporting $\hat{\beta}_j$ as a constant percentage-point change in probability per unit increase is incorrect. The coefficient is a change in log odds in an additive model; the probability derivative is $\hat{\beta}_j\hat{p}_i(1-\hat{p}_i)$, which varies with fitted probability. The correction is to calculate the derivative at each profile and average those derivatives when the target is an AME. Similarly, $\exp(\hat{\beta}_j)$ is an odds ratio for a one-unit increase under the additive specification, not an AME.

Key takeaway: An average marginal effect is an average of model-based probability derivatives across specified covariate profiles. It differs from a log-odds coefficient, an odds ratio, a finite probability difference, and generally from the derivative at the mean profile.

Check Your Understanding

Use the logistic derivative and distinguish the target summaries in each question.

In an additive logistic model, what two factors determine the probability derivative for a continuous predictor at a given profile?
For a model with linear predictor $\eta=\beta_0+\beta_1x+\beta_2z+\beta_3xz$, write the derivative of the probability with respect to $x$.
Explain why the average marginal effect over observed profiles need not equal the derivative evaluated at the mean profile.
In an additive logistic model, what does $\exp(\beta_j)$ describe, and why is it not a probability derivative?
If a predictor is measured in years, what unit should accompany its probability derivative?