Linear Contrasts of Logistic Coefficients

Why Compare Linear Combinations of Coefficients?

A logistic-regression coefficient is not always the quantity that answers a scientific question. A coefficient for a continuous predictor describes a one-unit change, while a study may ask about a five-unit change. With an interaction, the treatment coefficient alone does not describe the treatment comparison at every value of the interacting variable. In both cases, the target is a linear combination of fitted coefficients.

Once that combination is written down, the calculation is systematic: estimate the combination, use the coefficient covariance matrix to find its standard error, and exponentiate the combination to express the comparison as an odds ratio. The direction of the comparison must be kept explicit: reversing the two profiles changes the sign of the log-odds contrast and reciprocates the odds ratio.

Defining the Contrast

For a binary outcome coded \(Y=1\) versus \(Y=0\), write the logistic model for a covariate vector \(\mathbf{x}\) as

\operatorname{logit}\{P(Y=1\mid\mathbf{x})\} =\log\left(\frac{P(Y=1\mid\mathbf{x})}{1-P(Y=1\mid\mathbf{x})}\right) =\mathbf{x}^{T}\boldsymbol{\beta}.

Here \(\boldsymbol{\beta}\) is the coefficient vector, including the intercept if the model has one. The covariate vector \(\mathbf{x}\) must contain the model terms for the profile in question, including any interaction or transformed terms. For two specified profiles, let \(\mathbf{x}_0\) be the starting profile and \(\mathbf{x}_1\) the comparison profile. Define \(\mathbf{d}=\mathbf{x}_1-\mathbf{x}_0\).

Definition: A linear contrast of the coefficient vector is a quantity \(C=\mathbf{d}^{T}\boldsymbol{\beta}\), where \(\mathbf{d}\) is a specified, nonrandom vector of known contrast weights. For a comparison of profiles \(\mathbf{x}_1\) versus \(\mathbf{x}_0\), the corresponding contrast vector is \(\mathbf{d}=\mathbf{x}_1-\mathbf{x}_0\).

The weights in \(\mathbf{d}\) encode the question. They are not estimated from the outcome data. A contrast can involve one coefficient, as for a one-unit change in a continuous predictor, or several coefficients, as for a treatment comparison in an interaction model.

From a Profile Comparison to an Odds Ratio

For any profile \(\mathbf{x}\), the model-implied odds of \(Y=1\) are the probability of \(Y=1\) divided by the probability of \(Y=0\). The logit model states that these odds equal \(\exp(\mathbf{x}^{T}\boldsymbol{\beta})\). Thus the odds ratio for profile \(\mathbf{x}_1\) relative to profile \(\mathbf{x}_0\) is determined by the difference of their linear predictors.

Result: In a logistic model, the odds ratio for \(\mathbf{x}_1\) versus \(\mathbf{x}_0\) is \(\exp(\mathbf{d}^{T}\boldsymbol{\beta})\), where \(\mathbf{d}=\mathbf{x}_1-\mathbf{x}_0\).

Proof. Let \(p_j=P(Y=1\mid\mathbf{x}_j)\), for \(j=0,1\). By the model, the odds at profile \(j\) are

\frac{p_j}{1-p_j}=\exp(\mathbf{x}_j^{T}\boldsymbol{\beta}).

Divide the odds at profile 1 by the odds at profile 0. The exponential difference rule gives

\frac{p_1/(1-p_1)}{p_0/(1-p_0)} = \frac{\exp(\mathbf{x}_1^{T}\boldsymbol{\beta})} {\exp(\mathbf{x}_0^{T}\boldsymbol{\beta})} = \exp\left\{(\mathbf{x}_1-\mathbf{x}_0)^{T}\boldsymbol{\beta}\right\} = \exp(\mathbf{d}^{T}\boldsymbol{\beta}).

This proves the result. Reversing the comparison replaces \(\mathbf{d}\) by \(-\mathbf{d}\), so the log-odds contrast changes sign and the odds ratio becomes its reciprocal. \(\square\)

Estimating the Contrast and Its Standard Error

Let \(\hat{\boldsymbol{\beta}}\) be the fitted coefficient vector, and let \(\widehat{\operatorname{Cov}}(\hat{\boldsymbol{\beta}})=\widehat{\mathbf{V}}\) be its estimated covariance matrix. For a specified contrast vector, the estimated log-odds contrast is \(\hat{C}=\mathbf{d}^{T}\hat{\boldsymbol{\beta}}\). Its covariance-based standard error follows from the variance of a linear combination.

Result: The estimated variance of \(\hat{C}\) is \(\widehat{\operatorname{Var}}(\hat{C})=\mathbf{d}^{T}\widehat{\mathbf{V}}\mathbf{d}\), and its estimated standard error is \(\operatorname{SE}(\hat{C})=\sqrt{\mathbf{d}^{T}\widehat{\mathbf{V}}\mathbf{d}}\). The estimated odds ratio is \(\widehat{\operatorname{OR}}=\exp(\hat{C})\).

Proof. Treat \(\mathbf{d}\) as fixed. Write the fitted coefficient vector as \((\hat{\beta}_1,\ldots,\hat{\beta}_q)^T\) and the contrast weights as \((d_1,\ldots,d_q)^T\). By expanding the variance of a sum,

\operatorname{Var}(\mathbf{d}^{T}\hat{\boldsymbol{\beta}}) = \operatorname{Var}\left(\sum_{j=1}^{q}d_j\hat{\beta}_j\right) = \sum_{j=1}^{q}\sum_{k=1}^{q}d_jd_k \operatorname{Cov}(\hat{\beta}_j,\hat{\beta}_k) = \mathbf{d}^{T}\operatorname{Cov}(\hat{\boldsymbol{\beta}})\mathbf{d}.

Replacing the unknown covariance matrix by its estimate gives the stated estimated variance. Taking its nonnegative square root gives the standard error. Exponentiating the estimated log-odds contrast gives the estimated odds ratio by the profile-comparison result above. \(\square\)

The covariance matrix matters because coefficient estimates can be correlated. A standard error is not generally obtained by adding the individual coefficient standard errors, nor by treating the estimates as independent. The formula uses the variances on the diagonal and the covariances off the diagonal. The covariance matrix may be a model-based estimate or, where appropriate for the analysis, a robust covariance estimate; the contrast calculation uses the matrix supplied for the fitted coefficients.

These are plug-in standard errors. Their usual inferential interpretation relies on an appropriate covariance estimate and the regularity conditions for the fitted model, such as an identifiable model with a finite maximum-likelihood estimate and the usual large-sample approximation. The contrast and its standard error are on the log-odds scale; the exponentiated contrast is on the odds-ratio scale.

Worked Calculations

Worked Example: A Four-Unit Increase in a Continuous Predictor

Suppose a model has an intercept and a continuous predictor \(x\), with \(\hat{\boldsymbol{\beta}}=(-0.80,\,0.18)^T\). The estimated covariance matrix, in the same coefficient order, is

\widehat{\mathbf{V}} = \begin{pmatrix} 0.0400 & -0.0030\\ -0.0030 & 0.000625 \end{pmatrix}.

Find the estimated odds ratio for increasing \(x\) by four units, holding the other model terms fixed. Because the starting value is \(x\) and the comparison value is \(x+4\), the intercept cancels and \(\mathbf{d}=(0,4)^T\). The estimated log-odds contrast is

\hat{C} = \begin{pmatrix}0&4\end{pmatrix} \begin{pmatrix}-0.80\\0.18\end{pmatrix} =4(0.18)=0.72.

The estimated variance and standard error are

\widehat{\operatorname{Var}}(\hat{C}) = \begin{pmatrix}0&4\end{pmatrix} \widehat{\mathbf{V}} \begin{pmatrix}0\\4\end{pmatrix} =16(0.000625)=0.0100, \qquad \operatorname{SE}(\hat{C})=\sqrt{0.0100}=0.10.

Finally, the estimated odds ratio is \(\exp(0.72)\approx 2.054\). The fitted model therefore estimates about 2.05 times the odds for the profile four units higher in \(x\), with the other terms unchanged. The negative intercept-slope covariance does not enter this particular contrast because its intercept weight is zero.

Worked Example: A Treatment Comparison at a Specific Interaction Level

Suppose a model contains an indicator \(T\) for treatment and a continuous covariate \(z\), with linear predictor

\beta_0+\beta_z z+\beta_T T+\beta_{zT}zT.

The fitted coefficients are \((\hat{\beta}_0,\hat{\beta}_z,\hat{\beta}_T,\hat{\beta}_{zT})=(-2.0,\,0.4,\,0.7,\,-0.2)\). For the treatment comparison \(T=1\) versus \(T=0\) at \(z=2\), the design vectors are \((1,2,0,0)^T\) and \((1,2,1,2)^T\). Their difference is \(\mathbf{d}=(0,0,1,2)^T\). Thus

\hat{C} = \begin{pmatrix}0&0&1&2\end{pmatrix} \begin{pmatrix}-2.0\\0.4\\0.7\\-0.2\end{pmatrix} =0.7+2(-0.2)=0.3.

Suppose the estimated covariance submatrix for \((\hat{\beta}_T,\hat{\beta}_{zT})\) is

\begin{pmatrix} 0.04 & -0.01\\ -0.01 & 0.01 \end{pmatrix}.

Only this submatrix is needed because the other two contrast weights are zero. Therefore,

\widehat{\operatorname{Var}}(\hat{C}) = 0.04+2(1)(2)(-0.01)+2^2(0.01) =0.04-0.04+0.04=0.04, \qquad \operatorname{SE}(\hat{C})=0.20.

The estimated treatment odds ratio at \(z=2\) is \(\exp(0.3)\approx1.350\). It is not \(\exp(0.7)\): that would use the treatment coefficient alone and omit the interaction contribution at \(z=2\).

Worked Example: Comparing Two Nonreference Factor Levels

Let a three-level factor have levels A, B, and C, with A as the reference. Suppose the coefficient order is intercept, indicator for B, indicator for C, and the fitted coefficients are \((-1.1,\,0.3,\,0.8)\). The comparison C versus A has contrast vector \(\mathbf{d}=(0,0,1)^T\), so \(\hat{C}=0.8\) and \(\widehat{\operatorname{OR}}=\exp(0.8)\approx2.226\).

Now compare C versus B. The profile vectors are \((1,0,1)^T\) for C and \((1,1,0)^T\) for B, so the contrast vector is \(\mathbf{d}=(0,-1,1)^T\). The estimate is

\hat{C}=-0.3+0.8=0.5, \qquad \widehat{\operatorname{OR}}=\exp(0.5)\approx1.649.

Suppose the estimated variances of the B and C coefficients are \(0.09\) and \(0.16\), respectively, and their estimated covariance is \(0.03\). Then

\widehat{\operatorname{Var}}(\hat{C}) = (-1)^2(0.09)+(1)^2(0.16)+2(-1)(1)(0.03) =0.09+0.16-0.06=0.19, \qquad \operatorname{SE}(\hat{C})=\sqrt{0.19}\approx0.436.

The odds ratio is for C relative to B. Reversing the comparison gives the estimate \(-0.5\) on the log-odds scale and \(\exp(-0.5)\approx0.607\) for B relative to C, the reciprocal of approximately 1.649.

A Reliable Calculation and a Common Pitfall

Write the profiles.
Specify the starting and comparison values of every model term, including interaction and indicator terms.

Subtract to get the contrast weights.
Compute \(\mathbf{d}=\mathbf{x}_1-\mathbf{x}_0\), keeping the comparison direction fixed.

Calculate the estimate and standard error.
Use \(\mathbf{d}^{T}\hat{\boldsymbol{\beta}}\) and \(\sqrt{\mathbf{d}^{T}\widehat{\mathbf{V}}\mathbf{d}}\).

Exponentiate the estimate.
The odds ratio for profile 1 versus profile 0 is \(\exp(\mathbf{d}^{T}\hat{\boldsymbol{\beta}})\).

Common mistake: reporting a coefficient when the question asks for a contrast. For example, one might say that the treatment odds ratio at \(z=2\) is \(\exp(\hat{\beta}_T)\), reasoning that \(\hat{\beta}_T\) is the treatment coefficient. In a model with a \(z\)-by-treatment interaction, that coefficient describes the treatment comparison at \(z=0\), not at \(z=2\). The correction is to build the two profile vectors and subtract them; at \(z=2\), the contrast is \(\hat{\beta}_T+2\hat{\beta}_{zT}\). A second frequent error is to omit covariance terms when several coefficients contribute. The full matrix expression includes them automatically.

Key takeaway: A requested comparison determines the contrast weights. Subtract the covariate design vectors in the desired order, apply those weights to both the coefficient estimates and their covariance matrix, and exponentiate the estimated contrast to obtain the odds ratio.

Check Your Understanding

For each question, identify the contrast before calculating.

In a logistic model with a continuous predictor \(x\), what contrast vector represents an increase of three units in \(x\), with all other covariates fixed?
A model contains treatment \(T\), covariate \(z\), and interaction \(zT\). Which coefficients contribute to the treatment comparison at \(z=4\)?
Why does the variance of a contrast generally require covariances between coefficient estimates as well as their variances?
If the estimated log-odds contrast for profile 1 versus profile 0 is \(-0.4\), what is the estimated odds ratio, and what happens to the log-odds contrast when the comparison is reversed?
With a categorical predictor whose reference level is A, how would you construct the contrast for C versus B rather than C versus A?