Contrasts for Combined Predictor Changes

Why Combined Predictor Changes Need Contrasts

A GLM often describes a response using several predictors at once. A practical comparison may change more than one predictor: for example, a hypothetical policyholder might move to a different rating category while also having a different exposure measure. The fitted change depends on the coefficients for all the terms that change, including any relevant interactions.

A linear contrast is a specified linear combination of model coefficients. It lets us estimate and quantify uncertainty for the particular comparison of interest, rather than treating each coefficient as if it answered the whole question. The same contrast can have different interpretations depending on the link: under a log link it can imply a rate ratio, while under a logit link it is a change in log odds. A response-scale difference generally requires evaluating the inverse link at both profiles.

Definition: Let a fitted GLM have coefficient vector \(\hat{\boldsymbol{\beta}}\) and estimated coefficient covariance matrix \(\widehat{\mathbf{V}}\). For a specified vector \(\mathbf{c}\), the estimated linear contrast is \(L=\mathbf{c}^{\mathsf T}\hat{\boldsymbol{\beta}}\). Its estimated standard error is \(\operatorname{SE}(L)=\sqrt{\mathbf{c}^{\mathsf T}\widehat{\mathbf{V}}\mathbf{c}}\). The vector \(\mathbf{c}\) specifies the comparison; its entries are not necessarily all 0 or 1.

Constructing and Quantifying a Contrast

For a GLM, write the linear predictor at a covariate profile \(\mathbf{z}\) as \(\eta(\mathbf{z})=\mathbf{z}^{\mathsf T}\boldsymbol{\beta}+o(\mathbf{z})\), where \(o(\mathbf{z})\) is a known offset, if the model has one. For two profiles, called 0 and 1, their link-scale difference is

\eta_1-\eta_0 =(\mathbf{z}_1-\mathbf{z}_0)^{\mathsf T}\boldsymbol{\beta} +\bigl(o_1-o_0\bigr).

If the offset is the same at both profiles, it cancels. Otherwise its known difference must be included in the estimated link-scale change; it does not add coefficient-estimation variance. The coefficient contrast has vector \(\mathbf{c}=\mathbf{z}_1-\mathbf{z}_0\). With a fitted coefficient vector, its estimate is \(\hat L=\mathbf{c}^{\mathsf T}\hat{\boldsymbol{\beta}}\), and its standard error uses the estimated covariance matrix, not just the individual standard errors.

Why the variance formula holds: A contrast is a weighted sum of estimated coefficients. By the variance rule for a linear combination, \(\operatorname{Var}(\mathbf{c}^{\mathsf T}\hat{\boldsymbol{\beta}})=\sum_j\sum_k c_jc_k\operatorname{Cov}(\hat\beta_j,\hat\beta_k)=\mathbf{c}^{\mathsf T}\mathbf{V}\mathbf{c}\). The off-diagonal covariance terms matter whenever coefficient estimates are correlated. Replacing this expression with a sum of squared standard errors omits both the contrast weights and those covariances.

In large samples, a Wald statistic for testing a zero contrast is \(z=\hat L/\operatorname{SE}(\hat L)\), when the null value is zero and the usual approximate normal inference is appropriate. A corresponding approximate 95% confidence interval on the link scale is \(\hat L\pm1.96\operatorname{SE}(\hat L)\). These are model-based approximations: they rely on the fitted model being suitable and the coefficient estimates having an approximately normal sampling distribution. A contrast interval concerns the specified combination; separate tests of its component coefficients do not replace it.

Worked Example: A Combined Change Under a Log Link

Suppose a hypothetical claim-frequency model has a log link and linear predictor \(\eta=\beta_0+\beta_xx+\beta_gg+\beta_{xg}xg\), where \(g=0\) or \(1\). Compare the fitted rates for profile \((x=0,g=0)\) and profile \((x=5,g=1)\), with the same exposure offset at both profiles. The estimates are \(\hat\beta_x=0.03\), \(\hat\beta_g=0.25\), and \(\hat\beta_{xg}=-0.01\). For these three coefficients, the estimated variances are \(0.0004, 0.0064, 0.0009\), respectively; the covariances are \(\operatorname{Cov}(\hat\beta_x,\hat\beta_g)=0.0002\), \(\operatorname{Cov}(\hat\beta_x,\hat\beta_{xg})=-0.0001\), and \(\operatorname{Cov}(\hat\beta_g,\hat\beta_{xg})=0.0006\).

Method and contrast. The two profiles differ by 5 units in \(x\), by 1 in \(g\), and by 5 in the product \(xg\). Thus \(\mathbf{c}=(5,1,5)\), and the intercept cancels. The estimated change in log rate is

\hat L=5(0.03)+1(0.25)+5(-0.01) =0.15+0.25-0.05=0.35.

Standard error. Applying the contrast variance formula gives

\begin{aligned} \widehat{\operatorname{Var}}(\hat L) &=25(0.0004)+1(0.0064)+25(0.0009)\\ &\quad+2(5)(1)(0.0002)+2(5)(5)(-0.0001)+2(1)(5)(0.0006)\\ &=0.0100+0.0064+0.0225+0.0020-0.0050+0.0060\\ &=0.0419,\\ \operatorname{SE}(\hat L)&=\sqrt{0.0419}=0.2047. \end{aligned}

Interpretation and check. Under the log link, exponentiating a link-scale difference gives the rate ratio. The estimated ratio for profile 1 relative to profile 0 is \(\exp(0.35)=1.419\), or about 1.42. The approximate 95% interval on the log scale is \(0.35\pm1.96(0.2047)=(-0.0512,0.7512)\); exponentiating the endpoints gives an approximate rate-ratio interval of \((0.950,2.120)\). This calculation uses all three changing terms and their covariances. A common mistake is to use only \(5\hat\beta_x\), overlooking both the category change and the interaction; that would give a log-rate change of 0.15 rather than 0.35.

From a Link-Scale Contrast to a Response-Scale Change

The contrast \(\eta_1-\eta_0\) is not generally the difference in fitted responses. Let \(h=g^{-1}\) be the inverse link. The fitted response-scale difference between two profiles is

\hat\Delta=h(\mathbf{z}_1^{\mathsf T}\hat{\boldsymbol{\beta}}+o_1) -h(\mathbf{z}_0^{\mathsf T}\hat{\boldsymbol{\beta}}+o_0).

For a log link, exponentiating the link difference gives a ratio of fitted means, not their difference. For a logit link, exponentiating gives an odds ratio, not a probability difference. In both cases, the response-scale difference depends on the fitted values at both profiles.

When uncertainty for a response-scale difference is needed, the delta method gives an approximate standard error. If \(\Delta(\boldsymbol{\beta})=h(\mathbf{z}_1^{\mathsf T}\boldsymbol{\beta}+o_1)-h(\mathbf{z}_0^{\mathsf T}\boldsymbol{\beta}+o_0)\), its gradient is

\nabla\Delta =h'(\eta_1)\mathbf{z}_1-h'(\eta_0)\mathbf{z}_0, \qquad \widehat{\operatorname{SE}}(\hat\Delta) \approx\sqrt{(\nabla\Delta)^{\mathsf T}\widehat{\mathbf{V}}(\nabla\Delta)}.

This follows by a first-order Taylor approximation to the smooth function \(\Delta(\hat{\boldsymbol{\beta}})\) around the fitted coefficients. It quantifies uncertainty in the fitted mean difference, not the variation of a future individual response around that mean. The approximation can be less reliable when the response-scale function is strongly curved over the range of coefficient uncertainty.

Worked Example: A Logistic Contrast and Probability Difference

Consider a hypothetical binary-outcome model with logit link, \(\eta=\beta_0+\beta_xx+\beta_gg+\beta_{xg}xg\). The estimated coefficients are \((-1.2,0.4,0.5,-0.2)\), in that order. Their estimated covariance matrix is diagonal, with variance \(0.04\) for each coefficient. Compare profile 0, \((x=0,g=0)\), with profile 1, \((x=1,g=1)\).

Link-scale contrast. The design vectors are \(\mathbf{z}_0=(1,0,0,0)\) and \(\mathbf{z}_1=(1,1,1,1)\), so the contrast vector is \((0,1,1,1)\). Therefore,

\hat L=0.4+0.5-0.2=0.7, \qquad \widehat{\operatorname{Var}}(\hat L)=0.04+0.04+0.04=0.12, \qquad \operatorname{SE}(\hat L)=\sqrt{0.12}=0.3464.

The estimated change is 0.7 in log odds. Equivalently, the odds ratio is \(\exp(0.7)=2.014\), approximately 2.01.

Response-scale difference. The linear predictors are \(\hat\eta_0=-1.2\) and \(\hat\eta_1=-1.2+0.4+0.5-0.2=-0.5\). Applying the inverse logit \(h(\eta)=1/(1+\exp(-\eta))\) gives \(\hat p_0=0.2314753\) and \(\hat p_1=0.3775407\). Subtracting the unrounded fitted probabilities gives

\hat\Delta=0.3775407-0.2314753=0.1460654\approx0.1461.

Thus the fitted probability is about 14.61 percentage points higher at profile 1. The correctly rounded difference is based on the unrounded probabilities; subtracting the displayed four-decimal values gives 0.1460 and does not preserve the correct rounding.

Approximate response-scale standard error. For the inverse logit, \(h'(\eta)=h(\eta)\{1-h(\eta)\}\). Here \(h'(\hat\eta_0)\approx0.1779\) and \(h'(\hat\eta_1)\approx0.2350\). The gradient for the four coefficients, in their stated order, is approximately \((0.0571,0.2350,0.2350,0.2350)\): the intercept component is \(0.2350-0.1779\), and each other component is \(0.2350\). Because the covariance matrix is diagonal with entries 0.04,

\widehat{\operatorname{Var}}(\hat\Delta) \approx0.04\{(0.0571)^2+3(0.2350)^2\} \approx0.006758, \qquad \widehat{\operatorname{SE}}(\hat\Delta)\approx0.0822.

This delta-method standard error is on the probability-difference scale. A common mistake is to report the odds ratio of about 2.01 as if it were a probability increase; the probability change is the separately computed difference of about 0.1461.

Combined Changes on the Identity Scale

With an identity link, the linear predictor is the fitted mean, so a coefficient contrast directly gives the fitted response difference. Interactions still matter: a change in one predictor can have a different effect at different values of another predictor. The covariance calculation remains the same as for any other coefficient contrast.

Worked Example: A Combined Difference With an Interaction

Suppose a hypothetical continuous response is modeled with a Gaussian distribution and identity link: \(\mu=\beta_0+\beta_xx+\beta_gg+\beta_{xg}xg\), where \(g=0\) or \(1\). Compare \((x=2,g=0)\) with \((x=5,g=1)\). Let the estimates be \(\hat\beta_x=1.2\), \(\hat\beta_g=-2\), and \(\hat\beta_{xg}=0.4\). Their variances are \(0.09,0.16,0.04\), and their covariances are \(\operatorname{Cov}(\hat\beta_x,\hat\beta_g)=-0.03\), \(\operatorname{Cov}(\hat\beta_x,\hat\beta_{xg})=0.015\), and \(\operatorname{Cov}(\hat\beta_g,\hat\beta_{xg})=-0.02\). Assume the supplied covariance estimates are appropriate for large-sample Wald inference.

Construct the contrast. The design vectors are \((1,2,0,0)\) and \((1,5,1,5)\), so their difference is \(\mathbf{c}=(0,3,1,5)\). The fitted response difference is

\hat L=3(1.2)+1(-2)+5(0.4)=3.6-2+2=3.6.

Calculate its uncertainty. The intercept has contrast weight zero and drops out. Using the variances and covariances,

\begin{aligned} \widehat{\operatorname{Var}}(\hat L) &=9(0.09)+1(0.16)+25(0.04)\\ &\quad+2(3)(1)(-0.03)+2(3)(5)(0.015)+2(1)(5)(-0.02)\\ &=0.81+0.16+1.00-0.18+0.45-0.20=2.04,\\ \operatorname{SE}(\hat L)&=\sqrt{2.04}=1.4283. \end{aligned}

The estimated mean response is 3.6 units higher at profile 1. The Wald statistic is \(3.6/1.4283\approx2.52\), and the approximate 95% interval is \(3.6\pm1.96(1.4283)\), or approximately \((0.80,6.40)\) response units. The interval is for the combined profile difference. Looking only at whether the separate \(x\), \(g\), or interaction coefficients are individually significant would not directly answer this comparison.

Common Contrast Traps

Forgetting an interaction. The coefficient for a predictor may not describe its change across all values of another predictor. Subtract the complete design vectors for the two profiles.
Ignoring covariance. The contrast variance includes covariance terms, which may increase or decrease the result. Use the full covariance matrix whenever it is available.
Confusing scales. A log-link contrast is a log mean ratio, and a logit-link contrast is a log odds ratio. Neither is directly a response difference.
Reversing the comparison. Define profile 1 minus profile 0 before calculating. Reversing the profiles changes the sign of a difference, although it inverts a ratio.
Rounding too early. Retain unrounded fitted responses until their difference is calculated, then round the final result to the requested precision.

Key takeaway: To analyze a combined predictor change, subtract the design vectors, form the coefficient contrast, and use the full coefficient covariance matrix for its standard error. Interpret that result on the link scale first; compute a response-scale difference separately using the inverse link at both profiles.

Check Your Understanding

For each problem, define the requested contrast before calculating. Answers are rounded as stated.

A log-link model has coefficients \(\hat\beta_x=0.08\) and \(\hat\beta_g=-0.15\), with no interaction. Compare a profile with \(x=0,g=0\) to one with \(x=2,g=1\), holding the offset fixed. Find the log-rate change and rate ratio. (Answer: log-rate change \(0.01\); rate ratio \(\exp(0.01)\approx1.0101\).)
In a logistic model, two profiles have fitted linear predictors \(-1.0\) and \(-0.4\). Find the fitted probability difference, higher-predictor profile minus lower-predictor profile. (Answer: \(0.4013-0.2689\approx0.1324\), or 13.24 percentage points.)
A contrast has weights \((1,-2)\), coefficient variances \(0.04\) and \(0.01\), and covariance \(-0.005\). Calculate its variance and standard error. (Answer: variance \(0.04+4(0.01)+2(1)(-2)(-0.005)=0.10\); standard error \(\sqrt{0.10}\approx0.3162\).)
In an identity-link model, a combined profile contrast has weights \((2,1)\), coefficient estimates \((0.5,-1)\), variances \(0.04\) and \(0.09\), and covariance \(0.01\). Find the estimated response difference and its standard error. (Answer: estimate \(2(0.5)-1=0\); variance \(4(0.04)+0.09+2(2)(1)(0.01)=0.29\); standard error \(\sqrt{0.29}\approx0.5385\).)
Two profiles in a logistic model have fitted linear predictors \(-0.2\) and \(0.6\). Find their link-scale difference and fitted probability difference, profile with predictor \(0.6\) minus profile with predictor \(-0.2\). (Answer: link-scale difference \(0.8\); probability difference \(0.6457-0.4502\approx0.1955\).)

What You'll Learn