Linear Combinations of Regression Coefficients

Why Combine Regression Coefficients?

A regression question does not always concern one coefficient by itself. In a model with an interaction, for example, the slope for one group may be the sum of a reference-group slope and an interaction coefficient. A difference between two group slopes may instead be a difference between interaction coefficients. Being able to express these quantities precisely is essential: the combination determines the model parameter being estimated or tested.

This tutorial focuses on identifying and interpreting those combinations. It does not calculate their standard errors or carry out their confidence intervals and tests; those require additional inference steps. The first task is to specify the target parameter correctly.

Definition: Let $\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)^\mathsf{T}$ be the regression coefficient vector, and let $\mathbf{c}=(c_1,\ldots,c_p)^\mathsf{T}$ be a specified vector of constants. A linear combination of the coefficients, also called a linear contrast in regression inference, is

\theta=\mathbf{c}^\mathsf{T}\boldsymbol{\beta} =\sum_{j=1}^{p}c_j\beta_j.

Its estimator is the same combination of the estimated coefficients:

\hat{\theta}=\mathbf{c}^\mathsf{T}\hat{\boldsymbol{\beta}} =\sum_{j=1}^{p}c_j\hat{\beta}_j.

The entries of $\mathbf{c}$ specify which coefficients are included and with what weights. A zero means that coefficient is not part of the combination. The order of the entries must match the stated order of $\boldsymbol{\beta}$. In classical comparisons of means, the word “contrast” is sometimes reserved for weights that sum to zero. In regression, linear contrast commonly refers more broadly to a linear function of regression coefficients; the intended parameter should always be stated explicitly.

Model Setup and a Justification

Use the multiple linear regression model $\mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}$, where $\mathbf{X}$ is a fixed design matrix with full column rank. For the usual exact coefficient-inference framework, assume the errors are independent and normally distributed with mean zero and common variance $\sigma^2$. The least-squares estimator is $\hat{\boldsymbol{\beta}}=(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{X}^\mathsf{T}\mathbf{Y}$. The linearity of the estimator gives the simple rule for estimating a contrast.

Start with the fitted coefficient vector.
By least squares, $\hat{\boldsymbol{\beta}}=(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{X}^\mathsf{T}\mathbf{Y}$.

Apply the contrast weights.
Multiplying on the left by $\mathbf{c}^\mathsf{T}$ gives $\mathbf{c}^\mathsf{T}\hat{\boldsymbol{\beta}}=\mathbf{c}^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{X}^\mathsf{T}\mathbf{Y}$. Thus the estimate is the same weighted sum of the fitted coefficients.

Check the target under the model assumptions.
Since $E[\mathbf{Y}\mid\mathbf{X}]=\mathbf{X}\boldsymbol{\beta}$, the least-squares estimator satisfies $E[\hat{\boldsymbol{\beta}}\mid\mathbf{X}]=\boldsymbol{\beta}$. Therefore $E[\mathbf{c}^\mathsf{T}\hat{\boldsymbol{\beta}}\mid\mathbf{X}]=\mathbf{c}^\mathsf{T}\boldsymbol{\beta}$. The contrast estimator is unbiased under the stated model.

Normality is needed for exact small-sample inference, not for the algebraic identity $\hat{\theta}=\mathbf{c}^\mathsf{T}\hat{\boldsymbol{\beta}}$. The unbiasedness argument uses the zero conditional mean of the errors and the full-rank design. If the design is rank deficient, the separate coefficients may not be uniquely estimable, so a proposed combination must be checked for estimability rather than treated automatically as an ordinary fitted contrast.

Contrasts in Models with Interactions

An interaction makes a coefficient combination especially important. Consider a binary indicator $G$, coded 0 for a reference group and 1 for another group, and a quantitative predictor $x$:

E[Y\mid x,G]=\beta_0+\beta_1x+\beta_2G+\beta_3xG.

For $G=0$, the mean response is $\beta_0+\beta_1x$, so its slope in $x$ is $\beta_1$. For $G=1$, the mean response is $(\beta_0+\beta_2)+(\beta_1+\beta_3)x$, so its slope is $\beta_1+\beta_3$. The slope difference, group 1 minus group 0, is therefore $\beta_3$. The intercept difference at $x=0$ is $\beta_2$. These are different targets, even though they come from the same fitted model.

Worked Example: Estimating the Slope for a Coded Group

A hypothetical insurer models annual claim cost $Y$, in thousands of dollars, using a risk score $x$ and an indicator $G$ for a particular contract class. The fitted model is $E[Y\mid x,G]=\beta_0+\beta_1x+\beta_2G+\beta_3xG$. The coefficient order is $(\beta_0,\beta_1,\beta_2,\beta_3)$, and the fitted values are $(4.10,0.72,0.35,-0.18)$. Assume the regression errors are independent $N(0,1)$, with variance 1 in squared thousands of dollars, and the design matrix has full column rank. Find the estimated slope for $G=1$, and state a null value that could be tested for that slope.

Method. Substitute $G=1$ into the model. The coefficient of $x$ becomes $\beta_1+\beta_3$, so the target parameter is $\theta=\beta_1+\beta_3$.

The contrast vector in the stated coefficient order is $\mathbf{c}=(0,1,0,1)^\mathsf{T}$. Its entries select $\beta_1$ and $\beta_3$, and exclude the intercept and group-level shift.

Calculate the estimated contrast:

\hat{\theta} =0(4.10)+1(0.72)+0(0.35)+1(-0.18) =0.72-0.18 =0.54.

Thus the estimated slope for $G=1$ is $0.54$ thousand dollars per risk-score unit. A hypothesis about whether that slope equals $0.50$, for example, would concern $H_0:\beta_1+\beta_3=0.50$, not $H_0:\beta_3=0.50$. The latter concerns the difference in slopes between the two groups. Check: the reference-group slope is $0.72$, and adding the interaction contribution $-0.18$ gives $0.54$, as required. The common mistake avoided here is treating the interaction coefficient alone as the group-1 slope.

Contrasts for Differences at a Specified Predictor Value

A group difference in expected response can depend on $x$. Under the interaction model above, subtracting the $G=0$ mean from the $G=1$ mean gives

\{\beta_0+\beta_1x+\beta_2+\beta_3x\} -\{\beta_0+\beta_1x\} =\beta_2+x\beta_3.

Consequently, the group difference at a specific value $x=x_0$ is the contrast $\beta_2+x_0\beta_3$. It is not generally just $\beta_2$; that coefficient is the group difference specifically at $x=0$. If $x$ has been centered, zero corresponds to the centering point, not necessarily to a physically meaningful zero.

Worked Example: Comparing Expected Costs at a Given Risk Score

In a hypothetical portfolio analysis, annual expected claim cost $Y$ is measured in thousands of dollars, $G=1$ denotes a selected contract segment, and $x$ is the risk score measured in ten-point units. The fitted model is $E[Y\mid x,G]=\beta_0+\beta_1x+\beta_2G+\beta_3xG$, with coefficient order $(\beta_0,\beta_1,\beta_2,\beta_3)$ and estimates $(3.2,0.4,1.6,-0.25)$. Assume independent regression errors $N(0,9)$, with variance 9 in squared thousands of dollars, and a full-rank design. Find the estimated difference in expected cost, segment $G=1$ minus segment $G=0$, at $x=4$.

Method. The group difference at a general $x$ is $\beta_2+x\beta_3$. At $x=4$, the target is $\theta=\beta_2+4\beta_3$, with contrast vector $\mathbf{c}=(0,0,1,4)^\mathsf{T}$.

Apply the contrast to the estimates:

\hat{\theta} =0(3.2)+0(0.4)+1(1.6)+4(-0.25) =1.6-1.0 =0.6.

The estimated difference is $0.6$ thousand dollars, or $600, with segment $G=1$ having the higher expected cost at that score. Check: the fitted mean for $G=0$ at $x=4$ is $3.2+0.4(4)=4.8$. For $G=1$, it is $3.2+0.4(4)+1.6-0.25(4)=5.4$. Their difference is $5.4-4.8=0.6$ thousand dollars. The common mistake avoided is using $\beta_2$ alone and overlooking that the interaction changes the group difference away from $x=0$.

Contrasts Between More Than Two Groups

With several indicator variables, first write the fitted slope or mean for each group, then subtract in the order asked. For example, if group A is the reference and groups B and C have separate indicators and interactions with $x$, the coefficients for the group-specific slopes can be written directly. Subtracting those slopes cancels any shared reference slope. The signs depend on the requested order: “B minus C” is the negative of “C minus B.”

Worked Example: Difference Between Two Nonreference-Group Slopes

A hypothetical model relates a policy’s annual administrative expense $Y$, in dollars, to a workload measure $x$. Group A is the reference, and $I_B$ and $I_C$ indicate groups B and C. The model is $E[Y\mid x,I_B,I_C]=\beta_0+\beta_1x+\beta_2I_B+\beta_3I_C+\beta_4xI_B+\beta_5xI_C$. The coefficient order is $(\beta_0,\beta_1,\beta_2,\beta_3,\beta_4,\beta_5)$, with estimates $(120,0.12,15,-8,0.31,-0.07)$. Assume independent errors $N(0,25)$, with variance 25 dollars squared, and a full-rank design. Find the estimated slope for B minus the estimated slope for C.

Method. Group B has slope $\beta_1+\beta_4$; group C has slope $\beta_1+\beta_5$. Subtracting in the requested order gives

(\beta_1+\beta_4)-(\beta_1+\beta_5)=\beta_4-\beta_5.

The contrast vector is $\mathbf{c}=(0,0,0,0,1,-1)^\mathsf{T}$. The estimated difference is

\hat{\theta} =0(120)+0(0.12)+0(15)+0(-8)+1(0.31)-1(-0.07) =0.31+0.07 =0.38.

The estimated B slope exceeds the C slope by $0.38 per workload unit. As a check, the estimated B slope is $0.12+0.31=0.43$, and the estimated C slope is $0.12-0.07=0.05$; their difference is $0.43-0.05=0.38$. The common mistake avoided is reversing the requested subtraction or treating $-\hat{\beta}_5$ as $-0.07$: subtracting a negative estimate adds $0.07$.

Common Traps and a Reliable Procedure

A contrast is defined by both its coefficients and its direction. Before calculating it, identify the requested quantity in words: a group-specific slope, a difference in slopes, a group difference at a particular predictor value, or another specified model quantity. Then derive its formula from the model rather than guessing from coefficient labels.

For example, if the requested quantity is $2\beta_1-\beta_3$, and the estimates are $\hat{\beta}_1=0.6$ and $\hat{\beta}_3=-0.2$, the correct calculation is

$$ 2(0.6)-(-0.2)=1.2+0.2=1.4. $$

The tempting but incorrect calculation $1.2-0.2=1.0$ has treated the estimated coefficient $-0.2$ as though it were $+0.2$. The minus sign in the contrast means subtract the coefficient, including its sign. A useful final check is to expand the contrast in the original model or compute the relevant fitted quantities separately, as in the worked examples.

Keep coefficient order fixed. Write the order of $\boldsymbol{\beta}$ before writing $\mathbf{c}$.
Respect the requested direction. “Group A minus group B” reverses the sign of “group B minus group A.”
Include the predictor value when needed. With an interaction, a group difference may be $\beta_2+x_0\beta_3$, not simply $\beta_2$.
Name the parameter, not only the arithmetic. State whether the result is a slope, a slope difference, or a response difference and give its units.
Separate the target from its inference. A contrast identifies what is being estimated or tested; its standard error and inferential conclusion are separate calculations.

Key takeaway: Translate the requested model quantity into a formula in the regression coefficients, form the matching contrast vector in the correct coefficient order, and apply those same weights to the estimated coefficients. The contrast—not any single coefficient in isolation—defines the target parameter.

Check Your Understanding

For each hypothetical regression, identify the target contrast and calculate its estimate. Coefficients are listed in the order specified in each question.

A pricing model has estimates $\hat{\beta}_2=-0.40$ and $\hat{\beta}_3=0.15$. What is the estimate of $\beta_2+\beta_3$, and what contrast weights apply to these two coefficients? (Answer: contrast $(1,1)$; estimate $-0.25$.)
A fitted expense model has $\hat{\beta}_1=0.60$ and $\hat{\beta}_3=-0.20$. Find the estimate of $2\beta_1-\beta_3$, keeping track of the sign of the second estimate. (Answer: contrast $(2,-1)$; estimate $1.40$.)
In an interaction model with reference-group slope $\beta_1$ and group interaction $\beta_3$, the estimates are $\hat{\beta}_1=0.45$ and $\hat{\beta}_3=-0.12$. What is the estimated slope for the indicator group? (Answer: contrast $\beta_1+\beta_3$, with weights $(1,1)$; estimate $0.33$.)
A two-group model gives estimated group shift $\hat{\beta}_2=0.80$ and interaction $\hat{\beta}_3=-0.10$. Find the estimated difference, group 1 minus group 0, at $x=3$. (Answer: contrast $\beta_2+3\beta_3$, with weights $(1,3)$; estimate $0.50$.)
With group A as reference, suppose the fitted slopes for B and C are $\beta_1+\beta_4$ and $\beta_1+\beta_5$. If $\hat{\beta}_4=0.22$ and $\hat{\beta}_5=0.05$, find the estimated slope difference B minus C and state the contrast. (Answer: contrast $\beta_4-\beta_5$, with weights $(1,-1)$; estimate $0.17$.)

What You'll Learn