Standard Errors for Linear Contrasts

Why a Contrast Needs Its Own Standard Error

A regression coefficient’s standard error measures the sampling variability of that one coefficient estimate. Many questions, however, concern a combination of coefficients: the difference between two slopes, a weighted sum of effects, or the slope for one group in an interaction model. The standard error of that combination depends not only on the individual coefficient variances, but also on how the estimates vary together.

This tutorial focuses on deriving and calculating the standard error of a linear contrast. The contrast’s estimate is found by applying fixed weights to coefficient estimates. Its standard error must account for every pairwise covariance among those estimates. This material is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Definition: Let \(\boldsymbol{\beta}=(\beta_1,\ldots,\beta_p)^\mathsf{T}\) be a vector of regression coefficients, and let \(\mathbf{c}=(c_1,\ldots,c_p)^\mathsf{T}\) be a vector of fixed, known constants. The quantity \(L=\mathbf{c}^\mathsf{T}\boldsymbol{\beta}=\sum_{j=1}^p c_j\beta_j\) is a linear contrast. Its estimate is \(\widehat{L}=\mathbf{c}^\mathsf{T}\hat{\boldsymbol{\beta}}\). A contrast is often required to have coefficients that sum to zero when it represents a comparison of effects, but the variance formula applies to any fixed linear combination.

For example, if a model has two slopes \(\beta_1\) and \(\beta_2\), their difference is the contrast \(L=\beta_1-\beta_2\), with weights \(1\) and \(-1\). A weighted combination such as \(2\beta_1-\beta_2\) is also linear. The weights specify the quantity of interest; they do not change from sample to sample.

Variance and Standard Error Formula

Write \(\operatorname{Var}(\hat{\boldsymbol{\beta}})\) for the covariance matrix of the coefficient estimates. The diagonal entries are their variances, and the off-diagonal entries are their covariances. For \(\widehat{L}=\sum_j c_j\hat{\beta}_j\), the variance is

\operatorname{Var}(\widehat{L}) = \operatorname{Var}\!\left(\sum_{j=1}^p c_j\hat{\beta}_j\right) = \sum_{j=1}^p c_j^2\operatorname{Var}(\hat{\beta}_j) + 2\sum_{j<k}c_jc_k\operatorname{Cov}(\hat{\beta}_j,\hat{\beta}_k).

The standard error is the square root of the variance. When the coefficient covariance matrix is estimated, the corresponding estimated standard error is

\widehat{\operatorname{SE}}(\widehat{L}) = \sqrt{\mathbf{c}^\mathsf{T}\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\mathbf{c}}.

The factor of 2 in the expanded formula arises because each distinct pair appears twice in the matrix product: once as the entry in row \(j\), column \(k\), and once as the entry in row \(k\), column \(j\). Since a covariance matrix is symmetric, the two contributions are equal.

Linear-model result: In the model \(\mathbf{Y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\varepsilon}\), assume \(\mathbf{X}\) is fixed and has full column rank, and the errors are independent with mean zero and common variance \(\sigma^2\). Then \(\operatorname{Var}(\hat{\boldsymbol{\beta}})=\sigma^2(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\), so \(\operatorname{Var}(\widehat{L})=\sigma^2\mathbf{c}^\mathsf{T}(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\mathbf{c}\). When \(\sigma^2\) is unknown, replacing it with the residual mean square \(s^2\) gives the estimated covariance matrix \(s^2(\mathbf{X}^\mathsf{T}\mathbf{X})^{-1}\). These formulas rely on the stated error-variance assumptions; if errors are correlated or have unequal variances, the covariance matrix must reflect that structure instead.

A short derivation makes the role of covariance explicit. For any random vector \(\mathbf{Z}\) with covariance matrix \(\boldsymbol{\Sigma}\), each component of \(\mathbf{c}^\mathsf{T}\mathbf{Z}\) is multiplied by a fixed weight. Expanding the variance gives a sum over all ordered pairs:

\operatorname{Var}(\mathbf{c}^\mathsf{T}\mathbf{Z}) = \sum_{j=1}^p\sum_{k=1}^p c_jc_k\operatorname{Cov}(Z_j,Z_k) = \mathbf{c}^\mathsf{T}\boldsymbol{\Sigma}\mathbf{c}.

The terms with \(j=k\) are \(c_j^2\operatorname{Var}(Z_j)\). For \(j\ne k\), the pair \((j,k)\) and its reverse \((k,j)\) contribute equal terms, giving twice each covariance term when the sum is written over \(j<k\). Applying this identity to \(\mathbf{Z}=\hat{\boldsymbol{\beta}}\) yields the contrast formula. The square root is taken only after the full variance, including all covariance terms, has been calculated.

Worked Calculations

Worked Example: A Weighted Combination of Two Slopes

A hypothetical regression estimates two slopes as \(\hat{\beta}_1=1.8\) and \(\hat{\beta}_2=0.6\). The estimated coefficient covariance matrix is \(\begin{pmatrix}0.04&-0.03\\-0.03&0.09\end{pmatrix}\). Find the estimate and estimated standard error of \(L=2\beta_1-\beta_2\). Assume independent normal errors with mean zero and unknown variance \(\sigma^2\); the displayed covariance matrix is the plug-in estimate based on the fitted model. No interest-rate or mortality assumption is involved.

Method. Use the contrast weights \(c_1=2\) and \(c_2=-1\), retaining the covariance term. The estimated contrast is

\widehat{L}=2\hat{\beta}_1-\hat{\beta}_2 =2(1.8)-0.6=3.6-0.6=3.0.

The estimated variance is

\widehat{\operatorname{Var}}(\widehat{L}) = 2^2(0.04)+(-1)^2(0.09) +2(2)(-1)(-0.03) =0.16+0.09+0.12 =0.37.

Therefore,

\widehat{\operatorname{SE}}(\widehat{L}) =\sqrt{0.37} \approx 0.6083.

The covariance is negative, and the contrast weights have opposite signs. Their product is negative, so the covariance contribution is positive. The result is checked by noting that every variance contribution in the expanded expression is nonnegative here. The common mistake is to omit the covariance term or to give it a negative sign merely because the covariance is negative.

Worked Example: The Slope for a Group in an Interaction Model

A hypothetical model uses an indicator \(G\), where \(G=0\) denotes the reference group and \(G=1\) denotes another group: \(E[Y\mid x,G]=\beta_0+\beta_1x+\beta_2G+\beta_3xG\). The estimated slopes are \(\hat{\beta}_1=1.4\) and \(\hat{\beta}_3=0.5\). Their estimated variances are \(0.16\) and \(0.25\), and their estimated covariance is \(0.08\). Find the estimated slope and its standard error for \(G=1\). Assume independent normal errors with mean zero and unknown variance \(\sigma^2\), with the stated quantities estimated from the model. No interest-rate or mortality assumption is involved.

Method. When \(G=1\), the terms involving \(x\) are \(\beta_1x+\beta_3x\), so the slope is \(\beta_1+\beta_3\). The contrast weights for these two coefficients are both \(1\). Its estimate is

\widehat{L}=\hat{\beta}_1+\hat{\beta}_3=1.4+0.5=1.9.

The estimated variance includes twice the covariance because both weights are \(1\):

\widehat{\operatorname{Var}}(\widehat{L}) = 0.16+0.25+2(1)(1)(0.08) =0.41+0.16 =0.57.

Thus the estimated standard error is

\widehat{\operatorname{SE}}(\widehat{L})=\sqrt{0.57}\approx 0.7550.

The positive covariance increases the variance of the sum. The common mistake is to report the standard error of \(\hat{\beta}_1\) alone, or to add the two coefficient standard errors. Neither gives the standard error of their sum. The calculation is checked by using the correct group-1 slope, \(\beta_1+\beta_3\), and then taking the square root of its variance.

Worked Example: A Three-Coefficient Contrast

A hypothetical regression estimates \(\hat{\boldsymbol{\beta}}=(10,2,1)^\mathsf{T}\). Its estimated coefficient covariance matrix is \(\begin{pmatrix}0.25&0.05&-0.02\\0.05&0.16&0.04\\-0.02&0.04&0.09\end{pmatrix}\). Find the estimate and standard error of \(L=\beta_1-2\beta_2+\beta_3\). Assume independent normal errors with mean zero and unknown variance \(\sigma^2\), and treat the displayed matrix as the model-based estimated covariance matrix. No interest-rate or mortality assumption is involved.

Method. The contrast vector is \(\mathbf{c}=(1,-2,1)^\mathsf{T}\). The estimated contrast is

\widehat{L}=1(10)-2(2)+1(1)=10-4+1=7.

Calculate the three diagonal contributions first:

$$ 1^2(0.25)+(-2)^2(0.16)+1^2(0.09) =0.25+0.64+0.09=0.98. $$

Now calculate each covariance contribution separately:

2(1)(-2)(0.05)=-0.20,\qquad 2(1)(1)(-0.02)=-0.04,\qquad 2(-2)(1)(0.04)=-0.16.

The estimated variance and standard error are therefore

\widehat{\operatorname{Var}}(\widehat{L}) =0.98-0.20-0.04-0.16=0.58, \qquad \widehat{\operatorname{SE}}(\widehat{L})=\sqrt{0.58}\approx 0.7616.

The result is checked by summing all three diagonal and all three pairwise contributions exactly once; the variance is positive, as required. The common mistake is to treat every covariance as positive or to forget that the \(-2\) weight changes the signs of two pairwise terms.

Interpretation and Calculation Checks

A standard error is expressed in the same units as the contrast estimate. If all contrast weights are multiplied by a constant \(a\), then the estimate is multiplied by \(a\), the variance by \(a^2\), and the standard error by \(|a|\). This provides a useful scaling check:

\widehat{\operatorname{Var}}(a\widehat{L})=a^2\widehat{\operatorname{Var}}(\widehat{L}), \qquad \widehat{\operatorname{SE}}(a\widehat{L})=|a|\widehat{\operatorname{SE}}(\widehat{L}).

Covariance contributions can be positive or negative because they depend on both the covariance and the product of the contrast weights. A negative contribution does not mean that a variance is negative: the full quadratic form combines all diagonal and off-diagonal terms and must be nonnegative for a valid covariance matrix. Do not take square roots of individual terms.

A reliable calculation follows a fixed sequence: identify the coefficients in the requested quantity, write their contrast weights, calculate the contrast estimate, and then calculate the full variance before taking its square root. If the question supplies coefficient standard errors rather than variances, square each standard error to obtain the diagonal entries. If it supplies correlations, first use \(\operatorname{Cov}(\hat{\beta}_j,\hat{\beta}_k)=\operatorname{Corr}(\hat{\beta}_j,\hat{\beta}_k)\operatorname{SE}(\hat{\beta}_j)\operatorname{SE}(\hat{\beta}_k)\) to obtain the covariance.

Key takeaway: The standard error of a linear contrast is not generally the sum, difference, or root-sum-square of the individual standard errors. Apply the contrast weights to the full coefficient covariance matrix: calculate \(\mathbf{c}^\mathsf{T}\widehat{\operatorname{Var}}(\hat{\boldsymbol{\beta}})\mathbf{c}\), then take its square root.

Check Your Understanding

For each problem, find the requested estimated standard error. Use the stated estimated covariance information and retain all covariance terms.

A hypothetical model gives \(\widehat{\operatorname{Var}}(\hat{\beta}_1)=0.09\), \(\widehat{\operatorname{Var}}(\hat{\beta}_2)=0.04\), and \(\widehat{\operatorname{Cov}}(\hat{\beta}_1,\hat{\beta}_2)=-0.015\). Find the standard error of \(\hat{\beta}_1+\hat{\beta}_2\). (Answer: \(\sqrt{0.10}\approx 0.3162\))
A contrast is \(2\beta_1-\beta_2\). The estimated variances of \(\hat{\beta}_1,\hat{\beta}_2\) are \(0.25,0.16\), and their estimated covariance is \(0.05\). Find the standard error. (Answer: \(\sqrt{0.96}\approx 0.9799\))
In an interaction model, the slope for a particular group is \(\beta_1+\beta_3\). The estimated variances are \(0.16,0.09\), and the estimated covariance is \(0.03\). Find the slope’s standard error. (Answer: \(\sqrt{0.31}\approx 0.5568\))
For \(L=\beta_1-\beta_2+2\beta_3\), the estimated variances are \(0.04,0.09,0.01\), and the estimated covariances are \(\widehat{\operatorname{Cov}}(\hat{\beta}_1,\hat{\beta}_2)=0.01\), \(\widehat{\operatorname{Cov}}(\hat{\beta}_1,\hat{\beta}_3)=-0.005\), and \(\widehat{\operatorname{Cov}}(\hat{\beta}_2,\hat{\beta}_3)=0.015\). Find the standard error. (Answer: \(\sqrt{0.07}\approx 0.2646\))

What You'll Learn