Why Test a Linear Combination?
A regression coefficient often describes only part of the relationship of interest. In a model with an interaction, for example, the slope for one group may be the sum of a baseline slope and an interaction coefficient. A question about that group’s slope is therefore a question about a linear combination of coefficients, not necessarily about any single coefficient.
The same method applies when a benchmark concerns a weighted sum or difference of coefficients. The central tasks are to state the contrast and benchmark precisely, calculate the standard error using the covariance among coefficient estimates, and use the resulting t statistic with the regression’s residual degrees of freedom.
Model, Standard Error, and Test Statistic
Use the multiple linear regression model \(Y=X\boldsymbol{\beta}+\boldsymbol{\varepsilon}\), with \(n\) observations and \(p\) predictors in addition to the intercept. For the usual exact t test, assume the design matrix \(X\) has full column rank and the errors are independent and normally distributed with mean zero and common variance \(\sigma^2\). Thus \(\boldsymbol{\varepsilon}\sim N(\mathbf{0},\sigma^2 I)\). The residual degrees of freedom are \(n-p-1\).
The estimated covariance matrix of the coefficient estimates is \(MSE(X^{\mathsf T}X)^{-1}\), where \(MSE\) estimates the error variance. In practice, the regression output or a supplied covariance matrix may provide the needed entries. For a combination with weights \(c_1,\ldots,c_{p+1}\), its estimated variance is:
The standard error is the square root of this variance. For the null hypothesis \(H_0:L=L_0\), the test statistic is:
Choose the alternative to match the question. For \(H_1:L\ne L_0\), compare \(|t|\) with the two-sided critical value \(t_{1-\alpha/2,df}\), or use a two-sided p-value. For \(H_1:L>L_0\), compare \(t\) with \(t_{1-\alpha,df}\); for \(H_1:L<L_0\), compare \(-t\) with that upper-tail critical value. A rejection is evidence against the specified null value in the direction of the alternative. Failure to reject does not establish that the null value is true.
Why the Variance Formula Includes Covariance
The variance expression follows from the general variance rule for a weighted sum. For two coefficient estimates, for example, \(\operatorname{Var}(a\hat\beta_1+b\hat\beta_2) =a^2\operatorname{Var}(\hat\beta_1)+b^2\operatorname{Var}(\hat\beta_2)+2ab\operatorname{Cov}(\hat\beta_1,\hat\beta_2)\). The cross-term is present because the estimates may move together or in opposite directions across samples.
More generally, applying the matrix variance rule gives \(\operatorname{Var}(\mathbf{c}^{\mathsf T}\hat{\boldsymbol{\beta}}) =\mathbf{c}^{\mathsf T}\operatorname{Cov}(\hat{\boldsymbol{\beta}})\mathbf{c}\). Under the stated normal-error model, the estimated combination, centered at its true value and divided by its standard error, has a t distribution with \(n-p-1\) degrees of freedom. This yields the test statistic above under \(H_0\). If normality or the other model assumptions do not hold, the usual exact finite-sample t reference distribution need not apply.
Testing a One-Sided Contrast
Worked Example: A Weighted Effect Above a Benchmark
Method: Test the specified contrast with a one-sided t test. In a hypothetical regression of annual benefit-administration cost on two predictors, suppose the estimated coefficients are \(\hat\beta_1=0.50\) and \(\hat\beta_2=0.20\), in thousands of dollars per predictor unit. The estimated variances are \(0.01\) and \(0.04\), and their estimated covariance is \(0.002\). Test at \(\alpha=0.05\) whether \(3\beta_1-\beta_2\) exceeds \(1.0\). The model has 42 residual degrees of freedom.
The contrast weights are \(3\) and \(-1\). The hypotheses are \(H_0:3\beta_1-\beta_2=1.0\) and \(H_1:3\beta_1-\beta_2>1.0\).
First calculate the estimated contrast: \(3\hat\beta_1-\hat\beta_2=3(0.50)-0.20=1.50-0.20=1.30\).
Next calculate its estimated variance, retaining the covariance term: \(3^2(0.01)+(-1)^2(0.04)+2(3)(-1)(0.002) =0.09+0.04-0.012=0.118\). Thus \(SE(\hat L)=\sqrt{0.118}\approx0.3435\).
The test statistic is \(t=(1.30-1.00)/0.3435\approx0.8733\), with 42 degrees of freedom. The one-sided 5% critical value is approximately \(1.682\). Since \(0.8733<1.682\), fail to reject \(H_0\). At the 5% level, the data do not provide sufficient evidence that this weighted effect exceeds \(1.0\) thousand dollars per the corresponding weighted predictor unit.
Common mistake avoided: Omitting covariance would give variance \(0.09+0.04=0.130\), rather than \(0.118\). The coefficient weights multiply the covariance by \(2(3)(-1)\), so the covariance contribution is negative here.
Testing a Group-Specific Slope
Worked Example: A Slope Defined by an Interaction
Method: Express the group-specific slope as a linear combination, then use the covariance of its component estimates. Consider a hypothetical model for claim-processing cost, measured in thousands of dollars, with a continuous workload predictor \(x\), a program indicator \(G\), and an \(xG\) interaction. The coefficient of \(x\) is \(\beta_1\), and the interaction coefficient is \(\beta_3\). For the group with \(G=1\), the slope is \(\beta_1+\beta_3\).
Suppose \(\hat\beta_1=0.08\), \(\hat\beta_3=-0.03\), \(\widehat{\operatorname{Var}}(\hat\beta_1)=0.0001\), \(\widehat{\operatorname{Var}}(\hat\beta_3)=0.0004\), and \(\widehat{\operatorname{Cov}}(\hat\beta_1,\hat\beta_3)=-0.00005\). Test whether the group’s slope differs from \(0.04\), using a two-sided 5% test and 27 residual degrees of freedom.
The contrast is \(L=\beta_1+\beta_3\), so \(\hat L=0.08+(-0.03)=0.05\). The hypotheses are \(H_0:\beta_1+\beta_3=0.04\) and \(H_1:\beta_1+\beta_3\ne0.04\).
Its estimated variance is \(0.0001+0.0004+2(1)(1)(-0.00005)=0.0005-0.0001=0.0004\). Therefore \(SE(\hat L)=\sqrt{0.0004}=0.02\), and \(t=(0.05-0.04)/0.02=0.50\).
With 27 degrees of freedom, the two-sided 5% critical value is approximately \(2.052\). Since \(|0.50|<2.052\), fail to reject the null hypothesis. The fitted slope is \(0.05\), but the evidence is insufficient to conclude that the group-specific slope differs from \(0.04\) thousand dollars per workload unit.
Common mistake avoided: Testing \(\beta_1=0.04\) would test the baseline slope, not the slope for \(G=1\). The interaction changes that group’s slope, so the relevant contrast is \(\beta_1+\beta_3\).
A Contrast With Several Coefficients
Worked Example: A Weighted Comparison Across Segments
Method: Apply the full covariance formula to every coefficient pair in the contrast. In a hypothetical regression comparing three service segments, suppose a specified weighted relationship is \(L=\beta_1-\beta_2+2\beta_3\). The coefficient estimates are \(0.40,\ 0.10,\ 0.20\), respectively. Their estimated variances are \(0.04,\ 0.01,\ 0.0025\), and the estimated covariances are \(\widehat{\operatorname{Cov}}(\hat\beta_1,\hat\beta_2)=0.005\), \(\widehat{\operatorname{Cov}}(\hat\beta_1,\hat\beta_3)=0.002\), and \(\widehat{\operatorname{Cov}}(\hat\beta_2,\hat\beta_3)=0.001\). Test \(H_0:L=0.20\) against a two-sided alternative at 5%, with 18 residual degrees of freedom.
The estimated contrast is \(\hat L=0.40-0.10+2(0.20)=0.30+0.40=0.70\). The variance is the sum of the three weighted variances and the three covariance contributions:
Thus \(SE(\hat L)=\sqrt{0.054}\approx0.2324\). The test statistic is \(t=(0.70-0.20)/0.2324\approx2.1517\). For 18 degrees of freedom, the two-sided 5% critical value is approximately \(2.101\). Since \(2.1517>2.101\), reject \(H_0\). The data provide evidence that this specified weighted relationship differs from \(0.20\).
Check: The variance is positive, and the observed contrast is \(0.50\) above the null value. The positive t statistic is therefore consistent with the direction of the observed difference; the two-sided decision uses its absolute value.
Common Traps and Interpretation
A test is only as well specified as its contrast, benchmark, and alternative. Before calculating, identify what quantity the coefficients represent in the modeled relationship. If the quantity is a sum or difference, write it explicitly with its weights; do not substitute a test of one coefficient merely because that coefficient appears in the expression.
- Dropping covariance terms. Coefficient estimates from the same regression are generally not independent. Omitting covariance can overstate or understate the standard error.
- Using the wrong sign. A negative weight affects both the point estimate and each associated covariance term. In particular, the pairwise contribution is \(2c_jc_k\widehat{\operatorname{Cov}}(\hat\beta_j,\hat\beta_k)\).
- Using the wrong degrees of freedom. For this t test, use the residual degrees of freedom \(n-p-1\), not the number of observations or the number of coefficients in the contrast.
- Changing the alternative after seeing the statistic. A one-sided alternative must be selected from the question before using the result. A two-sided test is appropriate when differences in either direction matter.
- Interpreting failure to reject as proof. A nonsignificant result means the data did not provide enough evidence against the stated null at the chosen significance level; it does not prove exact equality.
A test of one specified linear combination is a single-contrast test. It does not, by itself, test several restrictions simultaneously or compare full regression models; those questions require other procedures. For the present test, the essential check is that the contrast, its covariance-based standard error, the null value, and the alternative all describe the same modeled quantity.
Check Your Understanding
For each problem, identify the contrast and use the stated test direction and residual degrees of freedom. Treat the usual regression t-test assumptions as satisfied.
- A regression gives \(\hat\beta_1=0.40\), \(\hat\beta_2=-0.10\), estimated variances \(0.01\) and \(0.04\), and estimated covariance \(0.005\). Test \(H_0:2\beta_1+\beta_2=0.50\) against a two-sided alternative at 5%, with 16 residual degrees of freedom. (Answer: \(\hat L=0.70\), \(SE=\sqrt{0.10}\approx0.3162\), \(t\approx0.6325\); fail to reject.)
- For a group-specific slope \(\beta_1-\beta_2\), suppose estimates are \(0.12\) and \(0.05\), estimated variances are \(0.0009\) and \(0.0016\), and estimated covariance is \(0.0003\). Test whether the slope equals \(0.05\), two-sided at 5%, with 22 residual degrees of freedom. (Answer: \(\hat L=0.07\), \(SE=\sqrt{0.0019}\approx0.0436\), \(t\approx0.4588\); fail to reject.)
- For \(L=2\beta_1-\beta_2\), the estimates are \(0.60\) and \(0.30\), the estimated variances are \(0.04\) and \(0.01\), and their covariance is \(0.005\). Test \(H_0:L=0.70\) against a two-sided alternative at 5%, with 20 residual degrees of freedom. (Answer: \(\hat L=0.90\), \(SE=\sqrt{0.15}\approx0.3873\), \(t\approx0.5164\); fail to reject.)
- A contrast \(2\beta_1+\beta_2\) has estimates \(\hat\beta_1=0.50\), \(\hat\beta_2=-0.10\), variances \(0.01\) and \(0.04\), and covariance \(-0.005\). Test whether the contrast exceeds \(0.40\), at 5%, with 25 residual degrees of freedom. (Answer: \(\hat L=0.90\), \(SE=\sqrt{0.06}\approx0.2449\), \(t\approx2.0412\); reject using the one-sided critical value.)
- For \(L=\beta_1-\beta_2+\beta_3\), the estimates are \(0.40,\ 0.10,\ 0.20\), the estimated variances are \(0.01\) each, and the covariances for pairs \((1,2),(1,3),(2,3)\) are \(0.002,-0.001,0.003\), respectively. Test \(H_0:L=0\) two-sided at 5%, with 14 residual degrees of freedom. (Answer: \(\hat L=0.50\), estimated variance \(0.018\), \(SE\approx0.1342\), \(t\approx3.7268\); reject.)