Testing Coefficients and Linear Contrasts

Why Test Coefficients and Contrasts?

A fitted regression equation describes estimated relationships, but an estimate alone does not show how precisely that relationship has been measured. Inference combines an estimate with its standard error to assess whether a coefficient, or a specified combination of coefficients, is compatible with a hypothesized value. These methods are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

A test of one coefficient asks about one model parameter while holding the other predictors in the model fixed. A linear contrast asks about a weighted sum or difference of coefficients. That distinction matters whenever the practical question concerns a combined effect, such as the difference between two slopes, rather than either slope by itself.

Definition: In the multiple linear regression model \(Y=X\beta+\varepsilon\), a linear contrast is \(c^\mathsf{T}\beta\), where \(\beta\) is the vector of regression coefficients and \(c\) is a specified vector of constants. Its estimate is \(c^\mathsf{T}\hat{\beta}\). A single coefficient is a special case: choose \(c\) to select that coefficient.

Model, Assumptions, and the t-Procedure

For exact finite-sample t-inference, assume the linear model is correctly specified and, conditional on the predictor matrix \(X\), the errors are independent with mean zero, common variance \(\sigma^2\), and a normal distribution. Also assume \(X\) has full column rank, so its columns do not create exact linear dependence. Write \(p\) for the number of fitted coefficients, including the intercept, and let \(n\) be the number of observations. The residual degrees of freedom are \(n-p\).

Under these assumptions, the least-squares estimator has covariance matrix \(\sigma^2(X^\mathsf{T}X)^{-1}\). Since \(\sigma^2\) is unknown, it is estimated by the residual mean square \(s^2=\mathrm{SSE}/(n-p)\). The estimated coefficient covariance matrix is therefore \(s^2(X^\mathsf{T}X)^{-1}\). Regression output often provides its diagonal standard errors; a contrast generally requires the off-diagonal covariances as well.

Inference rule: For a null hypothesis \(H_0:c^\mathsf{T}\beta=\theta_0\), calculate \(t=(c^\mathsf{T}\hat{\beta}-\theta_0)/\operatorname{SE}(c^\mathsf{T}\hat{\beta})\). Under the stated assumptions and \(H_0\), this statistic has a t distribution with \(n-p\) degrees of freedom. A two-sided level-\(\alpha\) test rejects when \(|t|>t_{\alpha/2,n-p}\). A corresponding \(100(1-\alpha)\%\) confidence interval is the estimate plus or minus \(t_{\alpha/2,n-p}\) times its standard error.

For a single coefficient \(\beta_j\), the contrast vector selects that coefficient, so its standard error is the usual reported standard error \(SE(\hat{\beta}_j)\). For a general contrast, with estimated coefficient covariance matrix \(\widehat{\operatorname{Cov}}(\hat{\beta})\), the standard error is

\operatorname{SE}(c^\mathsf{T}\hat{\beta}) = \sqrt{c^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})c}.

The covariance terms matter because coefficient estimates from the same fitted model are usually related. Simply adding their standard errors, or treating their estimates as independent, does not generally give the standard error of a sum or difference.

Why the Contrast Formula Works

The least-squares estimator is a vector of random quantities. A fixed linear combination of that vector has variance determined by the covariance matrix, not just by the separate variances. If \(Z=c^\mathsf{T}\hat{\beta}\), then expanding the variance of this sum gives

\operatorname{Var}(Z) = \operatorname{Var}\!\left(\sum_j c_j\hat{\beta}_j\right) = \sum_j c_j^2\operatorname{Var}(\hat{\beta}_j) + 2\sum_{j<k}c_jc_k\operatorname{Cov}(\hat{\beta}_j,\hat{\beta}_k) = c^\mathsf{T}\operatorname{Cov}(\hat{\beta})c.

Replacing the unknown covariance matrix by its estimate gives the estimated variance used for the standard error. Under the normal linear model, the contrast estimate, standardized by its true standard deviation, is normal. The residual variance estimate is independent of that standardized estimate, and \((n-p)s^2/\sigma^2\) has a chi-squared distribution with \(n-p\) degrees of freedom. Dividing the normal quantity by the square root of this independent chi-squared quantity divided by its degrees of freedom gives the stated t distribution. This is why the same t procedure applies to both an individual coefficient and a linear contrast.

Worked Inference Problems

Worked Example: Testing a Conditional Regression Slope

A hypothetical insurer models annual claim-cost change, measured in percentage points, using a multiple linear regression with an intercept and two predictors. For the predictor “risk score,” the estimated coefficient is \(1.8\), its standard error is \(0.6\), and the model has \(n=24\) observations. Test \(H_0:\beta_{\text{risk}}=0\) against a two-sided alternative at the 5% level, and give a 95% confidence interval. Assume the regression conditions stated above hold.

Method. Use the coefficient’s t-statistic with residual degrees of freedom \(n-p\). There are \(p=3\) fitted coefficients, so the degrees of freedom are \(24-3=21\).

The observed statistic is

t=\frac{1.8-0}{0.6}=3.0.

For a two-sided 5% test with 21 degrees of freedom, the critical value is approximately \(t_{0.025,21}=2.080\). Since \(|3.0|>2.080\), reject the null hypothesis at the 5% level. The data provide evidence that the conditional risk-score slope is nonzero.

The 95% confidence interval is

1.8\mathbin{\pm}2.080(0.6) =1.8\mathbin{\pm}1.248 =[0.552,\ 3.048].

The interval excludes zero, consistent with the test decision. The interval is in percentage points of annual claim-cost change per one-unit increase in risk score, conditional on the other predictor. Common mistake avoided: using \(n-1=23\) degrees of freedom ignores the three fitted coefficients; the residual degrees of freedom are \(n-p=21\).

Worked Example: Testing the Difference Between Two Slopes

A hypothetical analyst fits a regression with an intercept and two predictors, \(x_1\) and \(x_2\), to \(n=21\) observations. The estimated slopes are \(\hat{\beta}_1=1.2\) and \(\hat{\beta}_2=0.5\). Their estimated variances are \(0.04\) and \(0.09\), and their estimated covariance is \(0.03\). Test whether the slopes are equal and construct a 95% confidence interval for their difference. Assume the regression conditions hold.

Method. Equality of the slopes is the contrast \(H_0:\beta_1-\beta_2=0\), so \(c=(1,-1)\) for the two slope entries. The estimated difference is \(1.2-0.5=0.7\). Its estimated variance includes the covariance term:

\widehat{\operatorname{Var}}(\hat{\beta}_1-\hat{\beta}_2) =0.04+0.09-2(0.03)=0.07.

Thus the standard error is \(\sqrt{0.07}\approx0.2646\). The model has \(p=3\) coefficients, including the intercept, so \(n-p=21-3=18\) degrees of freedom. The test statistic is

t=\frac{0.7-0}{0.2646}\approx2.645.

The two-sided 5% critical value is approximately \(t_{0.025,18}=2.101\). Because \(2.645>2.101\), reject equality of the two slopes at the 5% level.

The 95% interval is

0.7\mathbin{\pm}2.101(0.2646) \approx 0.7\mathbin{\pm}0.556 =[0.144,\ 1.256].

The positive interval indicates that the first slope exceeds the second under this model. Common mistake avoided: omitting covariance would give variance \(0.04+0.09=0.13\), which is not the variance of the difference when the coefficient estimates have estimated covariance \(0.03\).

Worked Example: Testing a Weighted Combination of Slopes

A hypothetical maintenance-cost model has an intercept and three predictors, fitted to \(n=30\) observations. Two estimated slopes are \(\hat{\beta}_1=0.8\) and \(\hat{\beta}_2=0.3\). Their estimated variances are \(0.04\) and \(0.09\), and their estimated covariance is \(-0.01\). Test \(H_0:\beta_1+2\beta_2=0\) against a two-sided alternative at the 5% level, and form a 95% confidence interval. Assume the regression conditions hold.

Method. The contrast weights are \(1\) and \(2\), so the estimate is

\hat{\beta}_1+2\hat{\beta}_2=0.8+2(0.3)=1.4.

The estimated variance is

$$ 0.04+2^2(0.09)+2(1)(2)(-0.01) =0.04+0.36-0.04 =0.36. $$

Therefore, the standard error is \(\sqrt{0.36}=0.6\). There are \(p=4\) fitted coefficients, so the residual degrees of freedom are \(30-4=26\). The test statistic is

t=\frac{1.4-0}{0.6}=2.333.

The 5% two-sided critical value is approximately \(t_{0.025,26}=2.056\); since \(2.333>2.056\), reject the null hypothesis. The 95% confidence interval is

1.4\mathbin{\pm}2.056(0.6) =1.4\mathbin{\pm}1.234 =[0.166,\ 2.634].

The interval excludes zero, agreeing with the test. Common mistake avoided: the coefficient on the second predictor is doubled in the contrast, so its variance contribution is multiplied by \(2^2\), while the covariance contribution is multiplied by \(2(1)(2)\).

Interpretation and Common Traps

A coefficient test evaluates a particular conditional relationship in the specified model. For example, testing \(H_0:\beta_j=0\) assesses whether the coefficient for predictor \(j\) is zero while the other included predictors remain in the model. It is not automatically a test of whether the predictor has any relationship of any kind with the response, and it is not an overall test of the entire regression.

A contrast should be written from the scientific question before calculating anything. “Are the slopes equal?” translates to \(\beta_1-\beta_2=0\), whereas “is their sum zero?” translates to \(\beta_1+\beta_2=0\). The contrast estimate, its standard error, test statistic, and confidence interval must all use the same weights.

Forgetting the covariance: use \(c^\mathsf{T}\widehat{\operatorname{Cov}}(\hat{\beta})c\), not just a sum of coefficient variances.
Using the wrong degrees of freedom: use \(n-p\), where \(p\) counts all fitted coefficients, including the intercept.
Confusing a test with its alternative: a two-sided test compares \(|t|\) with \(t_{\alpha/2,n-p}\); a one-sided test uses a directional alternative and a one-tail critical value.
Mixing units or scales: a contrast inherits the units implied by its coefficients and weights. Interpret it in the units of the modeled response per the relevant predictor units.
Overstating evidence: failure to reject a null hypothesis does not prove it is true; it means the data do not provide sufficient evidence against it at the chosen significance level.

The exact t distribution relies on the normal-error model in addition to the other stated regression assumptions. If those assumptions are not reasonable, the reported classical standard errors and tests may not be reliable; a t calculation does not repair model misspecification or dependence. For any given two-sided test and matching confidence interval, the decision should agree: the null value is rejected at level \(\alpha\) precisely when it falls outside the \(100(1-\alpha)\%\) interval.

Key takeaway: To test a coefficient or contrast, specify the null value and contrast weights, calculate the estimate, obtain its standard error from the estimated covariance matrix, and use a t distribution with \(n-p\) degrees of freedom. A confidence interval provides the same evidence in an interpretable range of values.

Check Your Understanding

For each problem, use the stated regression assumptions and residual degrees of freedom.

A fitted model has 16 residual degrees of freedom. A coefficient estimate is \(-0.36\) with standard error \(0.12\). Test the coefficient against zero at the two-sided 5% level; use a critical value of \(2.120\). (Answer: \(t=-3.00\); reject \(H_0\) because \(3.00>2.120\).)
Two slopes have estimates \(0.7\) and \(0.2\), estimated variances \(0.04\) and \(0.09\), and estimated covariance \(0.01\). There are 20 residual degrees of freedom. Find the 90% confidence interval for their sum; use \(t_{0.05,20}=1.725\). (Answer: estimate \(0.9\), standard error \(\sqrt{0.15}\approx0.3873\), interval \([0.232,\ 1.568]\).)
Two slopes have estimates \(0.5\) and \(0.1\), estimated variances \(0.09\) and \(0.04\), and estimated covariance \(0.015\). There are 15 residual degrees of freedom. Test the contrast \(2\beta_1-\beta_2=0\) at 5%; use a two-sided critical value of \(2.131\). (Answer: estimate \(0.9\), variance \(0.34\), standard error \(\sqrt{0.34}\approx0.5831\), \(t\approx1.543\); do not reject \(H_0\).)
A coefficient estimate is \(1.25\), its standard error is \(0.40\), and there are 12 residual degrees of freedom. Construct a 95% confidence interval using \(t_{0.025,12}=2.179\), and state whether it excludes zero. (Answer: \(1.25\mathbin{\pm}2.179(0.40)=[0.378,\ 2.122]\); it excludes zero.)
Two slope estimates are \(0.6\) and \(0.3\), with estimated variances \(0.04\) and \(0.01\), and covariance \(-0.005\). There are 24 residual degrees of freedom. Test \(\beta_1+2\beta_2=0\) at 5%; use a two-sided critical value of \(2.064\). (Answer: estimate \(1.2\), variance \(0.06\), standard error \(\sqrt{0.06}\approx0.2449\), \(t\approx4.899\); reject \(H_0\).)

What You'll Learn