Testing an Individual Partial Regression Coefficient

Why Test One Coefficient?

A multiple-regression model can describe how the mean response changes with several predictors at once. Often the question is whether one predictor contributes a linear association with the response after accounting for the model’s other predictors. Testing an individual partial regression coefficient addresses that question. It does not test every possible contribution of a predictor, and it does not by itself establish a causal effect.

For example, a model might predict a patient’s recovery time from age, baseline severity, and a treatment-group indicator. The coefficient of age describes the modelled change in mean recovery time for a one-year increase in age, holding baseline severity and treatment group fixed. An individual coefficient test asks whether that conditional slope is distinguishable from zero, given the model and its assumptions.

Definition: In a multiple-regression model, an individual partial regression coefficient is the coefficient of one predictor, interpreted conditional on the model’s other included predictors. Testing whether that coefficient is zero assesses whether its modelled linear contribution, conditional on those predictors, is distinguishable from zero.

The Model and the Test

Suppose there are $n$ observations and $p$ predictors, not counting the intercept. Write the model for observation $i$ as

Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\varepsilon_i, \qquad i=1,\ldots,n.

Here $Y_i$ is the response, $x_{ij}$ is observation $i$’s value for predictor $j$, and $\beta_j$ is that predictor’s coefficient. The conditional mean is linear in the included predictors, and the error $\varepsilon_i$ is the difference between the observed response and that conditional mean. The least-squares estimate of $\beta_j$ is written $b_j$.

For a test of predictor $j$, the usual two-sided hypotheses are

H_0:\beta_j=0 \qquad\text{versus}\qquad H_a:\beta_j\ne 0.

The other predictors remain in the model under both hypotheses. Thus, the test is not asking whether predictor $j$ is associated with the response in isolation. It asks whether its coefficient is zero conditional on the other included predictors. A one-sided alternative can be used when a directional question was specified in advance and is scientifically appropriate; it should not be chosen after seeing the estimate.

The usual t procedure requires a suitable linear conditional-mean model, independent errors with mean zero and constant variance, and normally distributed errors for exact small-sample t inference. The predictors must not be perfectly redundant: the model matrix must have full column rank so the coefficients can be estimated separately. These conditions concern the model and data-generating process; they are not guaranteed just because software returns a coefficient table. If observations are sampled from a population, the sampling design must also support the intended generalization. In an observational setting, a conditional association does not become causal merely because a coefficient is statistically significant.

Test statistic: Let $SE(b_j)$ be the estimated standard error of coefficient $b_j$. For testing $H_0:\beta_j=0$, the statistic is $$ t=\frac{b_j}{SE(b_j)},\qquad df=n-p-1. $$ Under the stated regression assumptions and $H_0$, this statistic follows a t distribution with $df$ degrees of freedom.

The standard error measures the estimated sampling variability of $b_j$ under the fitted model. Its value depends on the residual variation and on how much independent information the observed predictor values provide about this coefficient, given the other predictors. The residual degrees of freedom are $n-p-1$: one degree is used for the intercept, and one for each of the $p$ estimated predictor coefficients.

Why the Statistic Has a t Distribution

A short derivation explains why the coefficient estimate is divided by its standard error. Treat the observed predictor values as fixed, and collect them, along with the intercept column, in a matrix $X$. Under the model, the least-squares coefficient vector is

\mathbf{b}=(X^\mathsf{T}X)^{-1}X^\mathsf{T}\mathbf{Y}.

If the errors are independent normal variables with mean zero and common variance $\sigma^2$, then $\mathbf{b}$ is normally distributed around the true coefficient vector. For coefficient $j$, its variance is $\sigma^2c_{jj}$, where $c_{jj}$ is the corresponding diagonal element of $(X^\mathsf{T}X)^{-1}$. The residual sum of squares, divided by $\sigma^2$, has a chi-square distribution with $n-p-1$ degrees of freedom; in this normal linear model it is independent of $\mathbf{b}$.

The unknown variance is estimated by $s^2=\text{SSE}/(n-p-1)$, where SSE is the residual sum of squares. Consequently, $SE(b_j)=s\sqrt{c_{jj}}$. Under the null hypothesis, $b_j/( \sigma\sqrt{c_{jj}})$ is standard normal, while $(n-p-1)s^2/\sigma^2$ is an independent chi-square variable with $n-p-1$ degrees of freedom. Dividing the standard normal variable by the square root of that chi-square variable divided by its degrees of freedom gives the t distribution. Replacing the unknown $\sigma$ with $s$ therefore gives the stated t statistic.

Worked Tests and Intervals

Worked Example: Testing a Conditional Slope

Suppose a hypothetical observational analysis models weekly study hours as a response using commute time and course load as predictors. In a sample of $n=30$ students, the fitted coefficient for commute time is $b_1=-0.80$ hours per additional commute hour, with $SE(b_1)=0.35$ hours per commute hour. Assume the linear model and its inference conditions are reasonable for these data. Test at $\alpha=0.05$ whether the commute-time coefficient is zero conditional on course load.

The hypotheses are $H_0:\beta_1=0$ and $H_a:\beta_1\ne0$. There are $p=2$ predictors, so

$$ df=n-p-1=30-2-1=27. $$

The observed statistic is

t=\frac{-0.80}{0.35}=-2.286.

For a two-sided test with 27 degrees of freedom, the p-value is approximately $0.030$. Since $0.030<0.05$, reject $H_0$ at the 5% significance level. Under the model, the data provide evidence that commute time has a nonzero conditional linear coefficient after accounting for course load. The negative estimate corresponds to 0.80 fewer modelled study hours per week for each additional commute hour, at a fixed course load. This is an adjusted association, not proof that longer commutes cause less study.

The 95% confidence interval uses $t^*=2.052$, the approximate 97.5th percentile of the t distribution with 27 degrees of freedom:

b_1\pm t^*SE(b_1) =-0.80\pm 2.052(0.35) =-0.80\pm0.718 =(-1.518,\,-0.082).

The interval estimates a conditional change between 0.082 and 1.518 fewer study hours per week for each additional commute hour. It excludes zero, in agreement with the two-sided 5% test. In repeated samples under the model, this interval procedure would cover the true conditional coefficient in 95% of samples.

Worked Example: A Nonsignificant Conditional Coefficient

In another hypothetical analysis, a response is modelled from four predictors. For one predictor, the estimated coefficient is $b_2=1.20$ response units per predictor unit and $SE(b_2)=1.00$ response units per predictor unit. The sample size is $n=24$. Assume the model and inference conditions are reasonable. Test $H_0:\beta_2=0$ against a two-sided alternative at $\alpha=0.05$.

There are $p=4$ predictors, so $df=24-4-1=19$. The statistic is

t=\frac{1.20}{1.00}=1.20.

The two-sided p-value with 19 degrees of freedom is approximately $0.245$. Because $0.245>0.05$, do not reject $H_0$. This analysis does not provide strong evidence that the coefficient differs from zero after adjusting for the other three predictors. It does not prove that the coefficient is zero or that the predictor has no relationship with the response.

For a 95% interval, $t^*$ is approximately $2.093$:

1.20\pm2.093(1.00) =1.20\pm2.093 =(-0.893,\;3.293).

The interval includes zero and also includes positive conditional changes as large as about 3.293 response units per predictor unit. The estimate is therefore imprecise enough that these data do not distinguish the coefficient from zero at the 5% two-sided level. The interval gives a range of values compatible with the data and model, not a probability statement about where the fixed coefficient lies.

Worked Example: Testing an Interaction Coefficient

Suppose a hypothetical model predicts plant height from water supplied, a group indicator, and their interaction. The group indicator equals 1 for plants receiving a treatment and 0 for control plants. With $n=40$, the estimated coefficient of the water-by-group interaction is $b_3=0.65$ cm per millilitre, with $SE(b_3)=0.30$ cm per millilitre. Assume the model’s linearity, independence, constant-variance, and normal-error conditions are reasonable. Test whether the interaction coefficient is zero at $\alpha=0.05$.

There are $p=3$ predictor terms, including the interaction, so $df=40-3-1=36$. The hypotheses are $H_0:\beta_3=0$ and $H_a:\beta_3\ne0$. The statistic is

t=\frac{0.65}{0.30}=2.167.

The two-sided p-value with 36 degrees of freedom is approximately $0.037$, which is below $0.05$. Reject $H_0$. In this model, the evidence suggests that the water slope differs between the treatment and control groups. The estimated difference in slopes is 0.65 cm of height per millilitre of water. Because the model contains an interaction, the water coefficient by itself is the slope for the reference group; the interaction coefficient describes the difference from that slope for the group coded 1.

Using $t^*=2.028$ for a 95% interval with 36 degrees of freedom gives

0.65\pm2.028(0.30) =0.65\pm0.608 =(0.042,\;1.258)\text{ cm per millilitre}.

The interval estimates the treatment-minus-control difference in water slopes as between 0.042 and 1.258 cm per millilitre. It excludes zero, consistent with the test. The interpretation is tied to the specified coding and model; changing the reference group changes the coefficient’s representation.

Using the Test and Interval Together

For a two-sided test at significance level $\alpha$, the test rejects $H_0:\beta_j=0$ exactly when the corresponding $100(1-\alpha)\%$ confidence interval excludes zero, subject to using the same model, standard error, and t reference distribution. The test summarizes evidence against the zero value; the interval also communicates the magnitude and precision of plausible coefficient values.

Always interpret a coefficient in the units of both variables. If the response is measured in dollars and a quantitative predictor in years, the coefficient is dollars per year. For a categorical predictor, the coefficient compares the coded category with its reference category while holding other predictors fixed. For an interaction coefficient, the units describe a difference in slopes or another change in a conditional relationship, depending on the interaction’s form.

A small p-value is evidence against the particular zero-coefficient null under the model. It is not the probability that $H_0$ is true, nor does it measure the practical importance of the coefficient. A large p-value is not proof of no conditional association: the standard error may be large, the sample may provide limited information, or the model may not separate the predictors’ contributions precisely. Statistical significance and substantive importance should therefore be considered separately.

Common Mistake

Wrong reasoning: “The p-value for this predictor is above 0.05, so the predictor has no effect and should be removed.” This treats failure to reject as proof of a zero coefficient, and it overlooks that the test is conditional on the other predictors and the specified model.

Correction: State only that the data do not provide sufficient evidence against the zero conditional coefficient at the chosen significance level. Examine the estimate and interval in context, and use subject-matter reasoning to decide how the model should be used. A coefficient test alone does not establish that a predictor is irrelevant in every model or that removing it is appropriate.

Key takeaway: An individual coefficient t test asks whether one modelled coefficient differs from zero while the other included predictors remain in the model. Report the estimate, units, standard error, degrees of freedom, t statistic, p-value, and—when useful—the confidence interval, all interpreted conditionally and in context.

Check Your Understanding

Use the model’s other predictors as fixed in each conditional interpretation.

A model has $n=52$ observations and 6 predictors. What are the residual degrees of freedom for an individual coefficient t test?
A coefficient estimate is $-2.4$, with standard error $0.8$. Calculate the t statistic for testing that the coefficient is zero.
In words, what does the null hypothesis $H_0:\beta_j=0$ mean in a multiple-regression model?
A 95% confidence interval for a coefficient is $(-0.3, 1.7)$. What does inclusion of zero imply about the corresponding two-sided test at the 5% level?
Why does a nonsignificant coefficient test not prove that a predictor has no relationship with the response?