Joint Tests and Uncertain Individual Coefficients

Why Test Predictors Together?

When predictors are correlated, a regression model can estimate their individual coefficients imprecisely. Each coefficient describes the association with the outcome when the other predictors are held fixed; if the predictors tend to vary together, the data may contain limited information about changing one while holding the others constant. Yet the predictors may still have a detectable combined association with the outcome. A joint test addresses that different question by testing whether a set of coefficients is zero all at once.

A joint result and a collection of individual results are therefore not competing answers to the same question. A joint test can reject the hypothesis that all coefficients in a group are zero even though none of the individual coefficient tests rejects its own null hypothesis. The joint test does not, by itself, identify which coefficient is nonzero or establish a causal effect.

The Model and the Joint Hypothesis

Consider a linear regression with an outcome \(Y_i\), two predictors of interest \(X_{i1}\) and \(X_{i2}\), and any other predictors that are kept in both models:

The coefficients \(\beta_1\) and \(\beta_2\) describe the conditional associations of their respective predictors with the mean outcome, holding the other included predictors fixed. To test the two predictors jointly, the null hypothesis is \(H_0:\beta_1=\beta_2=0\). The alternative is that at least one of those two coefficients is not zero. This is a test of two restrictions, so \(q=2\). More generally, a joint test can impose \(q\) coefficient restrictions.

A partial F-test compares two nested models fitted to the same observations. The full model contains the predictors whose coefficients are being tested, along with the predictors to be retained. The reduced model omits the tested predictors but retains the other predictors. Let \(RSS_F\) and \(RSS_R\) be the residual sums of squares for the full and reduced models, respectively. Let \(df\) be the full model’s residual degrees of freedom. The statistic is:

The numerator measures the reduction in residual sum of squares per restriction when moving from the reduced model to the full model. The denominator is the full model’s residual mean square. Under the null hypothesis and the assumptions below, the statistic follows an \(F\) distribution with \(q\) numerator degrees of freedom and \(df\) denominator degrees of freedom.

Assumptions for the Partial F-Test

The test uses the usual linear regression model assumptions for exact small-sample inference: the conditional mean is correctly described by the specified linear model; errors are independent, normally distributed with mean zero and constant variance; and the predictors have no exact linear dependence in the full model. Equivalently, conditional on the observed predictors, the errors satisfy \(\varepsilon_i\sim N(0,\sigma^2)\) independently. The full and reduced models must be nested and must use the same observations. Correlation among predictors is permitted: it is near or exact redundancy—not correlation by itself—that affects what can be estimated.

Normality and constant variance matter for the exact \(F\) reference distribution stated here. Independence and a suitable mean model matter for the regression inference more generally. Before interpreting a test, consider whether the data collection and model make these assumptions reasonable; a calculation does not repair a poorly specified model.

Why the Statistic Has an F Distribution

The comparison works because adding predictors to a nested model cannot increase its residual sum of squares, so \(RSS_R-RSS_F\) is nonnegative. Under \(H_0\), the added predictors do not improve the conditional mean beyond random variation. With normally distributed errors and a full-rank design, the scaled reduction in residual sum of squares, \((RSS_R-RSS_F)/\sigma^2\), has a chi-square distribution with \(q\) degrees of freedom. The scaled residual sum of squares from the full model, \(RSS_F/\sigma^2\), has a chi-square distribution with \(df\) degrees of freedom. These quantities are independent for nested least-squares models under the stated assumptions. Dividing each chi-square quantity by its degrees of freedom gives the partial F-statistic:

The p-value is the probability, assuming \(H_0\) and the model assumptions, of obtaining an F-statistic at least as large as the observed one. It is not the probability that \(H_0\) is true. A small p-value is evidence against the joint null; it does not tell us which coefficient is responsible.

Joint Evidence Can Coexist With Imprecise Individual Tests

An individual coefficient test usually uses a \(t\)-statistic: the estimated coefficient divided by its standard error. When predictors are correlated, the coefficient estimates can have substantial standard errors and can be negatively correlated with each other. A joint test uses the estimated coefficients and their covariance together. If the estimates move in a coordinated direction, the joint evidence can be stronger than either separate \(t\)-test suggests.

The following example gives values that illustrate this pattern. The coefficient covariance matters: a pair of individual standard errors alone is not enough to calculate a joint test. In practice, the partial F-test from the nested-model comparison incorporates that covariance information.

The Comparison Depends on What Is Retained

A partial F-test is conditional on the predictors included in both models. For example, testing two predictors while retaining an adjustment variable asks whether the two add explanatory information beyond that adjustment variable. Omitting the adjustment variable from both models would ask a different question and could produce a different result. The reduced model must represent the particular null hypothesis being tested; it is not simply any simpler model.

Common Mistake: Treating Separate Tests as the Joint Answer

• Wrong reasoning: “Neither individual coefficient has a p-value below 0.05, so the two predictors have no association with the outcome.”
• Correction: The individual tests address separate conditional coefficients. Test the joint null directly with the partial F-test if the question concerns whether the group of coefficients is zero.
• Wrong reasoning: “The joint test is significant, so both predictors are important.”
• Correction: Rejecting \(H_0:\beta_1=\beta_2=0\) means at least one coefficient is nonzero under the model; it does not establish that each is.
• Wrong reasoning: “The joint p-value gives the probability that the null hypothesis is true.”
• Correction: A p-value is calculated under the null and describes the tail probability of a statistic at least as extreme as the observed one. It is not a probability assigned to the null itself.

These distinctions matter when reporting results. State which coefficients were tested together, which predictors remained in both models, the test statistic and degrees of freedom, and the p-value. Then describe the evidence in context without turning an association into a causal claim. Correlated predictors may make individual coefficient estimates unstable or imprecise, even when a group-level test indicates that the predictors add information collectively.