Connecting the Overall F-Test to R-Squared

Why Connect the Overall F-Test to R-Squared?

R-squared describes the proportion of observed variation in an outcome accounted for by a fitted regression model with an intercept. The overall F-test asks a different question: is there evidence that the predictors, considered together, are associated with the outcome under the regression model? Connecting these quantities shows how the overall F statistic depends not only on R-squared, but also on the number of predictors and the sample size.

A high R-squared does not by itself establish that every predictor matters, that the model is useful outside the observed data, or that any relationship is causal. The overall F-test is a test of a specific joint null hypothesis about the slopes. Its calculation and interpretation require the model conditions described below.

Model, Hypotheses, and Conditions

Consider an ordinary least-squares regression with an intercept, \(k\) predictors, and \(n\) observations. Write its model as

Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_kx_{ik}+\varepsilon_i,\qquad i=1,\ldots,n.

Here \(\beta_0\) is the intercept, \(\beta_1,\ldots,\beta_k\) are the population slopes, and \(\varepsilon_i\) is the error for observation \(i\). The overall F-test compares the full model with an intercept-only model. Its hypotheses are

H_0:\beta_1=\cdots=\beta_k=0 \qquad\text{versus}\qquad H_a:\text{at least one }\beta_j\ne 0.

The alternative hypothesis says that at least one slope is nonzero. If the test rejects the null at the chosen significance level, the data provide evidence against the joint null hypothesis.

Conditions for the classical overall F-test: The conditional mean follows the stated linear regression model; conditional on the predictor values, the errors have mean zero, are independent, and have a common variance \(\sigma^2>0\). Normal errors give the exact small-sample F reference distribution. The augmented design matrix, containing a column of ones for the intercept and all \(k\) predictor columns, must have full column rank \(k+1\). Also require \(k\ge 1\), \(n>k+1\), and nonconstant observed outcomes so that R-squared is defined.

The full-rank condition means that no predictor column is an exact linear combination of the intercept column and the other predictor columns. It ensures that all \(k+1\) coefficients are separately estimable. The condition \(n>k+1\) gives positive residual degrees of freedom, \(n-k-1\). Without positive residual degrees of freedom, the usual estimate of error variance and the stated F statistic are not defined.

From Sums of Squares to R-Squared

Let \(\hat{Y}_i\) be the fitted value for observation \(i\), \(\bar{Y}\) the sample mean of the outcomes, and \(e_i=Y_i-\hat{Y}_i\) the residual. For a least-squares model with an intercept, define the total, regression, and residual sums of squares by

SST=\sum_{i=1}^{n}(Y_i-\bar{Y})^2,\qquad SSR=\sum_{i=1}^{n}(\hat{Y}_i-\bar{Y})^2,\qquad SSE=\sum_{i=1}^{n}e_i^2.

The intercept ensures that the fitted values and residuals satisfy the usual decomposition \(SST=SSR+SSE\). Thus, when \(SST>0\),

R^2=\frac{SSR}{SST}=1-\frac{SSE}{SST}.

The regression sum of squares has \(k\) degrees of freedom, one for each slope tested jointly. The residual sum of squares has \(n-k-1\) degrees of freedom, after estimating the intercept and \(k\) slopes. The overall F statistic is the regression mean square divided by the residual mean square:

F=\frac{SSR/k}{SSE/(n-k-1)}.

Deriving the R-Squared Formula

The connection follows by replacing each sum of squares in the F statistic with its expression in terms of R-squared. Since \(SSR=R^2SST\), and \(SSE=(1-R^2)SST\), substitution gives

F =\frac{(R^2SST)/k}{((1-R^2)SST)/(n-k-1)} =\frac{R^2/k}{(1-R^2)/(n-k-1)}.

The common factor \(SST\) cancels. This is why the statistic can be calculated from R-squared, \(n\), and \(k\), without knowing the outcome's measurement scale.

Result: For a least-squares regression with an intercept, \(k\ge1\) predictors, \(n>k+1\), a full-column-rank augmented design matrix, and nonconstant outcomes, the overall F statistic is

F=\frac{R^2/k}{(1-R^2)/(n-k-1)} =\frac{(n-k-1)R^2}{k(1-R^2)},

provided \(R^2<1\). Under \(H_0\) and the classical error assumptions, this statistic has an \(F\) distribution with \(k\) numerator and \(n-k-1\) denominator degrees of freedom.

The usual finite statistic requires \(SSE>0\), equivalently \(R^2<1\). A perfect fit has zero residual sum of squares, so the usual ratio has a zero denominator; it is not a finite F statistic. Under the classical model with positive error variance, a perfect fit has probability zero when residual degrees of freedom are positive.

To test the overall null hypothesis, compare the observed \(F\) with the \(F\) distribution having \(df_1=k\) and \(df_2=n-k-1\). The p-value is the probability, assuming \(H_0\) and the model conditions, of obtaining an F statistic at least as large as the observed one. Small p-values provide evidence against the joint null; they do not give the probability that the null hypothesis is true.

Worked Example: Calculating the Overall F Statistic

Suppose a hypothetical study models an outcome measured in points using two predictors and an intercept. There are \(n=20\) observations, and the fitted model has \(R^2=0.40\). Assume the augmented design matrix has full column rank, the outcomes are not constant, and the classical error conditions are reasonable for the intended test. Calculate the overall F statistic and interpret it.

There are \(k=2\) slopes, so the residual degrees of freedom are \(20-2-1=17\). Substitution gives

F=\frac{0.40/2}{(1-0.40)/17} =\frac{0.20}{0.60/17} =\frac{0.20}{0.03529} \approx 5.667.

The reference distribution under \(H_0\) is \(F_{2,17}\). Its upper-tail p-value is approximately \(0.0130\). At significance level \(\alpha=0.05\), this is evidence against the hypothesis that both population slopes are zero. It supports an association between the predictors considered jointly and the outcome under the model. It does not establish that both predictors have nonzero slopes or that either predictor causes changes in the outcome.

How Sample Size and Predictor Count Matter

For fixed \(k\), the expression \(F=(n-k-1)R^2/[k(1-R^2)]\) increases with R-squared when \(0\le R^2<1\). But R-squared alone does not determine the statistic: the sample size changes the residual degrees of freedom, and the number of predictors affects both degrees of freedom. As a result, equal R-squared values need not give equal overall F statistics.

Worked Example: The Same R-Squared With Different Sample Sizes

Imagine two hypothetical fitted models, each with two predictors and \(R^2=0.25\). One uses \(n=20\) observations; the other uses \(n=80\). Assume both augmented design matrices have full column rank, both outcomes vary, and the test conditions are appropriate.

For the first model, the residual degrees of freedom are \(20-2-1=17\):

F=\frac{0.25/2}{0.75/17} =\frac{0.125}{0.04412} \approx 2.833.

For the second, the residual degrees of freedom are \(80-2-1=77\):

F=\frac{0.25/2}{0.75/77} =\frac{0.125}{0.009740} \approx 12.83.

Both models account for 25% of the observed variation, but the second has a much larger F statistic because its residual mean square divides by more residual degrees of freedom while the explained proportion is held fixed. Their p-values also depend on their respective \(F_{2,17}\) and \(F_{2,77}\) reference distributions. This comparison illustrates why R-squared and the overall test answer related but different questions.

Worked Example: Recovering R-Squared From an F Statistic

A hypothetical regression output reports an overall F statistic of \(4.50\) from a model with \(n=40\) observations and \(k=3\) predictors. Assume the model has an intercept, the augmented design matrix has full column rank, and the other conditions for the test hold. What R-squared corresponds to this F statistic?

Starting with \(F=(n-k-1)R^2/[k(1-R^2)]\), multiply both sides by \(k(1-R^2)\), then collect the terms involving \(R^2\):

$$ Fk(1-R^2)=(n-k-1)R^2, $$

Fk=R^2\{n-k-1+Fk\}, \qquad R^2=\frac{Fk}{n-k-1+Fk}.

Here \(n-k-1=40-3-1=36\), so

R^2=\frac{4.50(3)}{36+4.50(3)} =\frac{13.5}{49.5} \approx 0.2727.

The reported statistic corresponds to an R-squared of about \(0.273\), or 27.3% of the observed variation accounted for by this fitted model. The statistic's significance is assessed against \(F_{3,36}\); R-squared itself is not a p-value and does not identify which predictor contributes to the result.

What the Overall Test Does—and Does Not—Tell You

The overall F-test is useful when the scientific question concerns whether a collection of slopes is jointly zero. It compresses that joint comparison into one statistic. A significant result says that the observed fit improvement over the intercept-only model is large relative to the residual variation, judged against the reference distribution and assumptions. It does not show that every predictor is individually significant, rank predictors by importance, or establish that the model predicts accurately for new observations.

The size of R-squared is a measure of in-sample fit, while the p-value measures how incompatible the observed F statistic is with the joint null under the model assumptions. A small p-value can occur with a modest R-squared in a sufficiently large sample. Conversely, a seemingly substantial R-squared may not yield strong evidence in a small sample with several predictors. Statistical evidence and practical usefulness are separate considerations.

Remember that this derivation depends on a model with an intercept and the stated rank and degrees-of-freedom conditions. R-squared without an intercept is defined differently, so this formula should not be transferred to a no-intercept fit. Likewise, the statistic tests all slopes together; it is not a test of a selected subset of predictors.

Common Mistake

Wrong reasoning: “The model has \(R^2=0.25\), so its overall F statistic and p-value are determined.”
Correction: The formula also requires \(n\) and \(k\): \(F=(n-k-1)R^2/[k(1-R^2)]\). The reference distribution also uses both degrees of freedom. Report the sample size and predictor count alongside R-squared when describing this test.
Another mistake: “A significant overall F-test proves that each predictor is associated with the outcome.”
Correction: It rejects only the joint hypothesis that all slopes are zero. It does not, by itself, determine which slopes differ from zero.

Key takeaway: With an intercept, valid residual degrees of freedom, and a non-perfect fit (\(R^2<1\), equivalently \(SSE>0\)), the overall F statistic is a rescaled form of R-squared: \(F=(n-k-1)R^2/[k(1-R^2)]\). Interpret it as a joint test of all slopes, conditional on the regression assumptions—not as a complete measure of model usefulness.

Check Your Understanding

Answer each question using the formula and conditions developed above.

A regression with an intercept has \(n=25\), \(k=4\), and \(R^2=0.32\). What are its numerator and denominator degrees of freedom, and what is its overall F statistic?
Why must the augmented design matrix include the intercept column when checking full column rank?
State the overall F-test hypotheses in words and explain what a significant result does not establish.
Two models have the same R-squared and the same number of predictors but different sample sizes. Why can their overall F statistics differ?
What condition on \(n\) ensures positive residual degrees of freedom, and what goes wrong with the usual F calculation if that condition fails?