Tutorials › Exam SRM - Statistics for Risk Modeling › The Relationship Between t Tests and F Tests

Home  /  Tutorials  /  Exam SRM / Inference and ANOVA

Exam SRM · Inference and ANOVA · Tutorial 72 of 926

The Relationship Between t Tests and F Tests

Connect individual t tests with partial F tests, and distinguish the single-restriction identity from joint tests of several restrictions.

Intermediate 12 min read

What You'll Learn

  • State the classical regression assumptions that support exact t and F tests.
  • Express a coefficient test as a single linear restriction.
  • Derive why its partial F statistic equals the squared t statistic.
  • Translate between two-sided t-test and one-restriction F-test conclusions.
  • Explain why a multiple-restriction F statistic is not one coefficient’s t statistic squared.
  • Identify the direction information lost when a signed t statistic is squared.

Why t Tests and F Tests Are Closely Related

Regression inference often presents two different-looking statistics for testing restrictions on coefficients. A t statistic is signed and is commonly used for one coefficient or one linear contrast. An F statistic is nonnegative and is used for joint restrictions. For a single restriction, the two statistics contain the same evidence: the partial F statistic is the square of the corresponding t statistic. The squaring removes the sign, so the equivalence is between an F test and a two-sided t test.

The distinction matters when interpreting output. A large absolute t statistic supports evidence against a single null restriction; its square is the corresponding F statistic. But if several restrictions are tested together, the F statistic summarizes their joint evidence and is not, in general, the square of any one coefficient’s t statistic.

Model and assumptions: Consider the linear regression model \(Y=X\beta+\varepsilon\), where \(Y\) is an \(n\)-vector of responses, \(X\) is a fixed, full-column-rank design matrix, and \(\beta\) is the coefficient vector. For the exact finite-sample t and F tests described here, assume the errors are independent and normally distributed with mean zero and common, unknown variance \(\sigma^2\): \(\varepsilon\sim N(0,\sigma^2 I)\). The tests use the full-model residual mean square to estimate \(\sigma^2\). If these distributional assumptions do not hold, the exact finite-sample reference distributions need not apply.

Write \(\hat{\beta}\) for the least-squares estimate, \(SSE_U\) for the residual sum of squares in the unrestricted (full) model, and \(\nu=n-\operatorname{rank}(X)\) for its residual degrees of freedom. Then \(s^2=SSE_U/\nu\) estimates \(\sigma^2\). For a single linear restriction \(c'\beta=c_0\), the null hypothesis is \(H_0:c'\beta=c_0\); the vector \(c\) specifies the coefficient or linear combination being tested. Its estimated standard error is \(SE(c'\hat{\beta})\).

Definition: The t statistic for the single restriction \(H_0:c'\beta=c_0\) is $$ t=\frac{c'\hat{\beta}-c_0}{SE(c'\hat{\beta})}. $$ The partial F statistic compares the restricted model, which imposes the null restriction, with the unrestricted model. For one restriction, $$ F=\frac{SSE_R-SSE_U}{SSE_U/\nu}, $$ where \(SSE_R\) is the restricted model’s residual sum of squares. Under \(H_0\), \(t\) has a \(t\) distribution with \(\nu\) degrees of freedom, and \(F\) has an \(F\) distribution with \(1\) and \(\nu\) degrees of freedom.

Why the Single-Restriction F Statistic Equals \(t^2\)

The identity follows by expressing the improvement in fit from removing the restriction in terms of the estimated contrast. Under the stated model, the covariance matrix of \(\hat{\beta}\) is estimated by \(s^2(X'X)^{-1}\). Therefore the estimated variance of \(c'\hat{\beta}\) is \(s^2c'(X'X)^{-1}c\), and its standard error is the square root of that quantity.

For one linear restriction, the increase in residual sum of squares caused by imposing \(c'\beta=c_0\) is the squared distance between the estimated contrast and its null value, divided by the estimated variance factor without \(s^2\). Substituting that increase into the partial F statistic gives the identity directly:

$$ SSE_R-SSE_U = \frac{(c'\hat{\beta}-c_0)^2}{c'(X'X)^{-1}c}, \qquad F = \frac{(c'\hat{\beta}-c_0)^2}{s^2c'(X'X)^{-1}c} = \left( \frac{c'\hat{\beta}-c_0}{SE(c'\hat{\beta})} \right)^2 =t^2. $$

Thus, for the same single restriction and the same full model, the partial F test and the two-sided t test have identical evidence. If the observed t statistic is \(t_{\text{obs}}\), then \(F_{\text{obs}}=t_{\text{obs}}^2\). In particular, the right-tail F probability equals the two-tail t probability:

$$ P(F_{1,\nu}\ge t_{\text{obs}}^2) = P(|T_\nu|\ge |t_{\text{obs}}|). $$

The tests also have matching critical values: a two-sided t test at significance level \(\alpha\) rejects when \(|t|\) exceeds the \(1-\alpha/2\) quantile of the \(t_\nu\) distribution. The equivalent F test rejects when \(F\) exceeds the \(1-\alpha\) quantile of the \(F_{1,\nu}\) distribution. Squaring the t critical value gives the F critical value in this one-restriction case.

Worked Example: A Single Slope Restriction

A hypothetical regression model has \(\nu=18\) residual degrees of freedom. For a slope coefficient, the estimate is \(1.8\), its standard error is \(0.3\), and the null value is \(0.6\). The full model has \(SSE_U=40.5\), while the model imposing the null has \(SSE_R=76.5\). Verify the t–F relationship. Use the classical normal-error model with unknown \(\sigma^2\).

Method. Calculate the signed t statistic from the coefficient estimate and its standard error, then calculate the partial F statistic from the two residual sums of squares. The common mistake to avoid is comparing a signed t statistic directly with a nonnegative F statistic rather than squaring t.

First calculate the coefficient test:

$$ t=\frac{1.8-0.6}{0.3} =\frac{1.2}{0.3}=4. $$

Next, the full-model residual mean square is \(SSE_U/\nu=40.5/18=2.25\). The restricted model increases the residual sum of squares by \(76.5-40.5=36\). Hence

$$ F=\frac{76.5-40.5}{40.5/18} =\frac{36}{2.25}=16. $$

Finally, \(t^2=4^2=16=F\). The positive sign of \(t\) indicates that the estimate exceeds the null value; the F statistic retains the strength of the discrepancy but not its direction. Check: both statistics test the same single restriction, and the calculated values satisfy the required identity.

Using the Identity for a Linear Contrast

The single restriction need not concern just one coefficient. For example, \(H_0:\beta_1-\beta_2=0\) is one restriction, even though it involves two coefficients. The t test uses the standard error of the entire contrast, including the covariance between the coefficient estimates. The equivalent partial F test must impose that same contrast—not test each coefficient separately.

Worked Example: Comparing Two Coefficients

In a hypothetical regression with \(\nu=24\), estimates for two effects are \(\hat{\beta}_1=1.4\) and \(\hat{\beta}_2=0.5\). Their estimated variances are \(0.09\) and \(0.16\), and their estimated covariance is \(0.045\). Test \(H_0:\beta_1-\beta_2=0\) using the t–F relationship. Assume the classical normal-error model with unknown \(\sigma^2\).

Method. Treat the difference as one linear contrast, calculate its variance using both variances and the covariance, and then square its t statistic. A common mistake is to omit the covariance term, which changes the contrast’s standard error.

The estimated contrast is \(1.4-0.5=0.9\). Its estimated variance is

$$ \widehat{\operatorname{Var}}(\hat{\beta}_1-\hat{\beta}_2) = 0.09+0.16-2(0.045) = 0.25-0.09 = 0.16. $$

Thus its standard error is \(\sqrt{0.16}=0.4\), and

$$ t=\frac{0.9-0}{0.4}=2.25, \qquad F=t^2=(2.25)^2=5.0625. $$

The F test imposes the single equality \(\beta_1-\beta_2=0\), so its numerator degrees of freedom are one and its denominator degrees of freedom are 24. Check: the contrast’s covariance contribution is included, and the single-restriction F statistic equals the square of its contrast t statistic.

What Changes for Multiple Restrictions?

A joint hypothesis may impose \(q\) restrictions at once, such as \(H_0:R\beta=r\), where \(R\) has \(q\) independent rows. Its partial F statistic is

$$ F = \frac{(SSE_R-SSE_U)/q}{SSE_U/\nu} = \frac{(R\hat{\beta}-r)' \left[R\,s^2(X'X)^{-1}R'\right]^{-1} (R\hat{\beta}-r)}{q}. $$

This statistic combines the departures from all \(q\) restrictions, accounting for their estimated variances and covariances. It has an \(F_{q,\nu}\) distribution under the joint null and the classical assumptions. When \(q=1\), the quadratic expression reduces to the square of a single standardized contrast, recovering \(F=t^2\). When \(q>1\), there is no single signed t statistic whose square generally gives the joint F statistic.

This distinction prevents a frequent misinterpretation: a joint F test is not obtained by selecting one coefficient’s t statistic, and separate individual tests do not automatically replace a joint test. A joint test asks whether all stated restrictions hold together. Its result can differ from the conclusions of separate tests because it incorporates the joint pattern and covariance of the estimates.

Worked Example: Two Restrictions Are Not One Squared t Test

A hypothetical full regression has \(\nu=20\). Two coefficients are estimated as \(3\) and \(1\), each with standard error \(1\), and their estimated covariance is zero. Test the joint null \(H_0:\beta_1=0,\ \beta_2=0\). Assume the classical normal-error model with unknown \(\sigma^2\).

Method. Use the two-restriction quadratic form. The common mistake to avoid is reporting either coefficient’s squared t statistic as the joint F statistic.

The contrast vector is \((3,1)'\), and its estimated covariance matrix is the identity matrix because each estimated variance is \(1\) and the covariance is zero. The quadratic form is therefore

$$ (3,1) \begin{pmatrix}1&0\\0&1\end{pmatrix}^{-1} \begin{pmatrix}3\\1\end{pmatrix} = 3^2+1^2=10. $$

There are \(q=2\) restrictions, so divide this value by 2:

$$ F=\frac{10}{2}=5, \qquad df=(2,20). $$

The individual t statistics are \(3/1=3\) and \(1/1=1\), whose squares are 9 and 1. Neither squared statistic is the joint F statistic of 5. Here the joint F is the average of the two squared standardized departures because the restriction estimates are uncorrelated and have equal variances. Check: the numerator combines both restrictions, and its division by \(q=2\) gives \(F=5\), not either individual t statistic squared.

Common Exam Traps and Interpretation

  • Forgetting the sign is lost. The identity is \(F=t^2\), not \(F=t\). A positive and a negative t statistic with the same magnitude produce the same F statistic.
  • Confusing a one-sided t test with an F test. A standard F test is right-tailed and does not reveal whether the estimate is above or below its null value. It matches the two-sided t test, not a directional alternative by itself.
  • Using the wrong restrictions. The F statistic equals \(t^2\) only when both tests use the same single null restriction and the same unrestricted model.
  • Treating a multi-restriction test as a single-coefficient test. For \(q>1\), the joint F statistic uses all restrictions and their covariance structure. It is not generally one coefficient’s squared t statistic.
  • Changing the denominator model. The partial F formula uses the full-model residual mean square, \(SSE_U/\nu\), with its residual degrees of freedom. Use a consistent unrestricted model for the corresponding t test.
Key takeaway: One linear restriction gives a two-sided t test and an equivalent one-numerator-degree-of-freedom partial F test, with \(F=t^2\). Multiple restrictions require a joint F statistic; do not interpret it as one t statistic squared. This connection is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each problem, identify whether the test has one or multiple restrictions and use the appropriate relationship.

  1. A coefficient estimate is \(1.2\), its standard error is \(0.25\), and the null value is \(0.2\). The full model has 16 residual degrees of freedom. Find \(t\) and the equivalent one-restriction \(F\). (Answer: \(t=4\), \(F=16\), with degrees of freedom \((1,16)\).)
  2. For a single contrast, the estimate is \(0.6\), its null value is zero, and its estimated variance is \(0.11\). Find \(t\) and \(F\). (Answer: \(t=0.6/\sqrt{0.11}\approx1.8091\), \(F=0.36/0.11\approx3.2727\).)
  3. Two uncorrelated coefficient estimates have standard errors of \(1\), estimates \(3\) and \(1\), and are jointly tested against zero. There are 20 residual degrees of freedom. Find the joint F statistic and compare it with the individual squared t statistics. (Answer: \(F=(3^2+1^2)/2=5\), while the individual squared t statistics are 9 and 1.)
  4. A one-sided t statistic is \(-2.1\) for a test whose alternative is that the coefficient is greater than its null value. What F statistic results from squaring it, and does that F value by itself support the directional alternative? (Answer: \(F=4.41\); no, F loses the sign and corresponds to a two-sided test.)
  5. A single restriction increases SSE from \(75\) in the unrestricted model to \(91\) in the restricted model. The unrestricted model has 20 residual degrees of freedom. Find the partial F statistic and the magnitude of the matching t statistic. (Answer: \(F=16/(75/20)=4.2667\), and \(|t|=\sqrt{4.2667}\approx2.0656\).)

Previous

Next

Back to Tutorials
Clinical Biostats