Exact Versus Large-Sample t and F Calibration

Why Calibration Depends on the Model

A test statistic does not determine its own reference distribution. A ratio that follows a Student t or F distribution exactly under one model may follow neither distribution under another. The distinction matters because calibration supplies the critical value, or p-value, used to assess evidence against a null hypothesis.

For the linear model, Gaussian errors with a common variance give exact finite-sample t and F distributions. When these assumptions do not hold, large-sample approximations may still justify inference, but only under appropriate conditions. In particular, heteroskedasticity can make the usual least-squares standard errors wrong, even when a large sample is available.

Write \(\hat\beta=(X'X)^{-1}X'Y\), \(SSE=(Y-X\hat\beta)'(Y-X\hat\beta)\), and \(s^2=MSE\). For an estimable scalar contrast \(c'\beta\), with \(c\ne0\), its usual estimated standard error is \(s_c=s\sqrt{c'(X'X)^{-1}c}\). For a full-rank \(q\times p\) restriction matrix \(R\), the usual general linear hypothesis statistic under \(H_0:R\beta=r\) is \(F=(SSE_R-SSE)/(q\,MSE)\), where \(SSE_R\) is the residual sum of squares under the restrictions.

Exact Finite-Sample Calibration

Under the Gaussian spherical model, the least-squares estimate and residual sum of squares have a special independence property. That property, together with the exact chi-square distributions of Gaussian quadratic forms, yields t and F reference distributions without taking a large-sample limit.

Proof for the scalar contrast. Under the null, \(c'\hat\beta-\theta_0=c'(X'X)^{-1}X'\varepsilon\), which is normal with mean zero and variance \(\sigma^2c'(X'X)^{-1}c\). The residual vector is \((I-P_X)\varepsilon\), where \(P_X=X(X'X)^{-1}X'\). The two quantities are jointly normal, and their covariance is zero because \(X'(I-P_X)=0\). They are therefore independent. Since \(I-P_X\) is a projection of rank \(n-p\), \(SSE/\sigma^2\sim\chi^2_{n-p}\). Thus \(T\) is a standard normal variable divided by the square root of an independent \(\chi^2_{n-p}/(n-p)\), which is the definition of \(t_{n-p}\). \(\square\)

Proof for the joint restrictions. Under the null, the restricted fitted vectors form an affine subspace of the unrestricted model space \(\operatorname{col}(X)\), of dimension \(p-q\). The difference \(SSE_R-SSE\) is the squared length of the projection of the Gaussian error onto the \(q\)-dimensional part of \(\operatorname{col}(X)\) excluded by the restrictions. Consequently, \((SSE_R-SSE)/\sigma^2\sim\chi^2_q\). The unrestricted residual sum of squares is the squared length of the projection onto the orthogonal complement of \(\operatorname{col}(X)\), so \(SSE/\sigma^2\sim\chi^2_{n-p}\). These projections are orthogonal, hence independent under Gaussian errors. Dividing the first chi-square by \(q\) and the second by \(n-p\) gives the stated \(F_{q,n-p}\) distribution. \(\square\)

Large-Sample Calibration When Exactness Is Unavailable

If errors are not Gaussian, the exact t and F derivations do not apply. If errors are heteroskedastic, the usual \(s^2(X'X)^{-1}\) covariance estimate is generally not the right covariance estimate, even asymptotically. Large-sample inference instead uses a central limit theorem and a consistent estimate of the covariance appropriate to the error structure.

For example, suppose the design is fixed, errors are independent with mean zero and variances that may differ, and regularity conditions ensure a central limit theorem for \(\hat\beta\). One common set of sufficient conditions requires \(X'X/n\) to converge to a positive-definite matrix, a Lindeberg condition for the weighted errors, and suitable moment and leverage bounds. Let \(\Omega=\operatorname{diag}(\operatorname{Var}(\varepsilon_i))\). The covariance of \(\hat\beta\) is then \(V_n=(X'X)^{-1}X'\Omega X(X'X)^{-1}\). A heteroskedasticity-consistent sandwich estimator, under conditions ensuring its consistency, estimates this covariance.

Derivation. The assumed central limit theorem gives the contrast error, divided by its true standard deviation, a limiting standard normal distribution. The assumed relative consistency gives \(\sqrt{c'\widehat V_n c}/\sqrt{c'V_nc}\xrightarrow{p}1\), so Slutsky’s theorem proves the first limit. For the joint restrictions, the true-covariance-standardized \(q\)-vector converges to \(N_q(0,I_q)\), and the assumed standardized covariance consistency makes the estimated and true covariance matrices asymptotically equivalent in that scaling. Slutsky’s theorem then gives the stated Wald limit, since the squared Euclidean length of a \(N_q(0,I_q)\) vector has a \(\chi^2_q\) distribution. These are asymptotic statements: they do not assert an exact finite-sample distribution. \(\square\)

What “Large-Sample F” Does—and Does Not—Mean

For fixed \(q\), \(F_{q,\nu}\) approaches \(\chi^2_q/q\) as \(\nu\) grows, because the denominator chi-square divided by \(\nu\) concentrates near one. This explains why a classical F calibration can resemble a chi-square calibration at large residual degrees of freedom when the classical covariance structure is valid. It does not repair a misspecified standard error: under heteroskedasticity, a conventional F statistic need not have the intended large-sample limit.

A robust Wald statistic is often reported as \(W\), with a \(\chi^2_q\) reference distribution, or as \(W/q\), with a \(\chi^2_q/q\) reference distribution. Some software may use an F reference distribution with a finite or adjusted denominator degrees of freedom as a practical approximation. Unless a separate result establishes that calibration, it should not be described as an exact \(F_{q,n-p}\) test.

Common Mistakes and Practical Checks

• Calling the usual t or F distribution exact without its assumptions. Exactness here requires Gaussian spherical errors and the stated null; large sample size alone does not create exactness.
• Forgetting positive residual degrees of freedom. The assumptions must include \(n>p\). If \(n=p\), then \(MSE\) divides by zero and the stated \(t_{n-p}\) and \(F_{q,n-p}\) references do not exist.
• Assuming nonnormality always invalidates every large-sample test. It invalidates the exact Gaussian argument, but a central limit theorem can still justify asymptotic inference under suitable moment and design conditions.
• Using a conventional standard error under heteroskedasticity and relying on a large \(n\). A large sample does not in general make that covariance estimate consistent. Use a suitable sandwich estimate when its assumptions are justified.
• Confusing an asymptotic chi-square Wald test with an exact F test. Their large-sample relationship is useful, but the reference distributions and assumptions should be stated accurately.