Why the F-Statistic Rule of Ten Is Limited

What a First-Stage F Value Can—and Cannot—Tell You

A first-stage F statistic summarizes how strongly the excluded instruments jointly predict an endogenous regressor after controlling for included exogenous variables. The familiar advice to look for an F value above ten is a useful warning signal, not a theorem that separates reliable instrumental-variable analyses from unreliable ones. Its meaning depends on the setting in which it is used.

In particular, the conventional cutoff does not automatically account for how many instruments are tested, which instrumental-variable estimator is used, or whether the errors satisfy the assumptions behind the conventional test. A value above ten is evidence about first-stage predictive strength under a particular calculation; it does not by itself certify instrument validity or guarantee accurate 2SLS estimates and inference.

The exact conventional F test has its usual finite-sample reference distribution when the first-stage errors are independent, normally distributed, and homoskedastic, conditional on the regressors. With homoskedasticity but without normality, the conventional calculation may still be used as an approximation in suitable large samples. If the error structure is heteroskedastic or clustered, the conventional standard error calculation is generally not the appropriate one. A conventional F value should therefore be reported with its assumptions, not detached from them.

Where the Formula Comes From

Let \(SSR_R\) be the residual sum of squares from the restricted first stage that includes \(w_i\) but omits \(z_i\), and let \(SSR_U\) be the residual sum of squares from the unrestricted first stage that includes both \(w_i\) and \(z_i\). Define the partial \(R^2\) of the excluded instruments as the fraction of the restricted residual variation removed by adding them:

The conventional F statistic compares the average reduction in residual sum of squares per tested restriction with the unrestricted residual mean square. Substituting the definition of partial \(R^2\) gives the formula below.

This derivation shows that F is not just another name for partial \(R^2\): it also depends on the number of excluded instruments and the residual degrees of freedom. A partial \(R^2\) is a scale-free summary of first-stage fit, while the F statistic compares fit improvement with residual noise under the conventional variance calculation.

Why Ten Is Not a Universal Cutoff

The rule of ten is a rough diagnostic convention associated with particular weak-instrument concerns and assumptions. It is not a probability statement, a test of instrument validity, or a universal boundary at which the behavior of an estimator suddenly changes. A statistic of 9.9 is not categorically unsafe while 10.1 is categorically safe. The risk depends on the full model and the inferential goal.

One reason instrument count matters is that the conventional F statistic has \(L\) numerator degrees of freedom. In the one-endogenous-regressor setting, the concentration parameter \(\mu^2\), introduced in the earlier material on the concentration parameter and instrument strength, describes the aggregate strength of the first stage. Under the standard homoskedastic setup, the first-stage F is related to that strength; a useful large-sample approximation is \(E(F)\approx 1+\mu^2/L\). Thus, the same observed F with different \(L\) does not correspond to the same aggregate concentration parameter. More importantly, the consequences of weak identification depend on the relationship between instrument strength and the number of instruments. A fixed cutoff cannot encode every such relationship or guarantee a specified bound on bias or test size.

The estimator matters as well. The first-stage F is computed from the first-stage regression, so it does not change merely because an analyst switches from 2SLS to another IV estimator. But the finite-sample behavior of those estimators need not be the same under weak instruments. In particular, weak-instrument bias and uncertainty can affect 2SLS differently from alternatives such as LIML. Therefore, a cutoff proposed as a rough guide to one estimator’s behavior is not automatically a guarantee for another estimator.

Finally, the conventional statistic relies on a particular error calculation. Heteroskedasticity or within-cluster dependence can make its estimated variance inappropriate. A robust or cluster-adjusted test may yield a different statistic and reference distribution. Comparing that statistic mechanically with ten, a cutoff developed for a conventional F calculation, does not preserve the original rule’s interpretation.

Estimator and Error Structure Change the Interpretation

A first-stage strength diagnostic answers a limited question: how much do the excluded instruments predict the endogenous regressor, conditional on the included exogenous regressors? It does not directly measure the performance of every estimator or inference procedure. When the first stage is weak, 2SLS can be pulled toward the corresponding OLS estimate, and conventional IV confidence intervals and tests can perform poorly. The F statistic does not report the size of that bias or the coverage of a particular interval. Different estimators can respond differently to the same weakness, so the selected estimator must be part of the interpretation.

Error structure imposes a separate qualification. The conventional F formula uses the homoskedastic residual variance. If observations have unequal error variances or dependence within clusters, the conventional test may have incorrect size: its nominal rejection rate need not match its actual rejection rate under the null. A robust first-stage test is then needed for the relevant error structure. Its value should be reported with its method and reference distribution, rather than treated as interchangeable with the conventional F value.

Practical Assessment and Common Pitfall

Use the conventional F as one piece of evidence, with its scope made explicit. A clear diagnostic report identifies the number of excluded instruments, the number of endogenous regressors, the included controls, the statistic and degrees of freedom, and whether the test uses conventional, heteroskedasticity-robust, or cluster-adjusted errors. Reporting the partial \(R^2\) alongside the test gives readers another view of first-stage fit. These summaries still do not prove the instruments’ exclusion restriction or independence from the structural error.

A useful comparison is between the F statistic and the partial \(R^2\). The F statistic tests joint predictive contribution relative to a variance estimate; partial \(R^2\) describes the share of residual variation in \(x\) explained by the excluded instruments. Neither is a complete diagnostic for weak-IV inference, and neither replaces checking that the instruments satisfy the substantive assumptions needed for IV analysis. If the error structure or estimator makes the conventional cutoff inappropriate, use diagnostics and inference methods suited to that setting rather than mechanically reusing ten.

• Mistake: “The first-stage F is 11, so the instruments are valid and the 2SLS estimate is reliable.”
  Correction: The statistic concerns conditional first-stage prediction under its variance assumptions. It does not test instrument validity or guarantee estimator and inference performance.
• Mistake: “Any statistic called F can be compared with ten.”
  Correction: The reference method and error assumptions matter. A robust or cluster-adjusted test is not automatically calibrated to the conventional cutoff.
• Mistake: “F=9.8 fails, while F=10.2 proves strength.”
  Correction: Ten is a heuristic, not a sharp boundary. Interpret the result in light of instrument count, estimator, errors, and the purpose of the diagnostic.