Why Examine the First Stage?
Instrumental variables estimation needs instruments that predict the endogenous treatment after accounting for included covariates. A first-stage regression makes that predictive relationship visible. Its F statistic is a familiar summary of whether the excluded instruments jointly contribute to that regression. It is useful, but its meaning depends on the model and variance assumptions behind it.
In particular, a small p-value for the first-stage test is not a general certificate of strong identification. The conventional F statistic does not test whether an instrument is valid, and the rule of thumb that an F statistic above 10 is reassuring is not universal. The number of endogenous regressors, the error structure, and the number of instruments all matter.
Definition and Classical Reference Model
For one endogenous treatment \(D_i\), let \(X_i\) denote the included covariates and \(Z_i\) the vector of \(q\) excluded instruments. The first-stage linear regression is
The coefficient vector \(\pi\) describes the instruments' conditional linear contribution to predicting treatment, holding the included covariates fixed. The conventional first-stage F test has null hypothesis \(H_0:\pi=0\): all \(q\) excluded-instrument coefficients are zero. Its alternative is that at least one is nonzero.
The classical finite-sample F reference distribution is exact under the linear first-stage model when the errors are independent, normally distributed, and have constant variance, with the design matrix treated as fixed. The conventional calculation also relies on the usual homoskedastic OLS variance formula. Without those conditions, the same numerical statistic need not have the stated F distribution. In large samples, related approximations can be useful under appropriate conditions, but dependence or heteroskedasticity calls for a variance estimator and reference procedure suited to that structure.
Let \(SSR_r\) be the residual sum of squares from the restricted regression, which includes \(X_i\) but excludes \(Z_i\). Let \(SSR_u\) be the residual sum of squares from the unrestricted regression, which includes both \(X_i\) and \(Z_i\). If the unrestricted regression has \(df\) residual degrees of freedom, the conventional statistic is
The numerator measures the reduction in residual variation per excluded coefficient; the denominator estimates the unrestricted regression's residual variance. The statistic compares the added explanatory power of the instruments with the remaining unexplained variation.
Partial R-Squared and a Short Derivation
The partial R-squared for the excluded instruments is the fraction of the restricted residual variation removed by adding them:
It describes their incremental fit after the included covariates have already been taken into account. From this definition, \(SSR_u=SSR_r(1-R^2_{\text{partial}})\), while \(SSR_r-SSR_u=SSR_rR^2_{\text{partial}}\). Substitution into the F formula gives
This identity shows why the statistic is not determined by partial R-squared alone: it also depends on the number of restrictions \(q\) and the residual degrees of freedom. The partial R-squared is a descriptive measure of fit; the F statistic additionally scales that fit by the model's degrees of freedom and its residual variation.
Worked Example: A Significant Test Below a Familiar Rule of Thumb
Suppose a hypothetical sample has 120 observations, three included covariates, an intercept, and two excluded instruments. The partial R-squared for the instruments is 0.12. The unrestricted regression estimates six coefficients in total (intercept, three covariates, and two instruments), so its residual degrees of freedom are \(120-6=114\). Under the classical assumptions, calculate the conventional first-stage F statistic.
Here \(q=2\), \(df=114\), and \(R^2_{\text{partial}}=0.12\). Therefore,
The statistic is compared with an \(F\) distribution with 2 and 114 degrees of freedom under the null and the classical assumptions. Its p-value is approximately 0.0007. Thus, these data provide strong evidence against the null that both excluded-instrument coefficients are zero. The statistic is nevertheless below 10, a commonly quoted but nonuniversal rule of thumb. The result demonstrates why “statistically distinguishable from zero” and “strong enough for a particular IV inference problem” are not interchangeable conclusions.
The F statistic is a property of the conditional first-stage regression and its assumptions. Even a very small p-value says nothing by itself about whether the instruments are independent of the outcome error or satisfy the exclusion restriction.
Why Heteroskedasticity and Clustering Matter
The conventional F calculation uses a pooled residual variance and assumes that the errors have the structure required for that variance estimate and reference distribution. If first-stage errors have different variances across observations, the homoskedastic variance formula is generally not appropriate. If observations within a clinic, school, or region share shocks, treating every observation as independent can also misstate uncertainty. These issues affect the test's calibration, not merely the way its result is worded.
A conventional F statistic can still be calculated from the regression sums of squares in either setting. But that calculation alone does not make its classical p-value valid. A researcher should not infer that an instrument is reassuringly relevant simply because a conventional statistic exceeds a familiar cutoff when the variance or dependence assumptions are doubtful. The appropriate robust first-stage testing procedures require choices that account for the error structure.
Worked Example: A Large Conventional F With Clustered Observations
Consider hypothetical data on 200 patients from 20 clinics. A first-stage regression includes an intercept, three covariates, and one excluded instrument. Suppose its partial R-squared is 0.08. The unrestricted regression has residual degrees of freedom \(200-5=195\), so the ordinary calculation is
If one incorrectly treats all 200 observations as independent, this may look reassuring by the conventional “above 10” heuristic. But suppose patients in the same clinic share unobserved influences on treatment, so their first-stage errors are correlated within clinic. The ordinary calculation does not account for that dependence; its usual \(F(1,195)\) reference distribution and resulting p-value are not justified merely by the value 16.96. There are 20 clinics, not 200 independent clusters. This example does not tell us what a dependence-adjusted statistic would be; it shows why the conventional statistic alone cannot settle the question.
Multiple Endogenous Regressors: Separate F Tests Can Miss the Issue
When there is more than one endogenous regressor, there is a first-stage regression for each one. Reporting the excluded-instrument F statistic from each regression is informative about its individual prediction, conditional on the included covariates. It is not, by itself, a complete diagnosis of whether the instruments provide enough distinct predictive variation to identify all endogenous coefficients.
The key distinction is between strength in each separate first-stage regression and the number of independent instrument-prediction directions across the endogenous regressors. If two endogenous variables are predicted by the same linear combination of instruments, their separate first stages can both look strong even though the instrument effects do not span the separate directions needed for identification. The relevant concern is the rank of the matrix of first-stage coefficients, not just whether each row has a large F statistic.
Worked Example: Strong Separate First Stages but Only One Direction
Suppose a hypothetical model has two endogenous regressors, \(D_1\) and \(D_2\), and two excluded instruments, \(Z_1\) and \(Z_2\). In 100 observations with an intercept and no other covariates, suppose each first-stage regression has partial R-squared 0.50. Each unrestricted first stage estimates three coefficients, so \(df=100-3=97\). For either endogenous regressor, the conventional joint F statistic is
Those are large separate first-stage F statistics under the stated conventional model. Now suppose the population first-stage coefficient matrix, with rows for \(D_1,D_2\) and columns for \(Z_1,Z_2\), is
The second row is twice the first, so the matrix has rank one, not rank two. The instruments shift the two endogenous regressors in only one independent direction. Thus, the large separate F statistics do not establish that two distinct endogenous effects are identified. This example isolates the limitation of reading each first-stage test separately; a full analysis of multiple-endogenous-regressor weak identification requires diagnostics beyond those individual tests.
Many Instruments and the Limits of a Single Number
The first-stage F statistic also depends on how many excluded instruments are tested jointly. Adding instruments can increase in-sample fit, but it also increases \(q\), the number of restrictions, and consumes residual degrees of freedom. A large instrument set relative to the sample can make the first stage fit treatment especially well in the observed sample without ensuring that the resulting IV estimator behaves well in finite samples. The F statistic is not a general diagnostic for that many-instrument problem.
In particular, counting instruments and looking at one F value cannot reveal every consequence of instrument proliferation. First-stage fit is measured in the sample used to estimate the regression; it need not represent stable predictive information outside that sample. A high F should therefore be interpreted alongside the instrument count, sample size, specification, and the inferential question, rather than treated as a universal pass/fail result.
Worked Example: Instrument Count Changes the F Calculation
Suppose a hypothetical first stage has 100 observations, an intercept, no included covariates, and 20 excluded instruments. If their partial R-squared is 0.30, the unrestricted residual degrees of freedom are \(100-1-20=79\). The conventional statistic is
The instruments jointly account for 30% of the residual variation left after the restricted regression, but the F statistic is only about 1.69 because that incremental fit is spread across 20 restrictions. This calculation does not mean that 30% fit is automatically adequate or inadequate; it demonstrates that a partial R-squared and an F statistic answer related but different questions. With many instruments, neither a substantial in-sample fit nor a single conventional F value resolves the broader finite-sample concerns.
Interpreting Thresholds Carefully
The “F greater than 10” heuristic is often encountered in discussions of a single endogenous regressor and conventional homoskedastic first-stage tests. It is a rough warning device, not a theorem that divides safe from unsafe designs. Its relevance depends on the setting and on what risk is being controlled. In particular, it cannot be transferred automatically to clustered or heteroskedastic data, several endogenous regressors, or a large instrument set.
Keep three questions separate:
- Conditional relevance: Do the excluded instruments predict the endogenous regressor(s) after adjustment for included covariates?
- Strength for inference: Is the predictive variation sufficient for the IV estimation and inferential task at hand?
- Instrument validity: Are the instruments independent of the outcome's unobserved causes and excluded from the outcome equation except through treatment?
The conventional first-stage F test directly addresses the first question under its assumptions. It does not answer the second in every design, and it does not answer the third. Relevance is necessary for IV identification, but relevance alone is not sufficient.
Common Mistakes
- “The F statistic exceeds 10, so the instruments are valid.” This confuses relevance with validity. A first-stage test does not assess independence from the outcome error or the exclusion restriction.
- “A tiny p-value proves the instruments are strong.” The p-value tests a zero-coefficient null under a reference model; it does not quantify all weak-identification risks or establish that an estimator has reliable finite-sample behavior.
- “A conventional F is enough even when errors are clustered.” The standard calculation does not automatically account for within-cluster dependence. Its classical reference distribution may then be inappropriate.
- “Each endogenous variable has a high F, so the whole system is well identified.” Separate tests can miss the possibility that instruments predict multiple endogenous regressors along the same direction.
- “More instruments can only help because they increase fit.” Additional instruments change the degrees of freedom and can introduce many-instrument concerns; in-sample fit alone is not a sufficient guide.
Check Your Understanding
Use the assumptions and distinctions in this tutorial to answer the following.
- In a first-stage regression, what null hypothesis does the conventional F statistic test?
- A regression has \(q=3\), \(df=90\), and partial R-squared 0.18. Calculate its conventional first-stage F statistic using the partial R-squared formula.
- Why can a conventional first-stage F statistic be misleading when observations are clustered by clinic?
- How could two endogenous regressors each have a large first-stage F statistic while the instrument coefficient matrix has rank one?
- Why does a first-stage F statistic above a familiar rule-of-thumb value not establish instrument validity?