A Worked First-Stage Diagnostic Assessment

What a First-Stage Assessment Can Establish

A first-stage diagnostic asks whether the excluded instruments explain variation in the endogenous treatment after accounting for the included covariates. It is a diagnostic of relevance in the specified first-stage model, not a test that the instruments are independent of unmeasured causes of the outcome or satisfy the exclusion restriction. Those are separate requirements for an instrumental-variables analysis.

This tutorial assesses a sample first stage with one endogenous treatment variable. It combines two complementary summaries: the joint F statistic for the excluded instruments and the partial R-squared measuring their added explanatory contribution. Both depend on the covariates and the sample used. Neither should be read in isolation from the number of instruments, the error structure, and other diagnostics.

Definition: Let $D_i$ be the endogenous treatment, $Z_i$ the vector of $q$ excluded instruments, and $X_i$ the vector of $K$ included covariates for observation $i$. The first stage is $$ D_i=\alpha+X_i'\gamma+Z_i'\pi+v_i. $$ The joint first-stage null is $H_0:\pi=0$: conditional on the included covariates, all excluded-instrument coefficients are zero. The alternative is that at least one component of $\pi$ is nonzero.

The unrestricted first stage includes the intercept, $X_i$, and $Z_i$. The restricted first stage includes the intercept and $X_i$, but omits $Z_i$. Let $RSS_u$ and $RSS_r$ denote their residual sums of squares, respectively. Both regressions must use the same observations and the same included covariates. If the unrestricted model has full column rank, its residual degrees of freedom are $df_u=n-(1+K+q)$.

Calculating the Joint F Statistic and Partial R-Squared

Under the conventional homoskedastic linear-regression model, the joint F statistic compares the reduction in residual sum of squares with the unrestricted model's residual variation. With independent, normally distributed errors of constant variance conditional on the regressors, it has an exact $F$ distribution under the joint null, with $q$ and $df_u$ degrees of freedom. Without normality, the same statistic is commonly used as a large-sample test under suitable regularity conditions and homoskedasticity. The model also requires adequate rank: the included regressors and excluded instruments must not be perfectly collinear.

F=\frac{(RSS_r-RSS_u)/q}{RSS_u/df_u}.

The numerator is the reduction in residual sum of squares per excluded-instrument restriction. The denominator is the unrestricted residual mean square. A larger value indicates a greater reduction relative to the remaining residual variation, but its interpretation depends on $q$, $df_u$, and the assumptions behind the test.

Partial R-squared expresses the same nested-model comparison on a proportion scale. It is the fraction of the restricted model's residual sum of squares removed when the excluded instruments are added, holding the included covariates fixed:

R^2_{\mathrm{partial}}=\frac{RSS_r-RSS_u}{RSS_r} =1-\frac{RSS_u}{RSS_r}.

The connection between these measures follows directly by writing $RSS_u=RSS_r(1-R^2_{\mathrm{partial}})$ and $RSS_r-RSS_u=RSS_rR^2_{\mathrm{partial}}$. Substituting into the F formula gives the identity below, which also provides a useful arithmetic check.

F=\frac{df_u}{q}\frac{R^2_{\mathrm{partial}}}{1-R^2_{\mathrm{partial}}} \qquad\Longleftrightarrow\qquad R^2_{\mathrm{partial}}=\frac{qF}{df_u+qF}.

Worked Example: Three Instruments and Four Covariates

Suppose a hypothetical analysis uses $n=238$ observations. The treatment is a binary indicator of receiving a service, and the three excluded instruments are hypothetical encouragement indicators. The first stage includes four covariates in both regressions. Assume the conventional homoskedastic model is being used for this calculation, and suppose the restricted and unrestricted residual sums of squares are $RSS_r=1300$ and $RSS_u=1170$. There are $1+4+3=8$ unrestricted regression coefficients, including the intercept, so $df_u=238-8=230$. The instrument set and covariates are assumed to have full rank.

The joint test has $q=3$ restrictions. Substitution gives

F=\frac{(1300-1170)/3}{1170/230} =\frac{43.3333}{5.0870} =8.519.

The instruments reduce the restricted model's residual sum of squares by $1300-1170=130$, so

R^2_{\mathrm{partial}}=\frac{130}{1300}=0.1000.

Thus, conditional on the four covariates, the three instruments together account for 10.00% of the treatment variation left unexplained by the restricted model. The joint F statistic is 8.519. These values describe this sample and this specification; they do not establish instrument validity. The F value is below the often-quoted value of 10, but that comparison is not a universal decision rule, especially with multiple instruments and potentially nonclassical errors.

Why Covariates and Instrument Count Matter

The joint test is conditional: it asks whether the excluded instruments add explanatory power after the covariates have entered the regression. Adding or changing covariates can alter both the residual variation available to explain and the estimated contribution of the instruments. Consequently, a first-stage statistic without a clear account of the included covariates is difficult to interpret. Covariates should be chosen for the design and included consistently in the first and second stages; changing them solely to improve a strength statistic can undermine the analysis.

The number of excluded instruments also matters. The F numerator divides the reduction in residual sum of squares by $q$, the number of joint restrictions. A large raw improvement from adding many instruments does not, by itself, imply strong conditional relevance. More instruments can also increase overfitting concerns, particularly when the instrument count is large relative to the sample. Report $q$, the covariates, the sample size, and the residual degrees of freedom alongside the statistic and partial R-squared.

Worked Example: Two Instruments Conditional on Covariates

Consider a hypothetical study of treatment dosage measured in sessions. The first stage uses $n=160$ observations, two excluded instruments, and three included covariates. The restricted model has $RSS_r=920$, and the unrestricted model has $RSS_u=850$. Both models use the same 160 observations and covariates. The unrestricted model has $1+3+2=6$ coefficients, giving $df_u=160-6=154$. Assume full rank and the conventional homoskedastic model for the classical F calculation.

Here $q=2$, and the residual sum of squares falls by $920-850=70$. Therefore,

F=\frac{70/2}{850/154} =\frac{35}{5.51948} =6.341.

The partial R-squared is

R^2_{\mathrm{partial}}=\frac{920-850}{920} =\frac{70}{920} =0.07609.

Conditional on the three covariates, the two instruments account for about 7.609% of the residual variation in treatment dosage that remained under the restricted model. This is a measure of added explanatory power, not a percentage of treatment variation explained by the entire first stage. The relatively small partial R-squared and the joint F statistic of 6.341 warrant caution about relevance; neither number alone decides whether a particular IV analysis is trustworthy.

Checking the Assessment and Choosing the Right Diagnostic

Before interpreting a reported first-stage result, verify what regression and test it represents. In particular, check the number of excluded-instrument restrictions, the included covariates, the observations used, the residual degrees of freedom, and whether the statistic is conventional or robust. Confirm that $RSS_r$ is at least as large as $RSS_u$ for genuinely nested models fitted to the same data. If not, the models, samples, or reported quantities may not match.

Residual structure and leverage can reveal whether a conventional calculation is a poor summary of the sample. Heteroskedasticity or clustering changes the appropriate variance calculation: a conventional RSS-based F test need not have its claimed reference distribution. In such settings, use the relevant heteroskedasticity-robust or cluster-robust first-stage test rather than presenting the conventional F as if its assumptions held. A robust Wald statistic need not satisfy the exact RSS-to-partial-R-squared identity above. With several endogenous regressors, one first-stage F statistic for one treatment does not establish adequate conditional strength for every endogenous regressor.

Other diagnostics answer related but different questions. Partial R-squared summarizes conditional sample fit; robust first-stage tests adjust inference to the specified error structure. The Kleibergen-Paap rk Wald F statistic is used in settings where heteroskedasticity-robust rank-based inference is needed, while the Cragg-Donald statistic and Stock-Yogo critical values apply under their stated conventional assumptions. The effective F statistic addresses strength assessment in settings where conventional summaries can be inadequate. These alternatives are not interchangeable labels for the same calculation; the design and error assumptions determine which is appropriate.

Worked Example: Five Instruments in a Modest Sample

Suppose a hypothetical first stage uses $n=205$ observations, five excluded instruments, and four included covariates to predict a treatment measured in hours. The restricted model has $RSS_r=1000$, and the unrestricted model has $RSS_u=960$, using the same observations and covariates. The unrestricted model has $1+4+5=10$ coefficients, so $df_u=205-10=195$. Assume full rank and use the conventional homoskedastic calculation.

The reduction is $1000-960=40$, and $q=5$. Thus,

F=\frac{40/5}{960/195} =\frac{8}{4.92308} =1.625.

The partial R-squared is

R^2_{\mathrm{partial}}=\frac{40}{1000}=0.04000.

The five instruments together account for 4.000% of the treatment variation left unexplained after the four covariates are included. The joint F is 1.625, indicating little sample evidence of a strong joint first stage under this specification. The five-instrument count should be reported rather than hidden: the number of restrictions and the modest sample are relevant to assessing the result. This diagnostic still says nothing about whether the instruments are independent of unmeasured outcome causes.

Common Pitfalls and Practical Conclusions

Using different covariates or samples in the two regressions. The RSS comparison is meaningful for the stated nested-model test only when the restricted and unrestricted models are fitted to the same observations with the same included regressors.
Reporting a statistic without its instrument count. The numerator degrees of freedom are $q$. State how many excluded instruments were jointly tested, as well as the residual degrees of freedom.
Treating a threshold as a verdict. The rule of ten is not universal. Instrument count, error structure, sample design, and the specific diagnostic all matter; a value above ten does not prove validity or guarantee reliable IV inference.
Calling partial R-squared the full-model R-squared. It measures the proportional reduction in the restricted model's residual sum of squares when the excluded instruments are added, conditional on the covariates.
Equating relevance with validity. A strong first stage cannot show that instruments satisfy independence or exclusion. Those assumptions require design knowledge and substantive justification.

A careful assessment therefore reports the joint statistic, its degrees of freedom, the partial R-squared, the instrument count, the included covariates, and the sample used. It also states whether the calculation is conventional or robust and checks whether its assumptions match the data. The result is evidence about the first-stage relationship in a particular specification—not a stand-alone certification of an IV design.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A nested first stage has $RSS_r=500$, $RSS_u=450$, $q=2$, and $df_u=90$. Calculate the conventional joint F statistic and partial R-squared.
In the same nested-model setup, why must the restricted and unrestricted regressions use the same covariates and observations?
A first-stage F statistic is above 10. What does this establish, and what does it not establish?
Why should a conventional RSS-based F test be reconsidered when errors are clustered?
What information should accompany a reported joint first-stage F statistic?

Solutions. (1) The reduction is $500-450=50$, so $F=(50/2)/(450/90)=25/5=5.000$. The partial R-squared is $50/500=0.1000$, or 10.00%. (2) The comparison relies on a nested-model reduction attributable to adding the excluded instruments. Different covariates or observations would confound that comparison. (3) It is evidence about the instruments' joint first-stage relevance under the reported specification and assumptions. It does not prove instrument validity, and 10 is not a universal guarantee of reliable inference. (4) Clustering changes the error dependence and the appropriate variance calculation, so the conventional reference distribution may not be justified; a cluster-robust test is needed. (5) Report the number of excluded instruments, numerator and residual degrees of freedom, covariates, sample, and whether the statistic is conventional or robust, together with the partial R-squared when available.