Reporting Strength Is More Than Giving a Statistic
A first-stage diagnostic is useful only when readers can tell what was tested, how uncertainty was estimated, and which reference distribution was used. A bare statement such as “the first-stage F was above 10” conceals the instrument set, the controls, the covariance assumptions, and whether the comparison is appropriate. Good reporting makes those choices visible and distinguishes evidence about instrument relevance from evidence about the validity of an instrumental-variable design.
Let \(Y_i\) be an outcome, \(D_i\) an endogenous treatment, \(Z_i\) a vector of \(q\) excluded instruments, and \(X_i\) a vector of included exogenous controls. The first stage is a regression of \(D_i\) on the excluded instruments and controls. Its joint relevance null is that all coefficients on the excluded instruments are zero, conditional on \(X_i\). Rejection indicates evidence of conditional relevance in the specified sample and model; it does not establish the exclusion restriction or other requirements for a causal interpretation.
What to Report for the First Stage
For a classical homoskedastic linear first stage, compare a restricted regression containing \(X_i\) with an unrestricted regression containing \(X_i\) and \(Z_i\). Write \(SSR_R\) and \(SSR_U\) for their residual sums of squares, and let \(k_U\) be the number of estimated coefficients in the unrestricted regression, including the intercept if present. The conventional partial F statistic for the excluded instruments is
This tests the joint null that the \(q\) excluded-instrument coefficients are zero. Under independent observations, a correctly specified linear model, and homoskedastic normal errors, it has an exact \(F_{q,n-k_U}\) distribution under the null. In large samples, related approximations may be justified under weaker conditions, but the classical statistic is not automatically valid under heteroskedasticity or clustering.
The partial \(R^2\) gives a scale-free description of how much residual variation in \(D_i\), after accounting for \(X_i\), is explained by adding \(Z_i\):
It complements the F statistic but does not replace it: the F statistic also reflects sample size and the number of excluded instruments. A clear report identifies the dependent variable in the first stage, the excluded instruments, the included controls, the sample size, the joint test, its degrees of freedom, and the covariance estimator.
Worked Example: Classical First-Stage F and Partial R-Squared
Suppose a hypothetical study has \(n=240\) observations and two excluded instruments. The restricted first stage, with controls but no excluded instruments, has \(SSR_R=960\). The unrestricted first stage has \(SSR_U=720\) and estimates \(k_U=7\) coefficients. Assume independent observations and homoskedastic normal errors for the classical finite-sample F reference distribution.
There are \(q=2\) restrictions, and the unrestricted residual degrees of freedom are \(240-7=233\). Thus
The partial \(R^2\) is
A suitable report is: “In the first stage for treatment \(D\), the two excluded instruments were jointly associated with \(D\) conditional on the prespecified controls, \(F(2,233)=38.83\), with partial \(R^2=0.250\) (\(n=240\)); the classical homoskedastic covariance assumptions were used.” The calculation is consistent with the stated regression sums of squares. Neither statistic tests whether the instruments are exogenous or excluded from the outcome equation.
Covariance Assumptions Change the Diagnostic
When errors may be heteroskedastic, a heteroskedasticity-consistent covariance estimator can be used for a joint Wald test of the excluded-instrument coefficients. If \(\hat{\pi}\) is the \(q\)-vector of estimated first-stage coefficients on \(Z_i\), and \(\widehat{V}\) is its estimated covariance matrix, the Wald statistic is
Under the joint null and suitable large-sample conditions, \(W\) is compared with a chi-square distribution with \(q\) degrees of freedom. Some software reports \(W/q\) as a robust F-style statistic, sometimes with a finite-sample correction. That label alone does not make the statistic comparable to a classical homoskedastic F statistic or to a critical value derived under different assumptions. Report the form actually used, its reference distribution, and the covariance estimator.
The same principle applies when observations are clustered or otherwise dependent: the covariance estimator must reflect the sampling structure, and the number of independent clusters can matter for the quality of the reference approximation. State the clustering unit and any degrees-of-freedom or small-sample correction. If the sample structure does not support reliable covariance estimation, a printed statistic should not be presented as decisive evidence of strength.
Worked Example: A Heteroskedasticity-Robust Joint Test
Suppose a hypothetical first stage has two excluded-instrument coefficient estimates \(\hat{\pi}=(0.18,0.12)^{\mathsf T}\), with heteroskedasticity-consistent covariance estimate
The determinant is \(0.0036(0.0025)-0.0012^2=0.00000756\), so the inverse is
Substitution gives
Against a large-sample \(\chi^2_2\) reference distribution, the p-value is approximately \(0.0047\). This is evidence against the joint zero-coefficient null under the stated robust covariance and asymptotic approximation. It is not a classical \(F(2,df)\) result, and no classical F critical value should be attached to it without a valid matching argument.
Critical Values Are Conditional Benchmarks
A threshold such as “10” is often repeated as if it were a universal boundary between strong and weak instruments. It is not. Critical values developed for conventional first-stage diagnostics depend on the model and on the performance criterion they are designed to control. Stock–Yogo critical values, for example, are derived for particular linear IV settings under homoskedastic assumptions and are indexed by features such as the number of endogenous regressors and instruments, as well as the chosen tolerance for bias or test-size distortion. They are not general-purpose cutoffs for every robust statistic, sampling design, or estimand.
The problem is not only that a threshold may be numerically inappropriate. A diagnostic may look adequate under one criterion yet still leave substantial uncertainty in the treatment effect. Conversely, a statistic below a conventional benchmark does not mean that every weak-IV-robust analysis is uninformative. When reporting a comparison with a published threshold, name the threshold, state the assumptions and criterion it represents, and explain why the reported statistic is on a compatible scale. If compatibility is uncertain, report the statistic and its assumptions without making a categorical “passes/fails” claim.
Binary instruments require the same care. The observed first-stage relationship can depend on the instrument’s prevalence and the covariate distribution, so do not infer strength from the instrument being binary or from its estimated coefficient alone. Report the actual conditional diagnostic and the relevant design details.
Pair the Diagnostic With Weak-IV-Robust Inference
First-stage strength is a warning and reporting tool, not a substitute for inference on the causal parameter. Conventional 2SLS standard errors and confidence intervals can be misleading when instruments are weak. Weak-IV-robust procedures, including Anderson–Rubin inference and conditional likelihood ratio methods, are designed to retain valid inference under weak identification, subject to their own model and covariance assumptions. They may yield wide, unbounded, or disconnected confidence sets; those outcomes are substantive information about how much the data identify, not computational failures.
A transparent report presents the conventional estimate and uncertainty as such, then identifies the weak-IV-robust procedure used, its covariance estimator and reference method, and the resulting test or confidence set. State the confidence level or significance level, units, and the causal parameter being assessed. If the robust confidence set is broad or unbounded, do not summarize it only by its midpoint or by the conventional interval.
Worked Example: Reporting an Anderson–Rubin Test
Consider hypothetical reduced-form and first-stage estimates for one instrument. The estimated reduced-form coefficient is \(\hat{\gamma}=1.20\), and the first-stage coefficient is \(\hat{\pi}=0.30\). Their estimated variances are \(0.04\) and \(0.01\), and their estimated covariance is \(0.006\). To test the candidate treatment effect \(\beta_0=2\), the Anderson–Rubin contrast is \(\hat{\gamma}-\beta_0\hat{\pi}\). Assume the covariance estimate is appropriate for the sampling design and that the large-sample one-restriction Wald reference is being used.
The contrast equals
Its estimated variance is
Therefore the one-restriction Wald statistic is
Compared with \(\chi^2_1\), the p-value is approximately \(0.0112\), so the candidate value \(\beta_0=2\) is rejected at the 5% level under the stated assumptions. A report should give the contrast, covariance method, statistic, reference distribution, and p-value. This single test does not by itself provide the entire confidence set; that set is obtained by inverting the tests over candidate parameter values. Nor does rejection or non-rejection establish instrument validity.
Common Reporting Failures and Diagnostics
The main practical failure is to report a statistic without the information needed to interpret it. Before finalizing a result, check that the model and instrument set match the scientific design, that all excluded instruments are tested jointly, and that the covariance estimator reflects heteroskedasticity, clustering, or other dependence as appropriate. Confirm that the degrees of freedom and sample size match the analysis actually run. If estimates change materially across plausible instrument sets, report that sensitivity rather than choosing only the most favorable diagnostic.
- Wrong reasoning: “The statistic exceeds 10, so the instruments are strong and valid.” Correction: A threshold is conditional on its assumptions and criterion; relevance is not instrument validity.
- Wrong reasoning: “A robust Wald statistic is an F statistic, so a classical F threshold applies.” Correction: Identify the statistic’s reference distribution and use only a benchmark justified for that statistic and design.
- Wrong reasoning: “A large first-stage statistic means the treatment effect is estimated precisely.” Correction: Report outcome-effect uncertainty, including weak-IV-robust inference where appropriate.
- Wrong reasoning: “A wide robust confidence set is a failed analysis.” Correction: It may honestly reflect limited identification; describe its shape and implications.
A useful reporting sequence is to specify the first-stage equation and sample, present the joint excluded-instrument statistic and partial \(R^2\) where appropriate, state the covariance estimator and reference distribution, explain any critical-value comparison, and then report weak-IV-robust inference for the causal parameter. Keep the conclusion narrow: the diagnostic describes evidence about conditional first-stage relevance under the stated model, not proof of causal identification.
Name the treatment, excluded instruments, included controls, sample size, and joint null.
State whether inference is classical, heteroskedasticity-robust, or cluster-robust, and give the corresponding degrees of freedom or asymptotic reference distribution.
Give partial \(R^2\) when useful and treat any threshold as a conditional benchmark, not a universal verdict.
Report weak-IV-robust tests or confidence sets with their assumptions, while separating them from claims about instrument validity.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A first stage has \(SSR_R=500\), \(SSR_U=440\), \(q=3\), \(n=180\), and \(k_U=8\). Calculate the conventional F statistic and partial \(R^2\), and identify the finite-sample reference degrees of freedom under classical assumptions.
- Why is it not automatically valid to compare a heteroskedasticity-robust Wald statistic with a classical homoskedastic F critical value?
- A diagnostic exceeds 10. What can you conclude, and what can you not conclude?
- What should be reported if a weak-IV-robust confidence set is disconnected or unbounded?
Solutions. (1) The numerator is \((500-440)/3=20\), and the denominator is \(440/(180-8)=440/172=2.5581\). Thus \(F=20/2.5581=7.818\), with reference distribution \(F_{3,172}\) under the stated classical assumptions. Partial \(R^2=(500-440)/500=0.120\). (2) The robust Wald statistic and classical F statistic need not share a reference distribution or assumptions. The covariance estimator and reference method must be reported, and a threshold is appropriate only if justified for that combination. (3) You can state that the statistic exceeds 10, but whether that indicates adequate strength depends on the model, covariance assumptions, and benchmark criterion. It does not establish exclusion, independence, or instrument validity. (4) Report the full set and its confidence level, method, and assumptions; explain that disconnected or unbounded regions indicate substantial uncertainty or limited identification rather than hiding them in a single point estimate.