Measuring Strength With Multiple Instruments

Why Strength Is a Joint Question

When several excluded instruments are used for one endogenous regressor, their first-stage contribution is assessed as a group. The instruments may each have a small estimated coefficient, or their estimated coefficients may be imprecise when considered separately, while the instruments jointly explain meaningful variation in the endogenous regressor. Conversely, a small p-value for one coefficient does not by itself summarize the contribution of the full instrument set.

The relevant classical diagnostic is a joint test in the first-stage regression. Its result depends on the number of excluded instruments, the residual variation after fitting the model, and the covariance among the coefficient estimates. It is therefore not generally recoverable by simply counting which individual t-tests are significant.

The null is about the excluded-instrument coefficients conditional on the included regressors. It is not a test of whether the instruments are exogenous, satisfy the exclusion restriction, or identify a causal effect. Those are distinct questions. Nor does rejecting the null automatically establish that the instruments are sufficiently strong for reliable IV inference.

The Conventional Joint First-Stage F Statistic

Fit the unrestricted first stage with both \(X_i\) and \(Z_i\), and the restricted first stage with \(X_i\) alone. Let \(SSR_U\) and \(SSR_R\) be their residual sums of squares, respectively. The unrestricted model estimates \(k_U\) parameters in total, including the intercept if present. Thus its residual degrees of freedom are \(df=n-k_U\). The joint test imposes \(q\) restrictions, one for each excluded-instrument coefficient.

The numerator measures the reduction in residual sum of squares from adding the excluded instruments, averaged per restriction. The denominator estimates the residual variance in the unrestricted first stage. The statistic is large when the instruments collectively reduce residual variation by a substantial amount relative to the remaining unexplained variation.

The same test can be expressed using a partial \(R^2\), which measures the fraction of the restricted model's residual sum of squares removed by adding the excluded instruments: $$ R^2_{\text{partial}} = \frac{SSR_R-SSR_U}{SSR_R}. $$ Substituting this expression into the F formula gives $$ F = \frac{R^2_{\text{partial}}/q}{(1-R^2_{\text{partial}})/(n-k_U)}. $$ This identity shows why partial \(R^2\) alone does not determine the F statistic: the number of instruments and the residual degrees of freedom matter too. The earlier material on the first-stage F statistic and partial R-squared develops those measures; here the emphasis is on their joint interpretation when \(q\) is greater than one.

Worked Tests With Multiple Instruments

For an equivalent calculation based on a regression output, the joint Wald form is $$ F = \frac{1}{q}(R\hat{\beta}-r)'\left[R\widehat{\operatorname{Var}}(\hat{\beta})R'\right]^{-1}(R\hat{\beta}-r), $$ where \(\hat{\beta}\) collects the unrestricted first-stage estimates and \(R\hat{\beta}=r\) encodes the restrictions being tested. For \(H_0:\pi=0\), \(R\) selects the \(q\) excluded-instrument coefficients and \(r=0\). With the usual homoskedastic OLS covariance estimate, this gives the same partial F test as the residual-sum-of-squares formula. The covariance matrix matters: separate coefficient estimates and standard errors are not enough to reconstruct the joint test.

Degrees of Freedom, Controls, and Interpretation

The denominator degrees of freedom come from the unrestricted first stage, not from the restricted model. Suppose \(n=160\), the model includes an intercept and two covariates, and it has four excluded instruments. Then \(k_U=1+2+4=7\), so \(df=160-7=153\). The joint test has \(q=4\) numerator degrees of freedom. Counting the restrictions and parameters carefully prevents a common arithmetic error: the F statistic is not calculated by dividing the reduction in residual sum of squares by the total number of fitted parameters.

The calculation assumes that the restricted and unrestricted regressions use the same observations and the same included covariates, with the unrestricted regression adding the \(q\) excluded instruments. For the stated exact F reference distribution, the first-stage errors must be independent, normally distributed with mean zero and constant variance, conditional on the regressors. In standard large samples, a conventional F test may be used as an approximation under weaker conditions, but its covariance assumptions still matter. If errors are heteroskedastic or dependent in ways that affect the variance calculation, the usual homoskedastic F reference may not be appropriate.

What the Joint F Does Not Tell You

A joint test answers a narrow question: whether the excluded-instrument coefficients are collectively distinguishable from zero under the test's assumptions. A low p-value may result with a large sample even when the improvement in prediction is modest. The p-value therefore measures evidence against the zero-coefficient null, not the practical size of the first-stage contribution. Conversely, a non-significant result is not proof that every instrument is irrelevant; it indicates that this sample and test did not provide sufficient evidence against the joint null.

The number of excluded instruments also matters. The F statistic averages the reduction in residual sum of squares across \(q\) restrictions, while its denominator uses the unrestricted residual variance. Two analyses with the same partial \(R^2\) can therefore have different F statistics if they use different numbers of instruments or residual degrees of freedom. The F-statistic rule of ten is not a universal threshold; its limitations depend on the design and the inferential goal.

A useful comparison is between the joint test and three other summaries:

These measures are complementary. Report the joint statistic with its numerator and denominator degrees of freedom, the p-value, and, where useful, the partial \(R^2\). Describe the result as a first-stage diagnostic, not as a finding that the instruments are valid or that the causal effect is identified without qualification.

Diagnostics and Common Pitfalls

Before interpreting a reported joint F statistic, verify that the test corresponds to the intended first stage. Check that the excluded instruments are tested jointly, that included covariates remain in both restricted and unrestricted models, that the restriction count equals the number of excluded-instrument coefficients being tested, and that the residual degrees of freedom match the unrestricted model. Confirm that both regressions use the same observations. A mismatch in any of these can produce a statistic that answers a different question.

• Counting individual significant coefficients. The wrong reasoning is that two significant t-tests mean the instrument set is jointly strong, or that no significant t-test means it is jointly irrelevant. The correction is to test the full set of coefficient restrictions using the joint covariance information.
• Calling a small p-value “strong instruments.” The wrong reasoning treats statistical significance as a complete measure of predictive strength. The correction is to interpret the test alongside the partial \(R^2\), sample size, number of instruments, and the assumptions behind the statistic.
• Using a causal interpretation for a first-stage test. The wrong reasoning is that a significant first stage proves the instruments are valid. The correction is that this test concerns prediction of \(D_i\); it does not test the exclusion restriction or instrument exogeneity.
• Applying the conventional formula under any error structure. The wrong reasoning assumes the classical F reference applies regardless of heteroskedasticity or dependence. The correction is to ensure the test's variance assumptions fit the data and design.