Effective F Statistics Under Heteroskedasticity

Why a Robust First-Stage Diagnostic Is Needed

A first-stage statistic measures how strongly the excluded instruments predict an endogenous regressor after accounting for included exogenous controls. Under homoskedastic errors, the familiar first-stage F statistic can be compared with critical values developed for that setting. If the first-stage errors have unequal variances, however, the usual homoskedastic covariance estimate may be wrong. The resulting F statistic can misstate the evidence about instrument strength.

For a scalar endogenous regressor, the effective F statistic replaces that homoskedastic covariance estimate with a heteroskedasticity-robust one. Its value is then compared with a critical value calibrated for a chosen weak-instrument criterion. This is a diagnostic for strength, not a test that establishes instrument validity and not a replacement for inference methods designed to remain reliable under weak identification.

First-Stage Model and Definition

Let $D_i$ be the scalar endogenous regressor, $Z_i$ a vector of $L$ excluded instruments, and $X_i$ a vector of included exogenous controls. Write the first stage as

D_i = X_i'\delta + Z_i'\pi + v_i,\qquad i=1,\ldots,n.

The vector $\pi$ contains the coefficients on the excluded instruments. The relevant first-stage null is $H_0:\pi=0$: conditional on $X_i$, the excluded instruments have no linear predictive content for $D_i$. A strong first stage requires evidence not only that these coefficients are jointly nonzero, but that their predictive strength is adequate for the intended IV analysis.

Estimate the first stage by ordinary least squares and let $\hat{\pi}$ be the estimated excluded-instrument coefficient vector. Let $\widehat V_{\!rob}(\hat{\pi})$ be its heteroskedasticity-robust estimated covariance matrix. The robust Wald statistic for $H_0:\pi=0$ is

W_{\!rob} = \hat{\pi}'\left[\widehat V_{\!rob}(\hat{\pi})\right]^{-1}\hat{\pi}.

Definition: For $L$ excluded instruments and one endogenous regressor, the heteroskedasticity-robust effective F statistic is $$ F_{\mathrm{eff}}=\frac{1}{L}\hat{\pi}'\left[\widehat V_{\!rob}(\hat{\pi})\right]^{-1}\hat{\pi}. $$ It is the robust joint Wald statistic for the excluded-instrument coefficients divided by the number of restrictions.

For example, with independent observations and an HC0 covariance estimator, let $A_i$ collect the included controls and excluded instruments in the first-stage regression, and let $\hat v_i$ be its residuals. The sandwich covariance estimate for the full coefficient vector is

\widehat V_{\!HC0} = \left(\sum_{i=1}^n A_iA_i'\right)^{-1} \left(\sum_{i=1}^n A_iA_i'\hat v_i^2\right) \left(\sum_{i=1}^n A_iA_i'\right)^{-1}.

The covariance matrix for $\hat{\pi}$ is the block of this matrix corresponding to the excluded instruments. Other HC estimators adjust the finite-sample scaling; the chosen estimator should match the intended procedure. This construction allows the conditional variance of $v_i$ to differ across observations. It still requires appropriate independence or other dependence assumptions for the covariance estimator being used. A heteroskedasticity-only estimator does not automatically accommodate clustering or serial correlation.

How the Statistic Is Calibrated

Under standard regularity conditions, the robust Wald statistic for $H_0:\pi=0$ is asymptotically distributed as chi-square with $L$ degrees of freedom when the null holds. Consequently, $F_{\mathrm{eff}}=W_{\!rob}/L$ is not generally an ordinary $F$ random variable with the usual finite-sample degrees of freedom. Dividing by $L$ puts the joint statistic on a per-restriction scale; it does not make its distribution an ordinary F distribution.

Weak-instrument calibration asks a different question from testing $H_0:\pi=0$: is the observed first-stage strength large enough to limit a specified form of IV distortion? Robust effective-F critical values are constructed for explicit criteria, such as a maximum relative-bias target or a maximum size-distortion target. The relevant threshold depends on the chosen criterion and the instrument setup, including the number of excluded instruments. Use a published calibration or software implementation that matches the robust statistic and the assumed error setting.

The homoskedastic Stock–Yogo critical values are not automatically valid for this heteroskedasticity-robust statistic. Nor is a universal rule such as “F above 10” a substitute for the appropriate calibration. State the criterion used, the corresponding critical value, and whether the computed statistic exceeds it. Passing a threshold indicates that the chosen criterion is met under the calibration’s assumptions; it does not certify that the instruments satisfy the exclusion restriction.

Worked Example: Computing Effective F From a Robust Covariance Matrix

Suppose a hypothetical first-stage regression has two excluded instruments. The estimated coefficient vector and HC covariance estimate for those coefficients are

\hat{\pi} = \begin{pmatrix}0.30\\0.20\end{pmatrix}, \qquad \widehat V_{\!rob}(\hat{\pi}) = \begin{pmatrix}0.010&0.002\\0.002&0.006\end{pmatrix}.

Assume the first-stage regression includes the prespecified exogenous controls, the observations are independent, and the reported covariance matrix is the heteroskedasticity-robust estimate from that regression. The determinant of the covariance matrix is $0.010(0.006)-0.002^2=0.000056$, so its inverse is approximately

\left[\widehat V_{\!rob}(\hat{\pi})\right]^{-1} = \begin{pmatrix} 107.143&-35.714\\ -35.714&178.571 \end{pmatrix}.

The robust Wald statistic and effective F are therefore

W_{\!rob} = \frac{0.006(0.30)^2-2(0.002)(0.30)(0.20)+0.010(0.20)^2}{0.000056} = \frac{0.000700}{0.000056} = 12.50,

F_{\mathrm{eff}}=\frac{12.50}{2}=6.250.

The joint robust Wald statistic is 12.50, and the effective F is 6.250. Neither number alone establishes whether strength meets a specified weak-instrument criterion: that requires the matching calibrated critical value. The coefficients’ units depend on how the instruments and $D_i$ are measured, but the Wald statistic is invariant to a nonsingular linear rescaling or re-expression of the excluded instruments when the covariance matrix is transformed consistently.

Why Robustness Changes the Answer

The homoskedastic and robust procedures can produce different statistics even when they use the same estimated first-stage coefficients. They differ in the estimated covariance matrix, which determines how precisely the joint coefficient vector is measured. Under heteroskedasticity, a homoskedastic covariance matrix can overstate or understate precision; there is no general direction to the discrepancy.

Worked Example: Same Coefficients, Different Covariance Estimate

Continue with two instruments and $\hat{\pi}=(0.30,0.20)'$, but suppose the robust covariance estimate is diagonal:

\widehat V_{\!rob}(\hat{\pi}) = \begin{pmatrix}0.040&0\\0&0.010\end{pmatrix}.

For this hypothetical output, suppose the first-stage regression and the independence assumptions are appropriate, and the covariance matrix was calculated using a heteroskedasticity-robust estimator. Its inverse is diagonal with entries 25 and 100. Thus

W_{\!rob} = \frac{(0.30)^2}{0.040} + \frac{(0.20)^2}{0.010} = 2.25+4.00 = 6.25, \qquad F_{\mathrm{eff}}=\frac{6.25}{2}=3.125.

If a homoskedastic calculation using a different covariance estimate had reported a conventional first-stage F of 15.4, that would not contradict this result: the two statistics use different covariance estimates. Under the stated robust calculation, the effective F is 3.125. The difference illustrates why choosing a robust covariance estimator matters; it does not by itself prove which statistic is appropriate without checking the error structure and the assumptions behind each estimator.

Using a Calibrated Threshold

A diagnostic has a clear interpretation only when its calibration target is specified in advance. For example, an analyst might choose a maximum relative-bias criterion and use the corresponding robust effective-F critical value for the number of excluded instruments. A different target, such as a bound on size distortion, can imply a different threshold. The choice should be tied to the inferential purpose, not selected after seeing the statistic.

Worked Example: Interpreting a Calibration Result

Suppose a hypothetical application has two excluded instruments and produces $F_{\mathrm{eff}}=6.250$, as in the first example. For illustration only, assume a calibration procedure matched to the chosen error assumptions and a prespecified strength criterion reports a critical value of 8.000. This number is stipulated for the example; it is not a general critical value for two instruments.

The comparison is

$$ 6.250 < 8.000. $$

The statistic does not exceed this particular threshold. The appropriate conclusion is that the first stage does not meet the selected calibrated strength criterion. This does not show that the instruments are invalid, and it does not prove that the IV estimator has a particular amount of bias. It signals that the chosen guarantee or distortion target is not supported by this diagnostic under its calibration assumptions. A different prespecified criterion could lead to a different threshold and comparison.

A Short Justification of the Construction

The effective F is built from a joint Wald measure because instrument strength is a joint property when there are multiple excluded instruments. To see the role of the robust covariance matrix, suppose the first-stage coefficient estimator is approximately normal:

\hat{\pi}\ \dot{\sim}\ N\!\left(\pi,\widehat V_{\!rob}(\hat{\pi})\right).

Under $H_0:\pi=0$, the standardized quadratic form has an asymptotic chi-square distribution with $L$ degrees of freedom. The argument is that if $V$ is the covariance matrix and the estimator is approximately $N(0,V)$, then $V^{-1/2}\hat{\pi}$ is approximately a standard $L$-dimensional normal vector. Its squared length is therefore approximately chi-square:

\hat{\pi}'V^{-1}\hat{\pi} = \left(V^{-1/2}\hat{\pi}\right)' \left(V^{-1/2}\hat{\pi}\right) \ \dot{\sim}\ \chi^2_L.

Replacing $V$ with a consistent robust covariance estimate gives the robust Wald statistic asymptotically. Dividing by $L$ yields the effective F scale. This justification concerns the joint test under the null; weak-instrument calibration concerns the statistic’s relationship to IV bias or size distortion under departures from the null. That additional calibration is why the effective F should be paired with purpose-specific critical values rather than interpreted through the null distribution alone.

Assumptions and Common Pitfalls

The first-stage model must include the controls used in the structural IV specification, and the excluded instruments must have sufficient variation after accounting for those controls. The estimated covariance matrix must be appropriate for the sampling dependence: HC standard errors address heteroskedasticity across independent observations, while clustered or serially correlated data require a covariance estimator and calibration suited to that dependence. Robustness of the covariance estimate does not repair misspecification of the conditional mean or instrument endogeneity.

Using homoskedastic critical values automatically. The statistic and its calibration must match. Stock–Yogo critical values derived for homoskedastic errors are not automatically valid for the robust effective F.
Calling effective F an ordinary F statistic. The robust Wald statistic has an asymptotic chi-square reference under the joint null; dividing by $L$ does not generally give an exact $F$ distribution.
Treating a large statistic as proof of validity. Strength concerns prediction of the endogenous regressor. Instrument exogeneity and the exclusion restriction are separate requirements.
Assuming one cutoff works for every purpose. A threshold depends on the strength criterion and setup. Report the calibration target and use a compatible critical value.
Ignoring the dependence structure. Heteroskedasticity-robust does not mean robust to arbitrary clustering, serial correlation, or other dependence.

Key takeaway: For one endogenous regressor, compute the robust joint Wald statistic for the excluded first-stage coefficients and divide by the number of instruments. Diagnose strength by comparing that effective F with a robust critical value chosen for a stated bias or size-distortion criterion.

Check Your Understanding

Answer each question using the distinction between the statistic, its covariance estimate, and its calibration.

With three excluded instruments, a robust Wald statistic is 18.0. What is the effective F statistic?
Why does a heteroskedasticity-robust effective F not generally have the usual finite-sample $F_{L,df}$ distribution?
What does it mean if an effective F statistic falls below the critical value for a prespecified maximum relative-bias criterion?
Can a large effective F establish that the exclusion restriction is true? Explain.
What additional issue must be addressed if observations are clustered, even when the analyst has used an HC covariance estimator?