The First-Stage F Statistic Under Weak Identification

Why the First-Stage F Statistic Matters

In an instrumental-variables model, the first stage asks whether the excluded instruments explain variation in an endogenous regressor after accounting for included controls. The joint F statistic for this question is often reported as a summary of instrument strength. To interpret it under weak identification, it helps to know more than its null distribution: under a specified alternative, its distribution is noncentral F, and its noncentrality parameter is the concentration parameter.

The derivation below is a finite-sample result for a homoskedastic Gaussian first stage conditional on the regressors. It shows precisely how the fitted signal from the excluded instruments competes with first-stage noise. The result is not, by itself, a universal rule for deciding whether instruments are sufficiently strong.

First-Stage Setup and Statistic

Let $x$ be an $n$-vector containing the endogenous regressor, $W$ an $n\times k$ matrix of included controls, and $Z$ an $n\times q$ matrix of excluded instruments. Assume the columns of $[W\ Z]$ are linearly independent, with $q\geq 1$. The first-stage regression is

x=W\gamma+Z\pi+v,

where $\pi$ is the $q$-vector of excluded-instrument coefficients. The joint first-stage null is $H_0:\pi=0$. Define the residual-maker for the controls by $M_W=I-W(W'W)^{-1}W'$, assuming $W$ has full column rank, and set $\widetilde Z=M_WZ$. The rank condition implies $\widetilde Z$ has rank $q$. Let $P_{\widetilde Z}=\widetilde Z(\widetilde Z'\widetilde Z)^{-1}\widetilde Z'$.

Definition: The conventional partial first-stage F statistic for testing $H_0:\pi=0$ is $$ F=\frac{(RSS_R-RSS_U)/q}{RSS_U/\nu}, $$ where $RSS_R$ and $RSS_U$ are the residual sums of squares in the restricted and unrestricted first-stage regressions, respectively, and $\nu=n-k-q$ is the unrestricted residual degrees of freedom. Equivalently, $F=(x'P_{\widetilde Z}x/q)/(x'M_{[W\,Z]}x/\nu)$, where $M_{[W\,Z]}$ projects onto the orthogonal complement of the columns of $[W\ Z]$.

The equivalent expression follows by first removing the component explained by $W$, then testing the remaining $q$ directions in $\widetilde Z$. The numerator measures the reduction in residual sum of squares from adding those directions; the denominator estimates the first-stage error variance using the unrestricted residual degrees of freedom.

Exact Noncentral F Distribution

Assume that, conditional on $W$ and $Z$, the first-stage errors satisfy $v\sim N(0,\sigma^2 I_n)$, with $\sigma^2>0$. This assumption supplies both homoskedasticity and Gaussianity. Define

\lambda=\frac{\pi'\widetilde Z'\widetilde Z\pi}{\sigma^2}.

This nonnegative quantity is the concentration parameter for the single-endogenous-regressor first stage in this fixed-design formulation. The exact distribution follows from orthogonal projections.

Theorem: Under the stated rank and conditional Gaussian assumptions, with $\nu=n-k-q>0$, $$ F\sim F_{q,\nu}(\lambda), $$ a noncentral F distribution with numerator degrees of freedom $q$, denominator degrees of freedom $\nu$, and noncentrality parameter $\lambda$. Under $H_0:\pi=0$, $\lambda=0$, so the distribution is central $F_{q,\nu}$.

Proof. Since $M_WW=0$, the partialled-out first stage is

M_Wx=\widetilde Z\pi+M_Wv.

The projection onto the added instrument space gives the numerator sum of squares:

x'P_{\widetilde Z}x =(M_Wx)'P_{\widetilde Z}(M_Wx).

Here $P_{\widetilde Z}\widetilde Z\pi=\widetilde Z\pi$, and $P_{\widetilde Z}M_Wv=P_{\widetilde Z}v$. Choose an orthonormal basis for the $q$-dimensional column space of $\widetilde Z$. In those coordinates, the projected error has $q$ independent $N(0,\sigma^2)$ components, and the projected mean has squared length $\pi'\widetilde Z'\widetilde Z\pi$. Therefore

\frac{x'P_{\widetilde Z}x}{\sigma^2}\sim\chi^2_q(\lambda).

The unrestricted residual sum of squares is $x'M_{[W\,Z]}x$. The projection $M_{[W\,Z]}$ has rank $\nu$ and annihilates both $W\gamma$ and $Z\pi$. Thus

\frac{x'M_{[W\,Z]}x}{\sigma^2}\sim\chi^2_\nu.

The two projections are onto orthogonal subspaces: the first lies in the part of the column space of $[W\ Z]$ added by $Z$, and the second is orthogonal to all of that column space. For a multivariate normal vector, orthogonal projected components are independent; hence these two chi-square variables are independent. Dividing each by its degrees of freedom and taking their ratio gives the stated noncentral F distribution. If $\pi=0$, then $\lambda=0$, and the numerator chi-square is central. $\square$

What the Noncentrality Parameter Measures

The expression $\lambda=\pi'\widetilde Z'\widetilde Z\pi/\sigma^2$ compares the first-stage signal attributable to the excluded instruments with the error variance. Its numerator is the squared length of the conditional mean $\widetilde Z\pi$, so it accounts both for the instrument coefficients and for the amount and geometry of instrument variation left after controls are removed. A larger $\lambda$ shifts probability toward larger F values, but the distribution also depends on $q$ and $\nu$.

With a fixed design and fixed $\pi$, increasing the scale of the instrument variation left after controls generally increases $\lambda$. In a sequence of samples, if the residualized instrument cross-product grows proportionally to $n$, fixed nonzero $\pi$ makes $\lambda$ grow proportionally to $n$. By contrast, if $\pi_n=c/\sqrt{n}$ and $\widetilde Z'\widetilde Z$ grows proportionally to $n$, the noncentrality can remain bounded. This is the local-to-zero regime: the first-stage signal does not become arbitrarily large relative to noise as the sample grows.

Worked Example: One Excluded Instrument

Suppose there is one excluded instrument, $\nu=48$, and after removing the controls its sum of squares is $\widetilde Z'\widetilde Z=40$. Let the first-stage coefficient be $\pi=0.20$, and let the error variance be $\sigma^2=0.25$. Find the F distribution under this alternative.

For one instrument, the noncentrality parameter is

\lambda=\frac{\pi^2\widetilde Z'\widetilde Z}{\sigma^2} =\frac{(0.20)^2(40)}{0.25} =\frac{0.04(40)}{0.25} =\frac{1.6}{0.25} =6.4.

Consequently, $F\sim F_{1,48}(6.4)$. If the null $\pi=0$ were true instead, $\lambda=0$ and $F\sim F_{1,48}$. The alternative distribution is noncentral, not central: the same numerator and denominator degrees of freedom do not determine the distribution without the value of $\lambda$.

Worked Example: Two Instruments and the Mean F Statistic

Suppose $q=2$, $\nu=96$, $\sigma^2=0.25$, and

\widetilde Z'\widetilde Z= \begin{pmatrix}100&0\\0&25\end{pmatrix}, \qquad \pi= \begin{pmatrix}0.10\\0.20\end{pmatrix}.

First calculate the signal:

\pi'\widetilde Z'\widetilde Z\pi =(0.10)^2(100)+(0.20)^2(25) =1+1=2.

Thus $\lambda=2/0.25=8$, and $F\sim F_{2,96}(8)$. The noncentral F mean can also be found explicitly. If $U\sim\chi^2_q(\lambda)$ and $V\sim\chi^2_\nu$ are independent, then $F=(U/q)/(V/\nu)$. For $\nu>2$, $E[U]=q+\lambda$ and $E[1/V]=1/(\nu-2)$, giving

E[F]=\frac{\nu}{q}E[U]E[1/V] =\frac{\nu}{\nu-2}\left(1+\frac{\lambda}{q}\right).

Here this is

E[F]=\frac{96}{94}\left(1+\frac{8}{2}\right) =\frac{96}{94}(5) =\frac{240}{47}\approx 5.106.

This is the mean under the specified alternative, not a cutoff and not the value the statistic must take in any particular sample.

Result: For $F\sim F_{q,\nu}(\lambda)$ with $\nu>2$, $E[F]=\dfrac{\nu}{\nu-2}\left(1+\dfrac{\lambda}{q}\right)$. To verify the formula, write $F=(U/q)/(V/\nu)$ for independent $U\sim\chi^2_q(\lambda)$ and $V\sim\chi^2_\nu$. The noncentral chi-square mean is $E[U]=q+\lambda$. The central chi-square density gives $E[1/V]=1/(\nu-2)$: integrating its density, the ratio of the integrals for $E[1/V]$ and $1$ is $1/(\nu-2)$. Independence then yields the displayed expression. The condition $\nu>2$ is needed for this expectation to be finite.

Worked Example: Fixed Strength Versus Local-to-Zero Strength

Consider a sequence of designs with two residualized instruments satisfying $\widetilde Z_n'\widetilde Z_n=nI_2$, error variance $\sigma^2=1$, and denominator degrees of freedom $\nu_n=n-k-2$. Compare fixed first-stage coefficients $\pi=(0.10,0.20)'$ with local-to-zero coefficients $\pi_n=(1,2)'/\sqrt n$.

Under fixed coefficients, the concentration parameter is

\lambda_n=\pi'\widetilde Z_n'\widetilde Z_n\pi =n\big((0.10)^2+(0.20)^2\big) =0.05n.

It grows with $n$. Under the local-to-zero sequence,

\lambda_n=\pi_n'\widetilde Z_n'\widetilde Z_n\pi_n =\frac{1}{n}(1,2)(nI_2)\begin{pmatrix}1\\2\end{pmatrix} =1+4=5.

The noncentrality stays at 5 even as the sample grows. The corresponding statistics have distributions $F_{2,\nu_n}(0.05n)$ and $F_{2,\nu_n}(5)$, respectively. This comparison shows why a growing sample does not by itself guarantee that the first-stage signal becomes strong: coefficients that shrink at the $1/\sqrt n$ rate can keep the concentration parameter bounded.

Scope, Interpretation, and a Common Pitfall

The exact distribution above requires a correctly specified linear first stage, full-rank regressors, and conditionally Gaussian errors with common variance. Without Gaussianity, the projection argument does not give this exact finite-sample noncentral F law. Heteroskedasticity also changes the appropriate diagnostic construction; the conventional homoskedastic F statistic should not automatically be assigned this distribution in that setting.

A common mistake is to treat a particular observed F value as a universal measure of strength, independent of the number of instruments, degrees of freedom, and inferential objective. The noncentral F derivation corrects that reasoning: the statistic has a distribution indexed by $q$, $\nu$, and $\lambda$, and the noncentrality summarizes the first-stage signal relative to error variance. A single cutoff cannot be justified from the noncentrality formula alone. In particular, the statistic’s null distribution describes a test of $\pi=0$; it does not by itself establish that conventional 2SLS inference behaves well under weak identification.

The practical value of the derivation is interpretive. It connects a joint first-stage test to the concentration parameter, clarifies why residualized instrument variation and error variance both matter, and explains how local-to-zero sequences can yield persistently limited first-stage information. The noncentral distribution is a finite-sample benchmark under its assumptions, not a substitute for checking whether those assumptions fit the application.

Check Your Understanding

Answer each question using the definitions and derivation above.

Why does the noncentrality parameter use $\widetilde Z'\widetilde Z$, rather than $Z'Z$, when the first stage includes controls?
Under what assumptions does the conventional first-stage statistic have an exact noncentral F distribution?
For $q=3$, $\nu=40$, and $\lambda=0$, what is the distribution of the first-stage F statistic?
If residualized instrument variation grows proportionally to $n$ while $\pi_n=c/\sqrt n$, why can the concentration parameter remain bounded?
Why does the noncentral F result not justify a universal first-stage F cutoff?