Weak Identification Along Parameter Directions

Why Identification Has Directions

With one endogenous regressor, instrument relevance is often summarized by a scalar first-stage measure. With several endogenous regressors, a single summary can hide an important feature: the instruments may predict some combinations of regressors well and others poorly. Since the structural coefficients multiply those regressors in combinations, weak prediction in even one direction can mean weak identification for a corresponding combination of coefficients.

The geometry comes from the first-stage coefficient matrix. Its rank determines whether every coefficient direction is identified from the instruments. Its singular values show how close the matrix is to losing rank and which directions are responsible. These are population properties; in a sample, they must be interpreted relative to the sampling uncertainty in estimating the first stage.

First-Stage Geometry

Let $x_i$ be a $q$-vector of endogenous regressors and $z_i$ a $k$-vector of excluded instruments, with $k\geq q$. After accounting for included exogenous regressors, suppose the population linear first stage is

x_i=\Pi' z_i+v_i,\qquad E[z_i v_i']=0,

where $\Pi$ is a $k\times q$ matrix. Write $Q_{zz}=E[z_i z_i']$, and assume $Q_{zz}$ is finite and positive definite. The first-stage matrix expressed in instrument-whitened coordinates is

R=Q_{zz}^{1/2}\Pi.

Whitening accounts for the scales and linear correlations among instruments. In particular, changing the units of the instruments does not, by itself, change the relevance geometry represented by $R$. The sizes of singular values can still depend on the units used for the endogenous regressors, so numerical comparisons require a stated scaling convention for those regressors.

Definition: The $q$ singular values of $R$ are denoted $s_1(R)\geq \cdots \geq s_q(R)\geq 0$. A unit vector $a\in\mathbb R^q$ describes a direction of change in the endogenous-regressor coefficient space. The instruments' first-stage signal in that direction has magnitude $\|Ra\|$. A direction $a$ is exactly unidentified by the excluded instruments when $\Pi a=0$, equivalently $Ra=0$.

The singular value decomposition makes the direction-by-direction interpretation explicit. There are orthonormal right singular vectors $a_1,\ldots,a_q$ and orthonormal left singular vectors $b_1,\ldots,b_q$ (with the left vectors in $\mathbb R^k$) such that $Ra_j=s_j(R)b_j$ when $s_j(R)>0$. Thus $a_j$ is a direction in the endogenous-regressor space, and $s_j(R)$ is its first-stage signal magnitude. The smallest singular value is the weakest signal among all unit directions.

Theorem: The excluded instruments provide nonzero first-stage signal in all $q$ endogenous-regressor directions in the population if and only if $\operatorname{rank}(\Pi)=q$, equivalently $s_q(R)>0$; this is the IV rank condition for identifying all structural coefficients when the structural model and instrument validity and exclusion conditions also hold. Moreover, $$ s_q(R)=\min_{\|a\|=1}\|Ra\|. $$ If $s_q(R)=0$, at least one nonzero direction is exactly unidentified. If $s_q(R)$ is positive but small relative to the sampling uncertainty in the first stage, the corresponding direction is weakly identified.

Proof. Since $Q_{zz}^{1/2}$ is nonsingular, left multiplication by it does not change rank or null space: $\operatorname{rank}(R)=\operatorname{rank}(\Pi)$, and $Ra=0$ if and only if $\Pi a=0$. By the singular value decomposition, for any unit vector $a=\sum_{j=1}^q c_j a_j$, with $\sum_j c_j^2=1$,

\|Ra\|^2=\sum_{j=1}^q s_j(R)^2c_j^2\geq s_q(R)^2.

Equality holds for a right singular vector corresponding to $s_q(R)$. Hence the stated minimum is $s_q(R)$, and it is zero exactly when $R$, and therefore $\Pi$, has deficient column rank. $\square$

When Is a Direction Weak?

Rank is a population yes-or-no condition. Weak identification is a comparison between the directional first-stage signal and the uncertainty with which that signal is estimated. Under fixed-dimensional, independent and identically distributed sampling, suppose $Q_{zz}$ is positive definite and $E\|z_i v_i'\|_F^2<\infty$, where $\|\cdot\|_F$ denotes the Frobenius norm. The sample first-stage coefficient matrix then has error of order $n^{-1/2}$. Consequently, a sequence of first-stage singular values of order $n^{-1/2}$ has signal on the same scale as first-stage estimation noise.

Directional strength under a sequence of models: Suppose the population first-stage matrices $R_n$ may change with sample size, and the first-stage estimation error is $O_p(n^{-1/2})$ in fixed dimensions. A singular direction with $\sqrt n\,s_j(R_n)\to\infty$ has signal larger than that error scale. If $\sqrt n\,s_j(R_n)$ remains bounded, its signal is on the error scale and the direction is weak under the usual nondegenerate-noise conditions. If $s_j(R_n)=0$, the direction is exactly unidentified.

The last classification is a scale comparison, not a complete distributional result. A precise weak-identification limit also depends on the joint sampling behavior of the first-stage estimates and the structural disturbances. In particular, the finite-variance condition above matters: finite second moments of $z_i$ and $v_i$ separately do not imply a finite variance for their product $z_i v_i'$. For a triangular array, one needs suitable uniform moment and covariance conditions to justify the same error scale.

Worked Examples

Worked Example: One Strong and One Weak Direction

Take $Q_{zz}=I_2$ and $$ \Pi=R= \begin{pmatrix} 1&0\\ 0&0.04 \end{pmatrix}. $$ The singular values are $1$ and $0.04$, with right singular vectors $e_1=(1,0)'$ and $e_2=(0,1)'$. For a unit direction $a=(a_1,a_2)'$, the signal is $$ \|Ra\|=\sqrt{a_1^2+0.04^2a_2^2}. $$

Along $e_1$, the signal is $1$; along $e_2$, it is $0.04$. Thus the first coefficient direction is much more strongly supported by the first stage, while the second is weak relative to it. At $n=100$, the weak-direction signal multiplied by $\sqrt n$ is $10(0.04)=0.4$, whereas at $n=10{,}000$ it is $100(0.04)=4$. The same population first stage can therefore be weak at one sample size and more precisely estimated at a larger one. These calculations describe the signal scale, not a universal cutoff for an inferential procedure.

Worked Example: Strong Individual Columns but a Weak Combination

Consider $$ R=\Pi= \begin{pmatrix} 0.51&0.49\\ 0.49&0.51 \end{pmatrix}, \qquad Q_{zz}=I_2. $$ Each column has squared length $0.51^2+0.49^2=0.5002$, so both individual regressors have substantial first-stage signal when considered separately. But consider the orthogonal unit directions $$ a_+=\frac{1}{\sqrt 2}(1,1)',\qquad a_-=\frac{1}{\sqrt 2}(1,-1)'. $$

Direct multiplication gives $$ Ra_+=\frac{1}{\sqrt 2}(1,1)'=a_+,\qquad Ra_-=\frac{1}{\sqrt 2}(0.02,-0.02)'=0.02a_-. $$ Thus $a_+$ has signal $1$, while $a_-$ has signal only $0.02$. Because these orthogonal vectors are eigenvectors of this symmetric matrix, they are also its right singular vectors; the singular values are $1$ and $0.02$. In this example the difference between the two regressor directions is weak even though neither column, viewed on its own, looks weak. Looking only at marginal first-stage strengths would miss the poorly identified combination.

Worked Example: Exact Rank Deficiency

Let $$ R=\Pi= \begin{pmatrix} 1&2\\ 2&4 \end{pmatrix}, \qquad Q_{zz}=I_2. $$ The second column is twice the first, so the matrix has rank one. The unit direction $$ a=\frac{1}{\sqrt 5}(-2,1)' $$ satisfies $$ Ra=\frac{1}{\sqrt 5} \begin{pmatrix} -2+2\\ -4+4 \end{pmatrix} =\begin{pmatrix}0\\0\end{pmatrix}. $$ Therefore $s_2(R)=0$, and the instruments provide no first-stage signal for this direction. Increasing the sample size can improve estimation of the nonzero first-stage direction, but it cannot create population relevance in this exact null direction. This is a rank problem, not merely a small-sample imprecision problem.

Why the Sampling Scale Matters

The signal-versus-noise comparison can be justified directly for the usual sample first stage. Suppose observations are independent and identically distributed, $E[z_i v_i']=0$, $Q_{zz}$ is positive definite, and $E\|z_i v_i'\|_F^2<\infty$. Define

\widehat Q_{zz}=\frac{1}{n}\sum_{i=1}^n z_i z_i', \qquad \widehat\Pi=\widehat Q_{zz}^{-1}\frac{1}{n}\sum_{i=1}^n z_i x_i'.

With $x_i' = z_i'\Pi+v_i'$, this gives

\widehat\Pi-\Pi =\widehat Q_{zz}^{-1}\frac{1}{n}\sum_{i=1}^n z_i v_i'.

Derivation. By the law of large numbers, $\widehat Q_{zz}\xrightarrow{p}Q_{zz}$, so its inverse is $O_p(1)$. The summands $z_i v_i'$ have mean zero and finite second moment. Independence therefore gives

E\left\|\frac{1}{n}\sum_{i=1}^n z_i v_i'\right\|_F^2 =\frac{1}{n}E\|z_i v_i'\|_F^2.

Markov's inequality implies that the sample average is $O_p(n^{-1/2})$. It follows that $\widehat\Pi-\Pi=O_p(n^{-1/2})$. If $Q_{zz}$ is known and $R=Q_{zz}^{1/2}\Pi$, the corresponding estimated whitened matrix has the same order of error. The singular-value perturbation inequality

|s_j(A)-s_j(B)|\leq \|A-B\|_{\mathrm{op}}

then implies that each estimated singular value differs from its population counterpart by $O_p(n^{-1/2})$. The inequality applies to matrices of the same dimensions and follows from the minimum-maximum characterization of singular values: perturbing $A$ to $B$ changes $\|Aa\|$ by at most $\|(A-B)a\|\leq\|A-B\|_{\mathrm{op}}$ for every unit vector $a$. Thus a population singular value much smaller than, or on the order of, the first-stage sampling error cannot be distinguished from zero reliably by this scale comparison alone. $\square$

When $Q_{zz}$ is estimated rather than known, its estimation error must also be accounted for in the whitening step. Under fixed dimensions, finite moments sufficient for $\widehat Q_{zz}-Q_{zz}=O_p(n^{-1/2})$, and eigenvalues of $Q_{zz}$ bounded away from zero, the matrix square root is locally well behaved, so estimating it does not change the $n^{-1/2}$ order. For weak-instrument sequences, analogous uniform conditions are needed; the fixed-model argument by itself does not establish a triangular-array limit.

Common Mistake and Practical Interpretation

Mistake: “Each endogenous regressor has a strong first stage, so all structural coefficients are well identified.” That conclusion treats the columns separately and ignores combinations of columns. As the second example shows, two substantial columns can be nearly aligned, leaving their difference very poorly predicted.

Correction: Examine the smallest singular values and their right singular vectors, using a stated scaling for the endogenous regressors. The right singular vector identifies which coefficient-space combination is at issue. Then compare its first-stage signal with the sampling uncertainty, using assumptions that actually control the variance of the first-stage moments. Singular values are a geometric diagnostic, not by themselves a complete weak-instrument test or a guarantee about the validity of a particular inference procedure.

This distinction matters because a model may contain both well-supported and poorly supported coefficient directions. A single global label such as “strong instruments” can conceal that heterogeneity. Directional analysis clarifies what the first stage supports and where inferential difficulty is concentrated.

Check Your Understanding

Use the rank, singular-value, and sampling-scale ideas developed above.

Why does multiplying the first-stage matrix on the left by $Q_{zz}^{1/2}$ preserve its rank and null space?
For $R=\begin{pmatrix}1&0\\0&0.1\end{pmatrix}$, identify the weakest right singular direction and its signal magnitude.
How can two first-stage columns each have substantial length while a linear combination of them is weakly predicted?
What does $s_q(R)=0$ imply that differs from a small but positive $s_q(R)$?
Why is $E\|z_i v_i'\|_F^2<\infty$ relevant to claiming an $n^{-1/2}$ error rate for the sample first stage?