The Matrix Formula for Two-Stage Least Squares

From Scalar 2SLS to Matrices

Scalar two-stage least squares can be written as a sequence of regressions when there is one endogenous treatment. Matrix notation expresses the same idea for several regressors and makes the role of covariates explicit: every included exogenous covariate is a regressor and also an instrument. The key operation is projection onto the column space spanned by all instruments.

Let $y$ be the $n \times 1$ vector of observed outcomes. Let $X$ be the $n \times K$ matrix of regressors in the structural equation, including any endogenous regressors and any included exogenous covariates. Let $Z$ be the $n \times L$ instrument matrix. It contains the included exogenous covariates, an intercept if the model has one, and the excluded instruments. The structural model is

y = X\beta + u,

where $\beta$ is a $K \times 1$ coefficient vector and $u$ is the $n \times 1$ structural error vector. A sample estimator requires the relevant matrices to have full rank. In particular, $Z'Z$ must be invertible, and $X'P_ZX$ must be invertible, where $P_Z$ is the projection matrix defined below. These are sample identification conditions: the instruments must provide enough distinct variation to identify the coefficients in $X$.

Definition: The projection matrix onto the column space of $Z$ is $$ P_Z = Z(Z'Z)^{-1}Z'. $$ It maps each $n \times 1$ vector to its least-squares fitted component using the columns of $Z$. The fitted regressor matrix is $\widehat{X}=P_ZX$.

For causal interpretation, the instruments must also satisfy the substantive IV assumptions. In the structural model, a central moment condition is that the instruments are uncorrelated with the structural error, expressed in a population formulation as $E[Z_i u_i]=0$. Excluded instruments must affect the outcome through the modeled endogenous regressors, rather than through another path; included exogenous covariates appear directly in the outcome equation as well as in $Z$. These conditions are not established merely by calculating a matrix formula. The formula gives an estimator; assumptions about the assignment process and causal pathways justify its use.

Deriving the Matrix Estimator

In the second stage, regress $y$ on the fitted regressors $\widehat{X}=P_ZX$. The ordinary least-squares coefficient formula for this regression is $(\widehat{X}'\widehat{X})^{-1}\widehat{X}'y$, provided that its cross-product matrix is invertible. The projection matrix is symmetric and idempotent: $P_Z'=P_Z$ and $P_ZP_Z=P_Z$. Substituting $\widehat{X}=P_ZX$ gives

\begin{aligned} \widehat{\beta}_{2SLS} &= (\widehat{X}'\widehat{X})^{-1}\widehat{X}'y \\ &= (X'P_Z'P_ZX)^{-1}X'P_Z'y \\ &= (X'P_ZX)^{-1}X'P_Zy. \end{aligned}

The same expression follows from the 2SLS criterion. Choose $\beta$ to minimize the squared length of the part of $y-X\beta$ that lies in the instrument space:

(y-X\beta)'P_Z(y-X\beta).

Because $P_Z$ is symmetric, differentiating this quadratic with respect to $\beta$ gives the normal equation $X'P_Z(y-X\beta)=0$. Rearranging yields $X'P_ZX\beta=X'P_Zy$, and solving gives the matrix formula above. The inverse exists only when the stated rank condition holds.

Key formula: With $X$ containing all structural regressors and $Z$ containing the included exogenous covariates and excluded instruments, $$ \widehat{\beta}_{2SLS}=(X'P_ZX)^{-1}X'P_Zy, \qquad P_Z=Z(Z'Z)^{-1}Z'. $$

If the model includes an intercept, its column belongs in both $X$ and $Z$. If $W$ is an included exogenous covariate, its column also belongs in both matrices. Excluded instruments belong in $Z$, but not in $X$ unless they are themselves part of the structural equation. This construction keeps the second-stage regressors and the instrument space aligned with the structural model.

Worked Example: Building the Matrices

Suppose a hypothetical study records an outcome $Y$, an endogenous treatment $D$, an exogenous covariate $W$, and two excluded instruments $A$ and $B$ for five individuals. The observed values are:

Individual	$Y$	$D$	$W$	$A$	$B$
1	4	1	0	1	0
2	7	2	1	0	1
3	5	1	1	1	1
4	9	3	0	0	1
5	6	2	1	1	0

The structural equation includes an intercept, treatment, and covariate. Therefore $K=3$, and the regressor matrix is

X= \begin{bmatrix} 1&1&0\\ 1&2&1\\ 1&1&1\\ 1&3&0\\ 1&2&1 \end{bmatrix}, \qquad y= \begin{bmatrix} 4\\7\\5\\9\\6 \end{bmatrix}.

The instrument matrix contains the intercept, both excluded instruments, and the included covariate, so $L=4$:

Z= \begin{bmatrix} 1&1&0&0\\ 1&0&1&1\\ 1&1&1&1\\ 1&0&1&0\\ 1&1&0&1 \end{bmatrix}.

Here $X$ has $5$ rows and $3$ columns, while $Z$ has $5$ rows and $4$ columns. Substitution into the projection formula gives $P_Z=Z(Z'Z)^{-1}Z'$, a $5 \times 5$ matrix, and the coefficient formula uses a $3 \times 3$ inverse. The dimensions match because $X'P_ZX$ is $3 \times 3$, and $X'P_Zy$ is $3 \times 1$. Dimensions alone do not guarantee that the required inverses exist; one must also check the rank conditions in the data.

A Numerical Calculation

Orthogonal columns make the projection and coefficient arithmetic especially transparent. Consider four hypothetical observations, with the following vectors:

\begin{aligned} c&=(1,1,1,1)',& w&=(-1,-1,1,1)',\\ z&=(-1,1,-1,1)',& v&=(1,-1,-1,1)'. \end{aligned}

Each vector has squared length $4$, and every pair has inner product $0$. Suppose the treatment is $D=z+w+v$, the structural outcome is $y=2D+3w+4v$, and the model includes an intercept, $D$, and $w$. The instrument matrix includes the intercept, $w$, and the excluded instrument $z$: $X=[c,D,w]$ and $Z=[c,w,z]$. The excluded instrument is relevant here because $D$ contains the component $z$.

Worked Example: Calculating 2SLS With an Instrument Projection

Since the columns of $Z$ are orthogonal and each has squared length $4$, the projection matrix is

P_Z=\frac{1}{4}(cc'+ww'+zz').

Projecting the treatment removes its component $v$, which is orthogonal to every instrument:

P_ZD=z+w, \qquad P_Zw=w.

The outcome can be written $y=2z+5w+6v$. Its projection is therefore $P_Zy=2z+5w$. Using the orthogonality and squared lengths, the cross-product matrix and right-hand side are

X'P_ZX= \begin{bmatrix} 4&0&0\\ 0&8&4\\ 0&4&4 \end{bmatrix}, \qquad X'P_Zy= \begin{bmatrix} 0\\28\\20 \end{bmatrix}.

For example, the treatment-and-covariate block has entries $(z+w)'(z+w)=8$, $(z+w)'w=4$, and $w'w=4$. The outcome cross-products are $(z+w)'(2z+5w)=8+20=28$ and $w'(2z+5w)=20$. Solving the normal equations gives

\begin{bmatrix} 8&4\\ 4&4 \end{bmatrix} \begin{bmatrix} \widehat{\beta}_D\\ \widehat{\beta}_W \end{bmatrix} = \begin{bmatrix} 28\\20 \end{bmatrix}, \qquad \begin{bmatrix} \widehat{\beta}_D\\ \widehat{\beta}_W \end{bmatrix} = \frac{1}{16} \begin{bmatrix} 4&-4\\ -4&8 \end{bmatrix} \begin{bmatrix} 28\\20 \end{bmatrix} = \begin{bmatrix} 2\\3 \end{bmatrix}.

The intercept estimate is $0$, since $c'y=0$ and $c$ is orthogonal to the other columns. Thus the fitted 2SLS coefficients are $0$ for the intercept, $2$ for treatment, and $3$ for the covariate, matching the coefficients used to generate this hypothetical outcome. This calculation illustrates the estimator, not evidence that the IV assumptions hold in an actual study.

Why Included Covariates Must Also Be Instruments

An included exogenous covariate is part of the structural equation, so its contribution to the outcome must be accounted for when estimating the treatment coefficient. Putting that covariate in $Z$ allows the projection to retain its variation as well as the variation supplied by excluded instruments. Omitting it from $Z$ can instead make an excluded instrument’s association with the covariate appear to be an association with the outcome through the treatment.

Worked Example: The Cost of Leaving Out a Covariate

Use the same hypothetical vectors and structural equation as above, but suppose someone leaves $w$ out of both the structural regressors and the instrument matrix. Let the excluded instrument be $a=w+z$, and retain an intercept. With the correct structural regressors $D$ and $w$, and instruments $c,w,a$, the instrument columns span the same space as $c,w,z$, because $z=a-w$. The covariate-adjusted calculation therefore gives the treatment coefficient $2$, as shown above.

If $w$ is omitted, the scalar IV ratio using $a$ is

\widehat{\beta}_{D,\text{omit }w} = \frac{a'y}{a'D}.

All these vectors are centered, so including the intercept does not change these cross-products. Since $a=w+z$, $D=w+z+v$, and $y=2z+5w+6v$, orthogonality gives

a'D=(w+z)'(w+z+v)=4+4=8, \qquad a'y=(w+z)'(2z+5w+6v)=8+20=28.

Thus the omitted-covariate ratio is $28/8=3.5$, not $2$. In the omitted model, the structural error includes the omitted term $3w$, and the instrument is associated with it: $a'(3w)=3(w+z)'w=12$. The moment condition for that omitted model fails in this hypothetical sample. The contrast shows why included covariates belong in both matrices when the intended structural model includes them.

Common Mistakes and Interpretation

Leaving an included exogenous covariate out of $Z$. The correction is to include each such regressor in the instrument matrix as well as in $X$.
Putting excluded instruments in $X$. They are instruments for the structural regressors, not outcome-equation regressors, unless the structural model specifically includes them.
Assuming the formula proves instrument validity. A matrix inverse can be computed even when the causal assumptions are implausible. Independence, exclusion, and relevance require substantive justification.
Checking dimensions but not rank. A correctly shaped matrix may still be singular. The formula requires $Z'Z$ and $X'P_ZX$ to be invertible.
Treating the second-stage regression as ordinary regression on observed $X$. The fitted regressors are $P_ZX$; replacing them with $X$ gives a different estimator. The matrix formula specifies the point estimate and does not, by itself, provide an inference procedure.

The matrix form is useful because it keeps the structure visible: $X$ states which coefficients are being estimated, $Z$ states which variation is used to estimate them, and $P_Z$ performs the projection. Correct matrix construction is essential, but the causal interpretation still depends on the IV assumptions for the study at hand.

Check Your Understanding

Use the matrix definitions and conditions to answer these questions.

A model includes an intercept, an endogenous treatment, and two exogenous covariates. What columns belong in $X$, and which of those columns must also appear in $Z$?
If $Z$ is $n \times L$, what are the dimensions of $P_Z$, $X'P_ZX$, and $X'P_Zy$ when $X$ is $n \times K$?
Why does the second-stage regression use $P_ZX$ rather than the observed $X$?
State the two sample invertibility conditions needed for the displayed matrix formula.
In the omitted-covariate example, why does the instrument moment condition fail for the model that leaves out $w$?

Individual	\(Y\)	\(D\)	\(W\)	\(A\)	\(B\)
1	4	1	0	1	0
2	7	2	1	0	1
3	5	1	1	1	1
4	9	3	0	0	1
5	6	2	1	1	0