First-Stage Plots and Leverage Diagnostics

Why Inspect the First Stage?

Instrument-strength statistics compress a first-stage regression into numerical summaries. Plots add a different kind of information: they show how prediction varies across observations and whether a small number of unusual observations, curvature, or changing spread may be shaping the summary. A high first-stage fit can be driven by a few influential points, while a low fit may conceal structure that a linear model fails to capture. Visual diagnostics help identify these patterns; they do not replace formal strength tests or establish that an instrument is valid.

Let \(D_i\) denote an endogenous regressor, \(Z_i\) an instrument or vector of instruments, and \(X_i\) the included exogenous covariates. The conventional linear first stage regresses \(D_i\) on an intercept, \(Z_i\), and \(X_i\). Its fitted values are \(\widehat D_i\), and its residuals are \(\widehat u_i=D_i-\widehat D_i\). The fitted values summarize the part of \(D_i\) predicted by the regressors in this model; they are not observed treatment values or causal effects.

Fitted Values and Partial Relationships

For a single instrument, an observed-versus-fitted plot places \(D_i\) on the vertical axis and \(\widehat D_i\) on the horizontal axis. Points close to the 45-degree line have small first-stage residuals; substantial scatter indicates prediction error. This plot is useful for spotting outliers and broad mismatch, but it is not a direct plot of instrument relevance. A narrow range of fitted values can still have points close to the line, and a high overall fit can be dominated by the variation in included covariates rather than by the instrument.

To inspect the instrument's relationship with \(D_i\) conditional on \(X_i\), remove the linear projection on \(X\) from both variables. Write \(M_X=I-X(X'X)^{-1}X'\), assuming \(X\) has full column rank and includes an intercept. Then \(\widetilde D=M_XD\) and \(\widetilde Z=M_XZ\) are the residuals from regressing \(D\) and \(Z\), respectively, on \(X\). Plotting \(\widetilde D_i\) against \(\widetilde Z_i\) displays their conditional linear association. With multiple instruments, the display is made separately for each residualized instrument or uses an appropriate multivariate diagnostic.

The justification is the Frisch-Waugh-Lovell result: in the linear regression of \(D\) on \(X\) and \(Z\), the coefficient on \(Z\) is the coefficient from regressing \(M_XD\) on \(M_XZ\). Thus the partial-residual display isolates the same conditional linear relationship represented by the first-stage coefficient. It is a diagnostic visualization of that relationship, not a new estimate with different identifying assumptions.

This expression assumes that \(Z'M_XZ\) is invertible. For a single instrument, the plotted slope is \(\sum_i\widetilde Z_i\widetilde D_i/\sum_i\widetilde Z_i^2\). Curvature in the display, clusters, or a relationship that appears only at the extremes can suggest that a linear first-stage summary is inadequate or fragile. Such patterns require investigation rather than automatic changes to the model.

Leverage and Influence

A first-stage plot should be paired with a check for observations that occupy unusual positions in the regressor space. For a linear regression with design matrix \(W=[X,Z]\), including the intercept, the projection matrix is \(H=W(W'W)^{-1}W'\) when \(W\) has full column rank. Its diagonal element \(h_{ii}\) is the leverage of observation \(i\). Leverage depends on the regressors, not on the observed value of \(D_i\) or the size of its residual. If the model has \(p\) linearly independent columns, then \(\sum_i h_{ii}=p\), so average leverage is \(p/n\).

In the simple regression with an intercept and one instrument, leverage has the form \(h_{ii}=1/n+(Z_i-\bar Z)^2/\sum_j(Z_j-\bar Z)^2\). A point far from the instrument mean can therefore have high leverage. To assess influence, compare the fitted first stage and its strength summaries with and without the flagged observation, while treating that comparison as a sensitivity diagnostic—not as permission to delete a point. Check the observation's data source and context, and report defensible analyses transparently. Large leverage combined with a large residual can be especially influential; Cook's distance is one standard measure that combines these features.

Patterns to Check and Common Pitfalls

Plot residuals \(\widehat u_i=D_i-\widehat D_i\) against fitted values, instruments, and important covariates. A curved residual pattern suggests that the conditional mean may not be adequately represented by the chosen linear terms. A funnel-shaped spread suggests nonconstant residual variance, and isolated large residuals suggest observations that are poorly predicted. These patterns can affect standard errors and can also expose misspecification; a plot does not identify the correct replacement model. Any added transformations, interactions, or nonlinear terms should be justified and the first-stage diagnostics recomputed.

With many instruments, the fitted values can track the observed \(D_i\) closely in the estimation sample even when the first stage does not provide stable, informative prediction. This is especially important after the earlier material on many instruments and first-stage overfitting: a visually impressive in-sample fit is not evidence that a complex first stage will generalize or that instruments are valid. Consider the number of instruments relative to the sample size, the distribution of leverage, and how results change under defensible specifications.

• Mistake: treating a high \(R^2\) or close observed-versus-fitted plot as proof of strong instruments. The fit may largely reflect included covariates, high leverage, or many-instrument overfitting. Examine the partial relationship and formal strength diagnostics.
• Mistake: deleting every high-leverage observation. Leverage is a warning about potential influence, not proof of an error. Verify the record and report sensitivity transparently.
• Mistake: treating a partial-residual plot as a validity test. It describes conditional prediction under the specified model. It cannot establish instrument exogeneity or the exclusion restriction.

Plots complement, rather than replace, the first-stage F statistic, partial \(R^2\), and robust or multiple-instrument strength procedures appropriate to the design. Formal statistics quantify specified aspects of relevance under their conditions; plots help diagnose shape and sensitivity. Neither approach alone establishes the causal assumptions needed for instrumental-variables interpretation. Visual conclusions can also be unstable in small samples, and plotting does not cure poor overlap in instrument values, measurement error, or an incorrectly specified model.