Common Failures in Sample-Selection Analysis

Why Sample-Selection Corrections Can Mislead

A sample-selection correction can address a specific problem: outcomes are observed only for a nonrandom subset, and the unobserved outcome error may be related to the process determining observation. But a correction is not a general repair for a poor design or a misspecified model. Its interpretation depends on the selection equation, the outcome equation, and the assumptions connecting their errors.

This tutorial focuses on four recurring failures: treating the inverse Mills ratio as an ordinary regressor, using second-stage standard errors that ignore its estimated construction, defending an implausible exclusion restriction, and describing corrected predictions as if they required no modeling assumptions.

The Correction and Its Assumptions

Let \(d_i\) indicate whether an outcome is observed. A standard Heckman selection model has a latent selection index \(z_i'\gamma+v_i\), with \(d_i=1\) when that index is positive, and an outcome equation \(y_i=x_i'\beta+u_i\) observed only when \(d_i=1\). Here \(x_i\) and \(z_i\) are covariate vectors, and \(\beta\) and \(\gamma\) are parameter vectors. The conventional parametric correction relies on a correctly specified selection index and a joint normal error assumption:

Under these assumptions, the probability of selection is \(\Phi(z_i'\gamma)\), where \(\Phi\) is the standard normal cumulative distribution function. Define the inverse Mills ratio for selected observations as \(\lambda(a)=\phi(a)/\Phi(a)\), where \(\phi\) is the standard normal density.

The correction follows from the covariance structure. Joint normality gives \(E[u_i\mid v_i]=\rho\sigma v_i\). Writing \(a=z_i'\gamma\), selection is equivalent to \(v_i>-a\). For standard normal \(v_i\), \(E[v_i\mid v_i>-a]=\phi(a)/\Phi(a)=\lambda(a)\). Therefore,

This yields the selected-sample conditional mean \(E[y_i\mid d_i=1,x_i,z_i]=x_i'\beta+\rho\sigma\lambda(z_i'\gamma)\). In a two-step procedure, the first step estimates \(\gamma\), often with a probit model; the second step regresses selected outcomes on \(x_i\) and the estimated correction term. The coefficient on that term estimates the scale combination \(\rho\sigma\) under the model, not a causal effect of “having a higher inverse Mills ratio.”

Failure 1: Treating the Correction as an Ordinary Regressor

The second-stage regression notation can make the inverse Mills ratio look like just another column of data. That appearance is misleading. Its role is to represent the selected observations’ conditional error mean under the assumed joint distribution. Its coefficient has a specific structural interpretation, and the correction itself is generated from the estimated selection equation.

Consequently, a coefficient on \(\lambda\) should not be described as the effect of a one-unit increase in a substantive covariate. Nor should the correction automatically be retained, transformed, or interpreted as though it were a directly observed covariate chosen independently of the model. If the distributional assumptions or selection equation are wrong, the correction may fail to represent the conditional error mean at all.

Failure 2: Ignoring Generated-Regressor Uncertainty

The estimated inverse Mills ratio is not known data. It is calculated using \(\hat{\gamma}\), which is estimated from the same sample-selection process. Ordinary least-squares standard errors from the second stage typically act as though the calculated correction values were fixed. That misses uncertainty from estimating the first stage and can also miss covariance between first- and second-stage estimation. Thus, those standard errors are generally not valid for inference on the two-step estimator.

Inference must account for the estimation procedure as a whole. Suitable approaches include a variance correction for the two-step estimator, a joint estimating-equation or sandwich calculation, or a bootstrap that repeats both stages in each resample. Simply adding a probit standard error to an OLS standard error is not a general solution: the stages are linked, and the relevant covariance structure matters.

Failure 3: Using an Implausible Exclusion Restriction

An excluded variable appears in \(z_i\), the selection equation, but not in \(x_i\), the outcome equation. A credible exclusion restriction requires a substantive argument that the variable affects selection while having no direct effect on the outcome, conditional on the included outcome covariates. It should also provide useful variation in selection. A strong association with selection alone does not establish the exclusion restriction.

For example, distance to a service might predict whether a person is observed in a sample. But if distance also changes access to care, and the outcome is health, excluding distance from the outcome equation may be indefensible. Statistical significance in the selection equation cannot rule out that direct pathway. Earlier material on exclusion restrictions, support, and weak exclusions provides the relevant identification diagnostics; the central practical check here is whether the exclusion makes sense for the specific outcome and setting.

Failure 4: Calling Corrected Predictions Assumption-Free

A selection-corrected prediction is conditional on the model used to construct it. Under the bivariate normal specification, the selected-sample conditional mean includes \(\rho\sigma\lambda(z_i'\gamma)\). Changing the selection equation, error distribution, dependence structure, or treatment of heteroskedasticity can change the correction and therefore the prediction. Correcting for selection does not make the result independent of assumptions.

It is also important to name the prediction target. The mean among selected observations, \(E[y_i\mid d_i=1,x_i,z_i]\), is not generally the same as the mean outcome in the full population at given covariates. In the model above, the latent outcome equation has mean \(x_i'\beta\) conditional on its covariates, while selection changes the conditional mean among those with \(d_i=1\). These are different quantities, not competing estimates of one universal mean.

A Practical Audit Before Reporting Results

A careful report distinguishes what the data show from what the selection model assumes. Before interpreting a corrected coefficient or prediction, check the following:

• State which outcomes are observed and how the selection indicator is defined.
• Describe the selection and outcome equations, and identify the distributional and dependence assumptions supporting the correction.
• Explain why each excluded predictor belongs in the selection equation but not directly in the outcome equation.
• Use inference that accounts for the estimated first-stage correction rather than treating it as fixed.
• Label predictions by their target population or conditioning event, and state that they rely on the fitted model.

These checks do not guarantee that a selection model is correct. They make the basis of its conclusions visible and help prevent a mechanically computed correction from being mistaken for identification by itself.