Heckman Two-Step Estimation from Probit to OLS

Why Two-Step Estimation Is Needed

When an outcome is recorded only for a selected subset of a population, an OLS regression on the observed outcomes may describe that subset rather than the population outcome relationship. The problem is not simply that fewer observations are available: if selection is related to the unobserved determinants of the outcome, the outcome errors among selected units can have a nonzero conditional mean. The Heckman two-step estimator addresses this dependence by estimating a selection equation first and using its fitted values to construct a correction for the outcome regression.

For example, wages may be observed only for people who work. If unobserved factors that raise wages also affect the probability of working, the employed sample can have a systematically different distribution of outcome errors from the full population. The two-step procedure uses the selection indicator for the full sample, then corrects the outcome equation estimated on workers. It does not, by itself, make the assumptions of the model true; the interpretation depends on a correctly specified selection model and the joint distributional assumptions below.

Definition: Let $S_i$ equal 1 if unit $i$'s outcome is observed and 0 otherwise. The Heckman sample-selection model has a latent outcome equation $y_i^*=x_i'\beta+\varepsilon_i$ and a latent selection equation $S_i=1[z_i'\gamma+u_i>0]$. We observe $y_i=y_i^*$ only when $S_i=1$. Here $x_i$ and $z_i$ are covariate vectors, and $\beta$ and $\gamma$ are parameter vectors.

Model Assumptions and the Selection Correction

The standard Heckman two-step estimator relies on a Gaussian sample-selection model. Conditional on the covariates, assume the outcome and selection errors are jointly normal with mean zero, and are independent of the covariates. The probit scale is normalized by setting the variance of $u_i$ to 1; its scale cannot be separately identified from $\gamma$ in the binary selection equation. Write the outcome-error standard deviation as $\sigma_\varepsilon$, and the correlation between the two errors as $\rho$. Thus, $\operatorname{Cov}(\varepsilon_i,u_i)=\rho\sigma_\varepsilon$. Observations are assumed independent across units, and the regressors used in the selected-sample outcome regression must have sufficient variation to identify its coefficients.

Under these assumptions, the conditional mean of the outcome error among selected units is proportional to an inverse Mills ratio. The selection rule $S_i=1$ means $u_i>-z_i'\gamma$, so the relevant probability in the denominator is the probability of selection, $\Phi(z_i'\gamma)$, not the probability of nonselection.

Key result: Define $\phi$ and $\Phi$ as the standard normal density and cumulative distribution functions. For selection into the observed sample when $z_i'\gamma+u_i>0$, $$ E(\varepsilon_i\mid S_i=1,z_i)=\rho\sigma_\varepsilon\lambda(z_i'\gamma), \qquad \lambda(t)=\frac{\phi(t)}{\Phi(t)}. $$ The function $\lambda$ is the inverse Mills ratio for this selection rule. Let $\delta=\rho\sigma_\varepsilon$. In the selected sample, the conditional outcome equation is therefore $E(y_i\mid S_i=1,x_i,z_i)=x_i'\beta+\delta\lambda(z_i'\gamma)$.

The earlier derivation of the inverse Mills ratio establishes the conditional-mean result. The key estimation implication is that selection changes the conditional mean of the outcome error by a term that is a known function of the selection index, apart from the unknown parameters. Replacing that index with a probit estimate gives a generated regressor that can be included in an outcome regression.

The Two Estimation Stages

Estimate selection by probit.
Use the full sample, including units with $S_i=0$, to estimate $\gamma$ from $P(S_i=1\mid z_i)=\Phi(z_i'\gamma)$. Denote the probit estimate by $\hat{\gamma}$.

Construct the fitted inverse Mills ratio.
For every unit, calculate $\hat{\eta}_i=z_i'\hat{\gamma}$ and $\hat{\lambda}_i=\phi(\hat{\eta}_i)/\Phi(\hat{\eta}_i)$. The correction is calculated for selected units used in the outcome regression.

Estimate the corrected outcome equation.
Using only observations with $S_i=1$, regress $y_i$ on $x_i$ and $\hat{\lambda}_i$ by OLS. The coefficients on $x_i$ estimate $\beta$, and the coefficient on $\hat{\lambda}_i$ estimates $\delta=\rho\sigma_\varepsilon$.

The first stage models whether an outcome is observed; it is not a regression of the outcome on the selection indicator. The second stage uses the observed outcome values, so units with $S_i=0$ do not enter that OLS regression. If $x_i$ includes an intercept, the correction term is added as another regressor. The two-step estimator is the pair of estimates produced by these stages, not a single OLS regression on the full sample.

Worked Example: Calculating Fitted Corrections From a Probit

Suppose a hypothetical sample contains workers and nonworkers, with work status recorded for everyone. A probit fitted to this full sample gives $\hat{\gamma}$. For one person, the fitted index is $\hat{\eta}_i=z_i'\hat{\gamma}=0.6$. Calculate the correction term for this person.

The estimated inverse Mills ratio is $$ \hat{\lambda}_i=\frac{\phi(0.6)}{\Phi(0.6)} =\frac{0.3332}{0.7257} \approx 0.4591. $$

This person has an estimated selection probability of $\Phi(0.6)\approx0.7257$, and the value added to the second-stage regression is approximately 0.4591. It is not the selection probability itself. The ratio is positive under this selection convention, including when the fitted index is negative.

For comparison, at an index of $-0.4$, $\phi(-0.4)\approx0.3683$ and $\Phi(-0.4)\approx0.3446$, giving $\hat{\lambda}\approx0.3683/0.3446=1.069$. The ratio is larger at this lower selection index because the denominator—the probability of selection—is smaller.

Why the Second-Stage Regression Works

After the probit stage, the model's conditional-mean result says that among selected observations, the mean outcome error is $\delta\lambda(z_i'\gamma)$. Adding this term to the outcome equation removes that modeled conditional mean. To see the regression logic, define $v_i=\varepsilon_i-\delta\lambda(z_i'\gamma)$ for selected observations. Then

y_i=x_i'\beta+\delta\lambda(z_i'\gamma)+v_i, \qquad E(v_i\mid x_i,z_i,S_i=1)=0.

If the true index were known, OLS on selected observations with $x_i$ and $\lambda(z_i'\gamma)$ as regressors would estimate the conditional-mean coefficients under the usual rank and sampling conditions. In practice, $\gamma$ is unknown, so the first stage estimates it and the second stage uses $\hat{\lambda}_i$. This is why the procedure is called two-step estimation.

The coefficient on the correction term, $\delta$, captures the scale of selection dependence in the outcome-error mean. Under the stated model, $\delta=0$ when $\rho=0$, since $\sigma_\varepsilon$ is positive. The coefficient is not itself the error correlation: recovering $\rho$ also requires $\sigma_\varepsilon$. The coefficients on $x_i$ remain the outcome-equation coefficients, provided the model and identification conditions are adequate.

Worked Example: OLS With the Fitted Correction

Consider a deliberately small, hypothetical selected sample. The outcome $y$ is measured in thousands of dollars; $x$ is a coded covariate; and the inverse Mills ratio values are supplied by a first-stage probit. The table also shows $e_i$, the residuals from the constructed outcome equation, to make the OLS arithmetic transparent.

Unit	$x_i$	$\hat{\lambda}_i$	$y_i$	$e_i$
1	1	0.3	8.9	1
2	-1	0.5	1.5	-3
3	-1	0.7	8.1	3
4	1	0.9	8.7	-1

The outcomes were constructed as $y_i=5+2x_i+3\hat{\lambda}_i+e_i$. For example, the first is $5+2(1)+3(0.3)+1=8.9$. The residuals sum to $1-3+3-1=0$, so they are orthogonal to the intercept. Their product sum with $x_i$ is $1(1)+(-3)(-1)+3(-1)+(-1)(1)=0$. Their product sum with $\hat{\lambda}_i$ is $1(0.3)+(-3)(0.5)+3(0.7)+(-1)(0.9)=0$. Thus they are orthogonal to all three OLS regressors.

The means are $\bar{x}=0$, $\overline{\hat{\lambda}}=0.6$, and $\bar{y}=27.2/4=6.8$. The centered cross-product between $x$ and $\hat{\lambda}$ is zero: $1(-0.3)+(-1)(-0.1)+(-1)(0.1)+1(0.3)=0$. Their centered sums of squares are $\sum x_i^2=4$ and $\sum(\hat{\lambda}_i-0.6)^2=0.20$. Because the residuals are orthogonal to both centered regressors, the OLS slopes are $2$ on $x$ and $3$ on $\hat{\lambda}$. The intercept is $6.8-2(0)-3(0.6)=5$.

The fitted outcome equation is therefore $\hat{y}_i=5+2x_i+3\hat{\lambda}_i$, with outcome units of thousands of dollars. The estimated correction coefficient is 3 thousand dollars per one-unit increase in the inverse Mills ratio, conditional on $x_i$. These four constructed observations illustrate the calculation, not a setting for reliable statistical inference.

Interpreting the Correction and Checking the Procedure

The inverse Mills ratio varies with the fitted selection index, so its contribution to the conditional outcome mean varies across units. For example, suppose a hypothetical second-stage estimate gives $\hat{\delta}=-1.5$ thousand dollars, and a unit has $\hat{\lambda}=0.4591$. The fitted selection correction for that unit is $$ \hat{\delta}\hat{\lambda}=(-1.5)(0.4591)=-0.6887 $$ thousand dollars. For otherwise identical $x_i$, this is the correction added to the outcome's fitted conditional mean at that selection index. It is not a causal effect of changing selection probability.

Before interpreting estimates, verify that the two stages match the observation process: $S_i$ must be observed for selected and nonselected units; the probit must use the correct selection coding; and the outcome regression must use only units whose outcomes are observed. Also check that the outcome and selection covariates are coded consistently and that the selected-sample regressors provide enough independent variation to estimate the regression. These checks do not establish the Gaussian assumptions, but coding or sample mistakes can invalidate the procedure even if those assumptions were otherwise plausible.

The second-stage correction is an estimated regressor, not a directly observed covariate. Consequently, the usual OLS standard errors from treating $\hat{\lambda}_i$ as fixed do not account for uncertainty from estimating $\gamma$ in the probit. OLS point estimates are the defining second-stage calculation, but inference must account for this first-stage estimation as well as the selected-sample error structure. The coefficient on the correction term and its conventional OLS standard error should therefore not be treated as a complete inferential analysis.

Common Mistake

A frequent error is to fit a probit only among units whose outcomes are observed, then treat its fitted probabilities as the correction. That discards the $S_i=0$ observations that define the selection process, and the fitted probability is not the inverse Mills ratio. The correction requires a probit for $S_i$ using the full sample, followed by $\phi(z_i'\hat{\gamma})/\Phi(z_i'\hat{\gamma})$ for selected units in the outcome regression. Another error is to use $\phi(t)/[1-\Phi(t)]$ while retaining the rule $S_i=1[z_i'\gamma+u_i>0]$; that denominator corresponds to the opposite tail and does not match the stated selection rule.

Key takeaway: Heckman two-step estimation first fits a probit selection equation on the full sample, then adds the resulting estimated inverse Mills ratio to an OLS outcome regression for selected observations. The correction is justified by the Gaussian selection model; it does not remove the need to assess the model assumptions or use inference that accounts for the generated correction.

Check Your Understanding

Answer each question using the selection rule and notation defined above.

Why does the first-stage probit use observations with both $S_i=1$ and $S_i=0$?
For selection defined by $S_i=1[z_i'\gamma+u_i>0]$, write the formula for the inverse Mills ratio and identify its denominator.
If the second-stage coefficient on the correction is $\hat{\delta}$, what model parameter combination does it estimate?
Explain why a usual OLS standard error that treats the fitted correction as fixed does not capture all uncertainty.
What is wrong with interpreting the correction coefficient as the causal effect of increasing a unit's probability of selection?