Deriving the Heckman Full-Information Likelihood

Why the Full-Information Likelihood Has Two Parts

In a sample-selection model, an outcome is observed only when an observation passes a selection rule. For example, wages may be recorded only for people who work. The selection indicator is observed for both workers and nonworkers, but the wage outcome is observed only for workers. A likelihood for this data must therefore account for both the observed selection decisions and the outcomes recorded for selected units.

Under joint normality, selection and outcome errors are allowed to be correlated. That dependence matters: among selected units, the outcome error changes the probability that the selection rule was met. The full-information likelihood captures this relationship in a single joint model, rather than estimating the selection and outcome equations in separate stages.

The Model and Its Assumptions

Definition: For observation $i$, let the latent selection index be $S_i^*=z_i'\gamma+u_i$, and define the observed selection indicator as $d_i=1$ when $S_i^*>0$, and $d_i=0$ otherwise. Let the latent outcome be $y_i^*=x_i'\beta+\epsilon_i$, observed as $y_i=y_i^*$ only when $d_i=1$. Assume $$ \begin{pmatrix}\epsilon_i\\u_i\end{pmatrix} \sim N\left[ \begin{pmatrix}0\\0\end{pmatrix}, \begin{pmatrix}\sigma^2&\rho\sigma\\\rho\sigma&1\end{pmatrix} \right], \qquad \sigma>0,\quad -1<\rho<1. $$

Here $x_i$ and $z_i$ are observed covariate vectors, while $\beta$ and $\gamma$ are parameters. The selection-error variance is normalized to one because only the sign of the selection index determines $d_i$. The outcome-error standard deviation is $\sigma$, and $\rho$ is the correlation between the outcome and selection errors. Assume observations are independent conditional on the covariates and that this joint-normal specification applies to each observation. The likelihood below is conditional on the observed covariates.

Write $\phi(\cdot)$ and $\Phi(\cdot)$ for the standard normal density and cumulative distribution function. Define the standardized outcome residual and selection index as $t_i=(y_i-x_i'\beta)/\sigma$ and $a_i=z_i'\gamma$. The likelihood must use $y_i$ only when $d_i=1$; when $d_i=0$, the outcome is unobserved.

Deriving the Selected-Observation Contribution

For a selected observation, the data reveal both $y_i$ and the event $u_i>-a_i$. The joint density contribution is the outcome density multiplied by the probability of selection conditional on that outcome: $f(y_i,d_i=1\mid x_i,z_i)=f(y_i\mid x_i)P(d_i=1\mid y_i,x_i,z_i)$. The first factor is a normal density with mean $x_i'\beta$ and standard deviation $\sigma$:

f(y_i\mid x_i)=\frac{1}{\sigma}\phi(t_i).

To obtain the second factor, condition the selection error $u_i$ on the outcome error $\epsilon_i$. Under the stated bivariate normal model, $u_i\mid\epsilon_i=e\sim N(\rho e/\sigma,1-\rho^2)$. At the observed outcome, $e=y_i-x_i'\beta=\sigma t_i$, so the conditional mean of $u_i$ is $\rho t_i$. Selection occurs when $u_i>-a_i$, giving

P(d_i=1\mid y_i,x_i,z_i) =P(u_i>-a_i\mid \epsilon_i=\sigma t_i) =\Phi\left(\frac{a_i+\rho t_i}{\sqrt{1-\rho^2}}\right).

The last equality follows by standardizing the conditional normal variable and using $1-\Phi(q)=\Phi(-q)$. Multiplying the outcome density and this conditional selection probability gives the selected-observation contribution:

L_i^{(1)} =\frac{1}{\sigma}\phi(t_i) \Phi\left(\frac{a_i+\rho t_i}{\sqrt{1-\rho^2}}\right), \qquad d_i=1.

The factor involving $\Phi$ is not an extra probability attached arbitrarily to the outcome density. It is precisely the probability of observing selection, conditional on the outcome error implied by the recorded $y_i$. If $\rho=0$, that probability no longer depends on $y_i$, and the contribution reduces to the ordinary outcome density times $\Phi(a_i)$.

Worked Example: A Selected Observation’s Joint Contribution

Suppose $x_i'\beta=0$, $y_i=0$, $\sigma=1$, $a_i=0.4$, and $\rho=0.5$. These are hypothetical parameter and observation values. The standardized residual is $t_i=(0-0)/1=0$, and $\sqrt{1-\rho^2}=\sqrt{0.75}$.

Substitute into the selected-observation formula:

L_i^{(1)} =\phi(0)\Phi\left(\frac{0.4+0.5(0)}{\sqrt{0.75}}\right) =(0.3989423)\Phi(0.4618802).

Using $\Phi(0.4618802)\approx0.6779156$,

L_i^{(1)} \approx(0.3989423)(0.6779156) \approx0.2704492 \approx0.2704.

This is a density contribution for the observed outcome together with the selection event, not a probability for a point-valued continuous outcome. Its units are the reciprocal units of $y_i$; here the outcome has been measured on a unit scale.

The Unselected-Observation Contribution

When $d_i=0$, the data tell us that $u_i\leq-a_i$, but do not reveal $y_i^*$ or $\epsilon_i$. The contribution is therefore the probability of nonselection:

L_i^{(0)} =P(d_i=0\mid z_i) =P(u_i\leq-a_i) =\Phi(-a_i).

This probability uses the marginal distribution $u_i\sim N(0,1)$. It does not need a conditional outcome density, because no outcome was observed. In particular, it would be incorrect to insert an invented or imputed value of $y_i$ into the selected-observation formula as though it had been recorded.

Worked Example: An Unselected Observation

Suppose a hypothetical observation has $a_i=0.7$ and $d_i=0$. Its outcome is unavailable. The contribution is

L_i^{(0)}=\Phi(-a_i)=\Phi(-0.7)\approx0.2420.

Thus the observed nonselection decision contributes a probability of about 0.2420 under these parameter values. No value of $x_i'\beta$, $\sigma$, or $\rho$ is needed for this individual contribution because its outcome was not observed.

Combining the Contributions into the Likelihood

For each observation, exactly one of the selected and unselected cases applies. With independent observations conditional on the covariates, multiply their contributions. Equivalently, use the indicator $d_i$ as an exponent to select the appropriate factor:

Theorem: Under the stated model and independence assumption, the full-information likelihood for observations $(d_i,y_i\text{ when }d_i=1,x_i,z_i)$, $i=1,\ldots,n$, is $$ L(\beta,\gamma,\sigma,\rho) =\prod_{i=1}^{n} \left[ \frac{1}{\sigma}\phi(t_i) \Phi\left(\frac{a_i+\rho t_i}{\sqrt{1-\rho^2}}\right) \right]^{d_i} \left[\Phi(-a_i)\right]^{1-d_i}. $$ Here $t_i=(y_i-x_i'\beta)/\sigma$ is evaluated only for selected observations, and $a_i=z_i'\gamma$ is evaluated for all observations.

Taking logarithms turns the product into a sum. The log-likelihood is

\ell(\beta,\gamma,\sigma,\rho) =\sum_{i:d_i=1} \left[ -\log\sigma+\log\phi(t_i) +\log\Phi\left(\frac{a_i+\rho t_i}{\sqrt{1-\rho^2}}\right) \right] +\sum_{i:d_i=0}\log\Phi(-a_i).

The parameters must respect $\sigma>0$ and $-1<\rho<1$, so that the covariance matrix is positive definite. Full-information maximum likelihood chooses admissible parameter values to maximize this joint log-likelihood. The likelihood construction is the focus here; score-based computation and inference require additional results.

Worked Example: Likelihood for One Selected and One Unselected Unit

Consider two hypothetical observations with $\sigma=1$ and $\rho=0.5$. For the selected unit, let $x_i'\beta=0$, $y_i=1$, and $a_i=-0.5$. For the unselected unit, let $a_j=0$ and $d_j=0$. The selected unit has $t_i=1$, so its conditional selection argument is $(-0.5+0.5(1))/\sqrt{0.75}=0$.

Its contribution is

L_i^{(1)} =\phi(1)\Phi(0) \approx(0.2419707)(0.5000000) =0.1209854.

The unselected unit contributes

L_j^{(0)}=\Phi(0)=0.5000000.

Independence makes the sample likelihood at these specified parameter values

L=L_i^{(1)}L_j^{(0)} \approx(0.1209854)(0.5000000) =0.0604927.

The selected factor includes both the density of its recorded outcome and the probability of its selection conditional on that outcome. The unselected factor includes only its probability of nonselection.

Joint Likelihood Versus Conditioning on Selection

For a selected observation, one can also write the outcome density conditional on having been selected. Since $P(d_i=1\mid z_i)=\Phi(a_i)$, the conditional density is

f(y_i\mid d_i=1,x_i,z_i) =\frac{f(y_i,d_i=1\mid x_i,z_i)}{P(d_i=1\mid z_i)} =\frac{\sigma^{-1}\phi(t_i) \Phi\left((a_i+\rho t_i)/\sqrt{1-\rho^2}\right)} {\Phi(a_i)}.

This density is appropriate if the likelihood is explicitly conditioned on the sample containing selected observations. It is not the selected contribution in the joint likelihood for a sample that includes both selected and unselected units. In the joint likelihood, the selected contribution is the numerator; the selection probability is already represented through the joint data likelihood and the unselected observations’ contributions.

Worked Example: Why the Conditional Density Has a Denominator

Let $a_i=0$, $t_i=0$, $\sigma=1$, and $\rho=0.5$. The joint selected contribution is $\phi(0)\Phi(0)=0.1994711$, while the marginal probability of selection is $\Phi(0)=0.5$. Therefore,

f(y_i\mid d_i=1,x_i,z_i) =\frac{0.1994711}{0.5} =0.3989423.

This is the conditional density of the recorded outcome among selected units at these values. Multiplying it by the selection probability recovers the joint contribution: $(0.3989423)(0.5)=0.1994711$. Omitting the denominator while claiming to use a density conditional on selection would fail to normalize that density.

Derivation Check and Common Pitfalls

Use the observed data.
For $d_i=1$, the data include $y_i$ and $d_i$. For $d_i=0$, they include the selection decision but not the latent outcome.

Condition for selected units.
Use $u_i\mid\epsilon_i=\sigma t_i\sim N(\rho t_i,1-\rho^2)$ to calculate the selection probability given the observed outcome.

Use the marginal selection probability for unselected units.
The event $d_i=0$ has probability $\Phi(-a_i)$, and there is no observed outcome density to multiply by it.

Multiply across independent observations.
Form the product of the case-specific joint contributions, or add their logarithms.

A common mistake is to give every observation an outcome-density factor, perhaps treating an unselected unit as if its missing outcome were observed. That changes the data being modeled: the likelihood must integrate over the unobserved outcome, leaving the marginal selection probability $\Phi(-a_i)$. A different but related error is to use $f(y_i\mid d_i=1,x_i,z_i)$ in a joint likelihood without also accounting for selection; the conditional density has a denominator $\Phi(a_i)$ because it is normalized within the selected group.

The likelihood specifies how the observed data relate to the model parameters. It does not by itself ensure that the parameters are well identified in a particular application; identification depends on the covariates and model structure. Nor does joint normality become empirically guaranteed merely because this likelihood is used. The dependence adjustment in the selected contribution is a consequence of the maintained joint-normal assumption.

Key takeaway: Under joint normality, a selected unit contributes its outcome density times its selection probability conditional on that outcome. An unselected unit contributes its marginal probability of nonselection. Multiplying these contributions yields the Heckman full-information likelihood.

Check Your Understanding

Answer each question by referring to the likelihood construction.

Why does the selected-observation contribution depend on $\rho$, even though the outcome density by itself does not?
For an unselected unit with selection index $a_i$, write its likelihood contribution and explain why no outcome density appears.
What is the conditional distribution of $u_i$ given $\epsilon_i=\sigma t_i$ under the model?
How does the joint selected contribution relate to the outcome density conditional on selection?
What constraint on $\rho$ ensures the stated bivariate-normal covariance matrix is positive definite?