Binary Outcomes Observed Under Nonrandom Selection

Why Selection Matters for Binary Outcomes

Suppose an outcome is binary but is recorded only for people who enter a program, respond to a survey, or meet an eligibility rule. If entry is related to unobserved characteristics that also affect the outcome, the observed outcomes need not represent the outcomes of the full population. A binary-outcome model that ignores this selection can therefore describe the selected group poorly, even when its probability model would be appropriate without selection.

The bivariate-probit sample-selection model represents selection and the binary outcome using two latent variables. Their errors may be correlated. The model’s key contribution is a likelihood that uses all observed information: whether each unit was selected, and, for selected units only, whether its outcome was zero or one.

Model and Assumptions

Definition: For unit $i$, let $S_i=1$ indicate that the unit is selected and its binary outcome is observed; let $S_i=0$ indicate that the outcome is unobserved. The latent selection and outcome equations are $$ S_i^*=z_i^\mathsf{T}\gamma+u_i,\qquad S_i=1[S_i^*>0], $$ $$ Y_i^*=x_i^\mathsf{T}\beta+\varepsilon_i,\qquad Y_i=1[Y_i^*>0]\quad\text{when }S_i=1. $$ Here $x_i$ and $z_i$ are observed covariate vectors, and $\beta$ and $\gamma$ are coefficient vectors. Conditional on the covariates, $(\varepsilon_i,u_i)$ has a bivariate standard normal distribution with correlation $\rho$, where $-1<\rho<1$. The error pairs are independent across units. The outcome $Y_i$ is observed only if $S_i=1$.

The unit variances of the two errors fix the scales of the latent variables, which otherwise cannot be identified separately from their coefficients. The threshold is normalized to zero in each equation. Write $\eta_i=x_i^\mathsf{T}\beta$ and $\mu_i=z_i^\mathsf{T}\gamma$. The marginal selection probability is $P(S_i=1\mid z_i)=\Phi(\mu_i)$; without selection, the outcome probability would be $P(Y_i=1\mid x_i)=\Phi(\eta_i)$.

The correlation $\rho$ describes dependence between the unobserved determinants of selection and of the binary outcome, conditional on the covariates. It is not the correlation between $S_i$ and $Y_i$, and it is not itself a causal effect. If $\rho>0$, units with a higher selection error tend also to have a higher outcome error. Because both latent variables are classified as positive above their respective thresholds, this dependence generally changes the outcome distribution among selected units.

The distributional assumptions are substantive: the normal errors, constant unit variances, linear indices, and common correlation structure determine the probabilities in the likelihood. Correct specification also requires that the recorded covariates and selection status match the model’s definitions. As in other sample-selection models, identification can be fragile if the two equations have no covariate variation that helps distinguish their indices; the likelihood formula alone does not guarantee that the parameters are well identified.

Deriving the Observed-Data Likelihood

Let $\Phi$ denote the standard normal cumulative distribution function and $\Phi_2(a,b;r)$ the joint standard normal cumulative distribution function with correlation $r$, evaluated at $(a,b)$. A nonselected unit contributes the probability of $S_i=0$. For a selected unit, the likelihood must use the joint event that selection occurred and the observed binary outcome took its recorded value.

Unselected unit.
$S_i=0$ means $u_i\leq-\mu_i$, so its contribution is $P(S_i=0)=\Phi(-\mu_i)$. Since $Y_i$ is unobserved, no outcome event is included.

Selected unit with $Y_i=1$.
The event is $u_i>-\mu_i$ and $\varepsilon_i>-\eta_i$. By changing signs on both errors, the two variables remain standard normal with correlation $\rho$. The probability is $\Phi_2(\mu_i,\eta_i;\rho)$, equivalently $\Phi_2(\eta_i,\mu_i;\rho)$.

Selected unit with $Y_i=0$.
The event is $u_i>-\mu_i$ and $\varepsilon_i\leq-\eta_i$. The variables $-u_i$ and $\varepsilon_i$ have correlation $-\rho$; applying the normal symmetry gives probability $\Phi_2(\mu_i,-\eta_i;-\rho)$.

Combining these mutually exclusive observed-data cases gives the likelihood for independent units:

L(\beta,\gamma,\rho) =\prod_{i=1}^{n} \bigl[\Phi(-\mu_i)\bigr]^{1-S_i} \bigl[\Phi_2(\eta_i,\mu_i;\rho)\bigr]^{S_iY_i} \bigl[\Phi_2(\mu_i,-\eta_i;-\rho)\bigr]^{S_i(1-Y_i)}.

The powers select the appropriate contribution: when $S_i=0$, only the first factor remains; when $S_i=1,Y_i=1$, only the second remains; and when $S_i=1,Y_i=0$, only the third remains. The log-likelihood is the sum of the logarithms of the corresponding contributions. Parameters can be estimated by maximizing it, subject to the model’s assumptions and adequate identification.

Worked Example: A Selected Success at Zero Indices

Consider a hypothetical unit with $\eta_i=0$, $\mu_i=0$, and $\rho=0.6$. Suppose the unit is selected and has $Y_i=1$. Its likelihood contribution is $$ \Phi_2(0,0;0.6)=\frac14+\frac{\arcsin(0.6)}{2\pi}. $$

Since $\arcsin(0.6)\approx0.6435$, the contribution is $$ 0.25+\frac{0.6435}{6.2832}\approx0.3524. $$

This is the joint probability of selection and a positive outcome, not the probability of a positive outcome conditional on selection. The marginal probability of selection is $\Phi(0)=0.5$, so the conditional probability of a positive outcome among selected units is approximately $0.3524/0.5=0.7048$. With $\rho=0$, the conditional probability would be $0.5$. Thus positive latent-error dependence raises the positive-outcome share among selected units in this zero-index example.

Conditional Outcome Probabilities

The likelihood also gives a direct expression for the probability of a positive outcome among selected units. Divide the joint probability of selection and a positive outcome by the probability of selection:

P(Y_i=1\mid S_i=1,x_i,z_i) =\frac{\Phi_2(\eta_i,\mu_i;\rho)}{\Phi(\mu_i)}.

This expression shows why the selected-sample probability is generally not just $\Phi(\eta_i)$. The denominator accounts for how likely a unit with covariates $z_i$ is to be selected; the numerator accounts for selection and outcome jointly. When $\rho=0$, the errors are independent, so the numerator factors into $\Phi(\eta_i)\Phi(\mu_i)$, and the conditional probability reduces to $\Phi(\eta_i)$. When $\rho\neq0$, that simplification does not hold.

The same logic applies to a selected failure: $$ P(Y_i=0\mid S_i=1,x_i,z_i) =\frac{\Phi_2(\mu_i,-\eta_i;-\rho)}{\Phi(\mu_i)}. $$ These two conditional probabilities sum to one because they partition the selected units according to their observed binary outcome.

Worked Example: Independent Errors With Unequal Indices

For a hypothetical unit, let $\eta_i=0.5$, $\mu_i=-0.2$, and $\rho=0$. Suppose the unit is selected and has $Y_i=0$. Independence gives a selected-failure contribution equal to the product of the selection and outcome probabilities: $$ \Phi(-0.2)\Phi(-0.5)\approx0.4207(0.3085)=0.1298. $$

For the same covariates, the selected-success contribution is $$ \Phi(-0.2)\Phi(0.5)\approx0.4207(0.6915)=0.2909, $$ and the unselected contribution is $$ \Phi(0.2)\approx0.5793. $$

The three possible observed-data contributions sum to approximately $0.1298+0.2909+0.5793=1.0000$. Conditional on selection, the positive-outcome probability is $0.2909/0.4207\approx0.6915$, which equals $\Phi(0.5)$ because the errors are independent.

How the Correlation Enters

The sign change in the selected-failure probability is important. For a success, selection and a positive outcome require both original errors to be high, so their dependence is represented by $\rho$. For a failure, selection requires $u_i$ to be high while a failure requires $\varepsilon_i$ to be low. The relevant transformed pair therefore has correlation $-\rho$. Omitting this sign change assigns the wrong joint probability to selected failures.

The correlation parameter can change the likelihood even when the marginal selection and outcome probabilities are held fixed. For example, at zero indices, the marginal probability of selection remains one half for every $\rho$, and the unconditional probability of a positive outcome also remains one half. What changes with $\rho$ is the overlap between the two positive-threshold events, and therefore the outcome composition of the selected sample.

Worked Example: Likelihood for a Small Hypothetical Sample

Suppose four hypothetical units all have $\eta_i=\mu_i=0$, and set $\rho=0.6$. The observed records are two selected successes, one selected failure, and one unselected unit. From the zero-index probabilities, each selected success contributes $0.3524$, the selected failure contributes $$ \Phi_2(0,0;-0.6)=\frac14-\frac{\arcsin(0.6)}{2\pi}\approx0.1476, $$ and the unselected unit contributes $0.5$.

The likelihood at these parameter values is therefore $$ L=(0.3524)^2(0.1476)(0.5)\approx0.00917. $$

This value is the probability of this particular observed pattern under the specified parameters, not an estimate of the parameters. Estimation compares such likelihood values across candidate parameters using the full sample. For comparison, at $\rho=0$ the corresponding contributions would be $0.25,0.25,0.5$, giving $L=(0.25)^3(0.5)=0.0078125$. The larger value at $\rho=0.6$ for this pattern does not by itself establish that $0.6$ is the best-fitting correlation; all observations and parameter values must be considered.

Common Pitfall and Practical Interpretation

A common mistake is to fit a probit model to the observed outcomes and treat its fitted probabilities as population outcome probabilities. That reasoning assumes the outcome is observed independently of its latent determinants, conditional on the included covariates. In the model here, this assumption corresponds to $\rho=0$. If $\rho\neq0$, the observed outcomes come from a selected distribution, and a probit model using only selected units does not generally represent $\Phi(\eta_i)$ for the full population.

The correction is not to treat unobserved outcomes for $S_i=0$ as zeros. Those outcomes are missing by the design represented here, not recorded failures. Instead, include each nonselected unit’s selection probability in the likelihood and each selected unit’s joint selection-outcome probability. Interpret the resulting coefficients within the specified latent-index model; a coefficient is not automatically a change in outcome probability or a causal effect.

Key takeaway: In a binary sample-selection model, the likelihood contribution depends on what is observed. Nonselected units contribute a selection probability; selected units contribute a joint probability for selection and their observed outcome. Correlated latent errors link the two equations and alter outcome probabilities within the selected sample.

Check Your Understanding

Answer each question using the model and likelihood developed above.

Why does an unselected unit contribute $\Phi(-\mu_i)$ rather than a probability involving its unobserved binary outcome?
Write the likelihood contribution for a selected unit with $Y_i=0$, and explain why the bivariate-normal correlation in that contribution is $-\rho$.
At $\rho=0$, why does $P(Y_i=1\mid S_i=1,x_i,z_i)$ reduce to $\Phi(\eta_i)$?
Distinguish the probability of selection and a positive outcome jointly from the probability of a positive outcome conditional on selection.
What does $\rho$ describe, and why should it not be interpreted as a causal effect?