Sensitivity to Error Distributions and Dependence Assumptions

Why Selection Corrections Need Sensitivity Analysis

A selection correction can be precise under its maintained assumptions and still be sensitive to those assumptions. In a sample-selection model, outcomes are observed only for selected units. The correction for this nonrandom observation depends on how the selection error and outcome error are related, and on the distributional assumptions used to describe them. If the correction changes substantially under plausible alternatives, a single estimate can conceal important uncertainty about the model.

This tutorial examines that dependence directly. The normal model provides a benchmark, not a guarantee. We compare it with a specified nonnormal model and show how an explicit sensitivity parameter can summarize uncertainty about selection–outcome dependence. These calculations assess robustness to assumptions; they do not, by themselves, identify which assumptions are correct.

The Benchmark Model and Its Correction

Let $S_i^*$ be a latent selection index, and let $S_i=1$ indicate that unit $i$'s outcome is observed. The outcome of interest is $Y_i^*$; it is observed only when $S_i=1$. Write the model as

S_i^* = Z_i'\gamma + u_i,\qquad S_i=1[S_i^*>0],\qquad Y_i^* = X_i'\beta+\varepsilon_i.

Here $X_i$ and $Z_i$ are observed covariates, while $\varepsilon_i$ and $u_i$ are unobserved outcome and selection errors. Define the index $\eta_i=Z_i'\gamma$. The following benchmark assumptions give the familiar inverse Mills ratio correction: $u_i$ has a standard normal distribution; $(\varepsilon_i,u_i)$ are jointly normal with mean zero, $\operatorname{Var}(u_i)=1$, $\operatorname{Var}(\varepsilon_i)=\sigma^2$, and correlation $\rho$; and the errors are conditionally distributed the same way across covariate values. The model also requires an appropriate conditional-mean specification for the outcome error, $E[\varepsilon_i\mid X_i,Z_i]=0$, before selection. Identification of the parameters requires further conditions, discussed in the earlier material on exclusion restrictions and support for selection identification.

Benchmark result: Under these assumptions, the selection probability is $P(S_i=1\mid Z_i)=\Phi(\eta_i)$, and the expected outcome error among selected units is $$ E[\varepsilon_i\mid S_i=1,X_i,Z_i]=\rho\sigma\lambda(\eta_i),\qquad \lambda(\eta)=\frac{\phi(\eta)}{\Phi(\eta)}. $$ Thus $E[Y_i\mid S_i=1,X_i,Z_i]=X_i'\beta+\rho\sigma\lambda(\eta_i)$. The correction depends on both the outcome-error scale and its correlation with the selection error.

The factor $\rho\sigma$ is a dependence parameter in outcome units. If $\rho=0$, the expected outcome error does not shift with selection under the benchmark model. If $\rho$ is positive, units with selection errors high enough to be selected tend, on average, to have positive outcome errors as well; a negative $\rho$ reverses that direction. These interpretations are conditional on the model and do not imply that selection causes changes in outcomes.

Worked Example: Correcting a Slope Under Joint Normality

Suppose two groups differ by one unit in $x$. Their observed selected-sample mean outcomes are 10 and 14 outcome units. Their fitted selection indices are respectively $\eta=0$ and $\eta=1$. For this calculation, suppose the benchmark joint-normal assumptions hold, $\sigma=2$ outcome units, and $\rho=0.8$. Calculate the corrected difference in conditional mean outcomes.

The normal inverse Mills ratios are $\lambda(0)=0.7978846$ and $\lambda(1)=0.2876000$. The expected selection corrections are

\rho\sigma\lambda(0)=0.8(2)(0.7978846)=1.2766154, \qquad \rho\sigma\lambda(1)=0.8(2)(0.2876000)=0.4601600.

Subtract each correction from its observed selected-sample mean to obtain the corresponding fitted structural mean: $10-1.2766154=8.7233846$ at the first index and $14-0.4601600=13.5398400$ at the second. The corrected slope across the one-unit difference in $x$ is therefore

\widehat{\beta}_1 =\frac{(14-0.8(2)\lambda(1))-(10-0.8(2)\lambda(0))}{1} =4.816455\ldots \approx 4.8165 \quad\text{outcome units per unit of }x.

This calculation treats the fitted indices, scale, correlation, and benchmark distribution as given. It illustrates how the correction changes the slope; it does not show that these assumptions or parameter values are appropriate in another application.

Relaxing the Error Distribution

The normal correction is not a generic correction for selection. It follows from a specific joint distribution. If the selection error is nonnormal, the selection probability need not equal $\Phi(\eta_i)$, and the conditional mean outcome error among selected units need not be proportional to $\lambda(\eta_i)$. A sensitivity analysis that changes the error distribution must therefore specify what distribution replaces the normal model and derive, or otherwise justify, the corresponding selection probability and conditional correction.

One simple alternative makes the change explicit. Let the selection error $U$ be uniformly distributed on $(-1,1)$, and suppose the outcome error is $\varepsilon=\kappa U+\nu$, where $\nu$ has mean zero and is independent of $U$. Assume the same distribution applies across covariate values. For an index $\eta$ between $-1$ and $1$, selection occurs when $U>-\eta$. The selection probability and conditional mean of $U$ can be calculated directly. In this model the selection probability is not the probit probability, and the correction has a different shape from the inverse Mills ratio.

Worked Example: A Uniform Selection Error

Suppose $U\sim\operatorname{Uniform}(-1,1)$, $\varepsilon=2U+\nu$, and $E[\nu\mid U]=0$. The two groups again have indices $-0.5$ and $0.5$, differ by one unit in $x$, and have observed selected-sample mean outcomes of 12 and 15 outcome units. Find the selection corrections and the corrected slope.

For $-1<\eta<1$, the cutoff for selection is $U>-\eta$. Since the uniform density is $1/2$, its selection probability is

P(S=1\mid \eta) =P(U>-\eta) =\frac{1-(-\eta)}{2} =\frac{1+\eta}{2}.

Conditional on selection, $U$ is uniform between $-\eta$ and 1. Its mean is the midpoint of those endpoints:

E[U\mid S=1,\eta] =\frac{-\eta+1}{2} =\frac{1-\eta}{2}.

Because $E[\nu\mid S=1,\eta]=0$, the outcome-error correction is $E[\varepsilon\mid S=1,\eta]=2(1-\eta)/2=1-\eta$ outcome units. At $\eta=-0.5$, the selection probability is $0.25$, the correction is $1.5$ outcome units, and the adjusted mean is $12-1.5=10.5$. At $\eta=0.5$, the selection probability is $0.75$, the correction is $0.5$ outcome units, and the adjusted mean is $15-0.5=14.5$. Thus the corrected slope is

\frac{14.5-10.5}{1} =4.000 \quad\text{outcome units per unit of }x.

This result is conditional on the stated uniform-error model, not a correction that can be substituted while retaining a probit selection probability. The example demonstrates why changing the selection-error distribution can change both the selection probabilities and the correction function.

Varying Dependence With an Explicit Sensitivity Parameter

Sometimes the main uncertainty is the strength of selection–outcome dependence rather than a fully specified alternative distribution. Under the joint-normal benchmark, let $\delta=\rho\sigma$. Then the conditional mean correction is $\delta\lambda(\eta)$. A sensitivity analysis can recompute the outcome estimates over a prespecified range of $\delta$, while holding the selection indices fixed. The range should be justified by substantive knowledge, external information, or clearly stated scenarios. It should not be chosen merely because it produces a preferred result.

This exercise is conditional on retaining the benchmark correction shape. Varying $\delta$ relaxes the assumed dependence strength, but it does not relax normality or the inverse Mills ratio form. To assess departures from both, compare against an alternative joint error model, as in the uniform example, or use a justified alternative conditional-mean specification. A proposed function for the correction is not automatically compatible with a valid joint distribution; its implications for probabilities and conditional moments must be checked.

Worked Example: A Dependence Sensitivity Grid

Return to observed selected-sample means of 10 and 14, indices $\eta=0$ and $\eta=1$, and a one-unit difference in $x$. Retain the normal correction shape $\lambda(\eta)$, but vary $\delta=\rho\sigma$ over $-1, 0,$ and $1.6$ outcome units. The corrected slope is the selected-mean difference minus the difference in corrections:

\widehat{\beta}_1(\delta) =4-\delta\{\lambda(1)-\lambda(0)\} =4+0.5102846\delta.

For $\delta=-1$, the slope is $4-0.5102846=3.4897154$, or about $3.490$ outcome units per unit of $x$. For $\delta=0$, it is $4.000$. For $\delta=1.6$, it is $4+0.8164554=4.8164554$, or about $4.8165$. These are conditional estimates under three assumed dependence values, not three estimates of the probability that a particular model is true.

The table summarizes the calculation. A real analysis would also report how the assumed range was selected and, where appropriate, recalculate uncertainty for each specification.

$\delta$ (outcome units)	Corrected slope (outcome units per unit of $x$)
-1.0	3.490
0.0	4.000
1.6	4.8165

How to Conduct and Report the Comparison

A useful sensitivity analysis makes the maintained assumptions visible and changes them in an interpretable sequence. For each specification, calculate the correction implied by that specification and then re-estimate the outcome relationship. If an alternative changes the selection-error distribution, also recalculate or refit the selection probabilities under that distribution; do not carry over the probit probabilities without justification. If the alternative changes only the dependence parameter while retaining the normal model, say so explicitly.

State the benchmark.
Report the selection and outcome equations, the joint error assumptions, and the correction they imply.

Specify plausible alternatives.
Identify whether each scenario changes the dependence strength, the error distribution, or both. Give the parameter values or the alternative distribution explicitly.

Recalculate the correction and estimate.
Use the conditional mean implied by that scenario, and use selection probabilities consistent with its selection-error distribution.

Compare estimates in context.
Report how key coefficients or predictions change, with units, and distinguish this specification sensitivity from sampling uncertainty.

Sensitivity results are informative when they show which conclusions are stable and which depend on strong assumptions. If a coefficient changes sign or substantively important predictions change across plausible specifications, the analysis should report that dependence rather than presenting only the benchmark estimate. The comparison does not establish causality on its own: causal interpretation still depends on the outcome model, selection mechanism, and identification conditions.

Common Pitfall

A common error is to treat the inverse Mills ratio as a universal adjustment: “The sample is selected, so add the inverse Mills ratio and the problem is solved.” That reasoning confuses a model-specific conditional expectation with a general property of selected samples. The correction $\rho\sigma\lambda(\eta)$ relies on the normal selection error and joint-normal dependence assumptions stated above. Under a different error distribution, the selection probability and conditional mean correction can both change. The correction also does not guarantee identification when the covariates and support provide insufficient information. The remedy is to state the assumptions, calculate each alternative consistently, and report sensitivity across defensible specifications.

Key takeaway: A selection-corrected estimate is conditional on a model for selection–outcome dependence and error distributions. Vary those assumptions explicitly, use corrections consistent with each alternative, and describe the resulting range of estimates without treating it as a confidence interval or proof of model validity.

Check Your Understanding

Answer each question using the assumptions and calculations in this tutorial.

Which joint error assumptions produce the correction $\rho\sigma\lambda(\eta)$, and what does $\rho\sigma$ represent?
In the uniform-error example, what is the selection probability at $\eta=0$, and what is the conditional mean of $U$ among selected units?
Why is it generally incorrect to change the selection-error distribution but retain the probit selection probability without further justification?
In the dependence grid, how does the corrected slope change when $\delta$ changes from 0 to 1.6 outcome units?
Explain why comparing estimates across sensitivity specifications does not establish which specification is true.