Prediction for Censored and Truncated Outcomes

One Model, Several Prediction Targets

A fitted limited-outcome regression can answer several different prediction questions. It might predict the mean of an underlying outcome, the mean of the value actually recorded after censoring, the chance that a value is censored, or the mean among cases that pass a truncation rule. These quantities are generally not equal. A useful first step is therefore to name the target before calculating a prediction.

For example, consider an outcome such as days of hospital stay that cannot be recorded below zero. The regression may describe a latent outcome that could mathematically take negative values, while the recorded outcome is set to zero whenever that latent value is nonpositive. The latent mean and the expected recorded value answer different questions, even when they are based on the same fitted parameters.

Write \(\phi(z)\) for the standard normal density and \(\Phi(z)\) for its cumulative distribution function. The latent conditional mean is \(\mu(x)\). It is not automatically the expected observed value: censoring changes the distribution of what is recorded by placing probability mass at one or both limits.

Left Censoring: Latent Mean, Observed Mean, and Censoring Probability

With left censoring at zero, the observation rule is \(Y=\max(0,Y^*)\). The latent conditional mean remains \(\mu\), but the probability of recording zero is \(P(Y^*\leq 0\mid x)=\Phi(-\mu/\sigma)\). The expected recorded value is

This expectation accounts for both parts of the observed distribution: the mass at zero and the positive outcomes that are recorded without alteration. In particular, it is not obtained by simply replacing every negative latent prediction with zero; that operation is a point prediction, whereas the formula averages over the conditional distribution of \(Y^*\).

Two-Limit Censoring

For lower and upper limits \(L\) and \(U\), define the standardized limits \(a=(L-\mu)/\sigma\) and \(b=(U-\mu)/\sigma\). The probabilities of recording the lower limit, an interior value, and the upper limit are respectively \(\Phi(a)\), \(\Phi(b)-\Phi(a)\), and \(1-\Phi(b)\). The expected observed outcome weights each boundary by its probability and averages the latent outcome over the uncensored interval:

The middle two terms together are the contribution from cases with \(L<Y^*<U\). The formula is a prediction of the recorded outcome on its original scale, not a prediction of the latent mean. As a quick check, the three probabilities sum to one, and the resulting expectation must lie between \(L\) and \(U\).

Truncation: Predicting Among Observed Cases

Censoring records a boundary value for cases beyond a limit; truncation excludes those cases from the observed sample. These observation rules therefore imply different prediction targets. Under left truncation at a threshold \(c\), a case is observed only when \(Y^*>c\). The mean outcome among observed cases with covariates \(x\) is a conditional mean, \(E(Y^*\mid Y^*>c,x)\), rather than the unconditional latent mean \(\mu\).

Let \(a=(c-\mu)/\sigma\). Under the conditional normal model, the probability of being observed is \(1-\Phi(a)\), and the mean among observed cases is

Why the Truncated Mean Has an Adjustment

Standardize the latent outcome as \(Z=(Y^*-\mu)/\sigma\), so \(Z\sim N(0,1)\) and selection requires \(Z>a\). By the definition of a conditional expectation,

The standard normal density satisfies \(\phi'(z)=-z\phi(z)\). Therefore the numerator is

because \(\phi(z)\) tends to zero as \(z\) tends to infinity. Thus \(E(Z\mid Z>a)=\phi(a)/(1-\Phi(a))\). Since \(Y^*=\mu+\sigma Z\), multiplying the conditional expectation of \(Z\) by \(\sigma\) and adding \(\mu\) gives the stated formula. For a lower-truncated sample, the adjustment is positive: selected observations have a mean above the unconditional latent mean.

Choosing and Reporting the Target

A prediction should state what is being averaged, under what observation rule, and for which covariates. When parameters are estimated, each formula is evaluated using the fitted \(\hat{\beta}\) and \(\hat{\sigma}\); the resulting value is a plug-in prediction. That substitution does not make distinct targets interchangeable. For example, the latent conditional mean \(x'\hat{\beta}\) describes the underlying regression distribution, whereas the expected recorded outcome applies the censoring rule, and the selected-sample mean conditions on being observed.

The practical importance is easy to miss. A policy question about the average recorded payment may require an expected censored outcome. A question about the probability of hitting a cap requires an upper-censoring probability. A forecast for a sample that excludes all values below an eligibility threshold requires a conditional truncated mean. Reporting the latent mean for any of those questions can give a numerically plausible but substantively wrong answer.