How Selection Shifts Observed Outcome Distributions

Why the Observed Distribution Can Differ

When observation at a visit depends on the current outcome, the outcomes recorded at that visit need not represent the outcome distribution in the target population. For example, if subjects with higher current symptom scores are more likely to remain observed, the observed scores can be higher on average even if no subject's underlying outcome has changed because of observation. The key operation is reweighting: each possible outcome is represented among observed subjects in proportion to its probability of being observed.

The calculations below concern one visit and a fixed value of any baseline covariates or previously observed history. Let $Y$ be the current outcome and let $R=1$ indicate that it is observed at this visit; $R=0$ indicates it is not observed. For the moment, suppress conditioning on covariates and history. The target distribution $f_Y(y)$ describes outcomes in the population before conditioning on whether the current outcome was observed.

Definition: The selection probability at outcome value $y$ is $\pi(y)=P(R=1\mid Y=y)$. If $P(R=1)>0$, the distribution of outcomes among subjects observed at this visit is $$ f_{Y\mid R=1}(y)=\frac{f_Y(y)\pi(y)}{P(R=1)},\qquad P(R=1)=\int f_Y(u)\pi(u)\,du. $$ For discrete outcomes, replace the densities and integral by probabilities and a sum. This identity requires a well-defined target outcome distribution and a selection probability that gives positive overall observation probability.

The identity says exactly how selection changes the distribution: the original probability at $y$ is multiplied by $\pi(y)$, then normalized so the resulting probabilities sum or integrate to one. A value with a higher observation probability receives greater representation among observed subjects. The corresponding distribution among those not observed uses $1-\pi(y)$ in place of $\pi(y)$, when $P(R=0)>0$.

Mean and Variance Under Selection

The distribution identity gives useful summaries without requiring a particular outcome family. Taking expectations under it yields

E(Y\mid R=1)=\frac{E\{Y\pi(Y)\}}{E\{\pi(Y)\}} =E(Y)+\frac{\operatorname{Cov}\{Y,\pi(Y)\}}{E\{\pi(Y)\}}.

Thus the mean shift is the covariance between the outcome and its observation probability, divided by the marginal probability of observation. If higher outcomes tend to have higher observation probabilities, this covariance is positive and the observed mean exceeds the target mean. If observation probability is constant across outcome values, the covariance is zero and selection does not change the outcome distribution. The selected variance follows from the same weighting:

\operatorname{Var}(Y\mid R=1) =\frac{E\{Y^2\pi(Y)\}}{E\{\pi(Y)\}} -\left(\frac{E\{Y\pi(Y)\}}{E\{\pi(Y)\}}\right)^2.

A mean shift does not by itself determine what happens to the variance or shape. Selection can concentrate observations in a narrower range, widen the observed distribution, or change its skewness, depending on the outcome distribution and the shape of $\pi(y)$.

Useful implication: If $\pi(y)$ is nondecreasing, then $E(Y\mid R=1)\geq E(Y)$, provided the expectations exist. If $\pi(y)$ is nonincreasing, the inequality reverses. This concerns the mean; it does not imply that every feature of the selected distribution moves in the same direction.

One justification uses an independent copy $Y'$ of $Y$. The covariance can be written as

\operatorname{Cov}\{Y,\pi(Y)\} =\frac{1}{2}E\left[(Y-Y')\{\pi(Y)-\pi(Y')\}\right].

For a nondecreasing $\pi$, the two differences inside the expectation always have the same sign, so their product is nonnegative. The covariance is therefore nonnegative, and division by the positive quantity $E\{\pi(Y)\}$ gives the stated mean inequality. This argument also shows why monotonicity of the selection probability is the relevant condition, not merely a positive coefficient in a particular model parameterization.

Gaussian Outcomes With Probit Selection

A useful closed-form case has a Gaussian outcome and a probit selection equation. Assume $Y\sim N(\mu,\sigma^2)$, and specify $\pi(y)=\Phi(\alpha+\beta y)$, where $\Phi$ is the standard normal cumulative distribution function. Here $\alpha$ is the selection intercept and $\beta$ controls how observation probability changes with the current outcome. The model assumes that this Gaussian outcome distribution and probit selection probability apply in the covariate and history stratum being considered, with $\sigma>0$ and positive observation probability.

For concise expressions, define $a=(\alpha+\beta\mu)/\sqrt{1+\beta^2\sigma^2}$ and $\lambda(a)=\phi(a)/\Phi(a)$, where $\phi$ is the standard normal density. The following moments describe the distribution among subjects observed at this visit. The formulas are stated here with their derivation so their assumptions and the direction of the shift are explicit; they are also the closed-form probit-normal selection calculations used in the earlier material on probit-normal dropout probabilities.

Result: Under the Gaussian-probit assumptions, $$ P(R=1)=\Phi(a),\qquad E(Y\mid R=1)=\mu+\frac{\beta\sigma^2}{\sqrt{1+\beta^2\sigma^2}}\lambda(a), $$ $$ \operatorname{Var}(Y\mid R=1) =\sigma^2\left[1-\frac{\beta^2\sigma^2}{1+\beta^2\sigma^2} \lambda(a)\{a+\lambda(a)\}\right]. $$

To derive these results, write $Y=\mu+\sigma Z$, with $Z\sim N(0,1)$, and introduce an independent $U\sim N(0,1)$. The probit selection rule is equivalent to $R=1$ when $U\leq\alpha+\beta Y$. Rearranging gives $V=U-\beta\sigma Z\leq\alpha+\beta\mu$. The variable $V$ is normal with mean zero and variance $1+\beta^2\sigma^2$, so standardizing the threshold gives $P(R=1)=\Phi(a)$.

The pair $(Z,V)$ is jointly normal, with $\operatorname{Cov}(Z,V)=-\beta\sigma$. Conditional on $V$, the mean of $Z$ is $-\beta\sigma V/(1+\beta^2\sigma^2)$, and its conditional variance is $1/(1+\beta^2\sigma^2)$. Given $R=1$, the standardized variable $V/\sqrt{1+\beta^2\sigma^2}$ is a standard normal truncated above at $a$. Its conditional mean is $-\lambda(a)$, and its variance is $1-a\lambda(a)-\lambda(a)^2$. Substituting these truncated-normal moments into the conditional mean and variance of $Z$, and then using $Y=\mu+\sigma Z$, gives the displayed formulas. In particular, when $\beta>0$, the mean shift is positive; when $\beta<0$, it is negative.

Worked Example: Mean and Variance With Positive Probit Selection

Suppose, hypothetically, that $Y\sim N(50,100)$, so the target mean is 50 outcome units and the target standard deviation is 10 units. Let $P(R=1\mid Y=y)=\Phi(-6+0.1y)$. The Gaussian-probit assumptions are specified directly, and $\beta=0.1>0$ means observation probability increases with the outcome.

First, $a=(-6+0.1(50))/\sqrt{1+(0.1)^2(100)}=-1/\sqrt{2}\approx-0.707107$. Thus $P(R=1)=\Phi(-0.707107)\approx0.239750$. With $\lambda(a)=\phi(a)/\Phi(a)\approx1.295918$, the selected mean is

E(Y\mid R=1) =50+\frac{0.1(100)}{\sqrt{2}}(1.295918) \approx50+9.164 =59.16\text{ outcome units}.

The observed mean is about 9.164 units above the target mean, in the direction expected from increasing selection probability. For the variance, use the unrounded values in the formula:

\operatorname{Var}(Y\mid R=1) =100\left[1-\frac{1}{2}(1.295918) \left(-0.707107+1.295918\right)\right] \approx61.85\text{ outcome units}^2.

The selected distribution has less variance than the target Gaussian distribution in this example. The probability of observation, the mean, and the variance all follow from the stated model; they are not estimates from an actual study.

Selection Without a Gaussian Model

The reweighting identity applies to discrete and non-Gaussian outcomes as well. In those settings, calculate the observation-weighted probabilities directly when the outcome distribution and selection probabilities are specified. This avoids assuming that a probit-normal closed form applies to a different outcome family.

Worked Example: A Binary Outcome Among Observed Subjects

Consider a hypothetical binary outcome $Y$, where $Y=1$ denotes a positive result. Suppose $P(Y=1)=0.40$, $P(Y=0)=0.60$, and the specified selection probabilities are $P(R=1\mid Y=1)=0.75$ and $P(R=1\mid Y=0)=0.50$. The outcome distribution and the selection probabilities are fully specified, and both outcome groups have positive probability of being observed.

The overall probability of observation is $P(R=1)=0.40(0.75)+0.60(0.50)=0.30+0.30=0.60$. Therefore,

P(Y=1\mid R=1) =\frac{P(Y=1)P(R=1\mid Y=1)}{P(R=1)} =\frac{0.40(0.75)}{0.60} =0.50.

For a binary outcome, the mean is its probability of being one. The target mean is 0.40 and the observed mean is 0.50, a shift of $0.10$ outcome units. The observed subjects have a higher positive-result proportion because positive outcomes were more likely to be observed. This is a distributional selection effect, not evidence that observation caused the positive results.

Worked Example: Reweighting a Three-Value Outcome

Suppose, hypothetically, that $Y$ takes values 0, 10, and 20 outcome units with probabilities 0.20, 0.50, and 0.30. The corresponding selection probabilities are 0.20, 0.50, and 0.80. These probabilities are valid and increase with the outcome, so the general monotonicity result predicts an observed mean at least as large as the target mean.

The target mean is $E(Y)=0(0.20)+10(0.50)+20(0.30)=11$ outcome units. The probability of observation is $P(R=1)=0.20(0.20)+0.50(0.50)+0.30(0.80)=0.53$. The unnormalized observed masses at 0, 10, and 20 are 0.04, 0.25, and 0.24, respectively. Dividing each by 0.53 gives observed probabilities approximately 0.07547, 0.4717, and 0.4528.

The observed mean is $(0(0.04)+10(0.25)+20(0.24))/0.53=7.30/0.53\approx13.77$ outcome units. For comparison, the selected second moment is $(0^2(0.04)+10^2(0.25)+20^2(0.24))/0.53=121/0.53\approx228.3$ outcome units squared. Hence the observed variance is $228.3-(13.77)^2\approx38.59$ outcome units squared, using unrounded values for the subtraction. This example shows that the weighted distribution yields both a mean shift and a variance that can be calculated without a Gaussian assumption.

Assumptions, Diagnostics, and Interpretation

These calculations describe a population distribution conditional on a specified selection mechanism. In applications with covariates or previously observed outcomes, the selection probability and the target outcome distribution generally need to be understood conditional on that information. A shift calculated within a stratum need not equal a marginal shift after averaging over strata with different outcome distributions or selection probabilities.

The central limitation is that the current outcome is unavailable when $R=0$. Consequently, observed data alone do not generally determine how selection depends on that unobserved value. The formulas can quantify the consequences of a specified selection model, but they do not establish that the model is correct. Model assumptions, external information, and sensitivity analyses are important when the dependence on the missing current outcome cannot be checked directly.

Several checks help identify problems in a proposed calculation or model:

Verify that every specified selection probability is between zero and one and that the marginal observation probability is positive.
Check the direction of the shift against the selection probabilities. Increasing $\pi(y)$ should not produce a lower selected mean when the mean exists.
Check that the outcome distribution and selection equation refer to the same visit and condition on the same covariates or observed history.
For a fitted model, inspect implied observation probabilities across plausible outcome values. Probabilities near zero or one across most of the relevant range can make the model highly sensitive to its assumptions.
Compare implied selected and target distributions, not only their means. A mean can conceal important changes in spread or shape.

A common mistake is to treat the mean among observed subjects as if it were automatically the mean in the target population. That is justified only when observation is independent of the current outcome in the relevant conditioning stratum, or when a particular selection mechanism happens to leave the mean unchanged. Another mistake is to read $\beta$ in a selection equation as a causal effect of the outcome on dropout. It parameterizes a conditional association under the specified model; the distributional calculations alone do not give it a causal interpretation.

The main alternative model descriptions emphasize different distributions. A selection model specifies observation probability conditional on the outcome, as here. A pattern-mixture model specifies outcome distributions by observation pattern. These approaches organize assumptions differently; neither makes the unobserved outcome distribution empirically known without additional assumptions. The present calculations are about the distributional effect implied by a selection equation, not a method for predicting an individual's outcome after dropout.

Check Your Understanding

Answer each question, then compare with the solutions below.

If $\pi(y)$ is constant in $y$, what does the weighting identity imply about the observed outcome distribution?
A binary outcome has target probability $P(Y=1)=0.30$, with observation probabilities 0.80 for $Y=1$ and 0.50 for $Y=0$. What is $P(Y=1\mid R=1)$?
For Gaussian-probit selection with $\beta<0$, what is the direction of the selected-mean shift, assuming positive observation probability?
Does agreement between a fitted selection model and the observed distribution prove that the model correctly describes selection on unobserved current outcomes? Explain.

Solutions. (1) If $\pi(y)=c>0$, then $f_{Y\mid R=1}(y)=f_Y(y)c/c=f_Y(y)$, so the distributions are identical. (2) The observation probability is $0.30(0.80)+0.70(0.50)=0.59$. Therefore $P(Y=1\mid R=1)=0.24/0.59\approx0.4068$. (3) The mean shift has the sign of $\beta$, so it is negative; the selected mean is below $\mu$. (4) No. The current outcomes of subjects with $R=0$ are unavailable, so observed data alone generally cannot verify the assumed dependence of observation on those outcomes.