Selection Models for MNAR Outcomes

Why Model the Chance an Outcome Is Observed?

When an outcome is missing not at random (MNAR), the probability that it is recorded depends on its value, even after conditioning on observed information. For example, patients with worse unobserved symptoms may be less likely to complete a follow-up assessment. The observed outcomes alone cannot reveal how much more or less likely those patients were to respond. A selection model makes this dependence explicit and asks how conclusions change across specified values of an unidentifiable sensitivity parameter.

The model concerns the selection process into the observed-outcome sample. It is not a model for treatment assignment, and it does not make an MNAR mechanism identifiable from the observed data. Its purpose is to combine a model for outcomes among respondents with an explicit assumption about how outcome values relate to response.

Definition: Let $R=1$ indicate that an outcome $Y$ is observed and $R=0$ that it is missing. A selection model factors the joint distribution of $Y$ and $R$, conditional on observed variables, as an outcome distribution $f(Y\mid A,X)$ and a response probability $P(R=1\mid Y,A,X)$. Here $A$ is treatment and $X$ denotes observed covariates. The model is MNAR when the response probability depends on the outcome after conditioning on $A$ and $X$.

A common specification is a logistic response model:

\operatorname{logit} P(R=1\mid Y,A,X) =\alpha(A,X)+\gamma Y.

The intercept $\alpha(A,X)$ allows response rates to vary by treatment and covariates. The parameter $\gamma$ describes the association between outcome value and response on the log-odds scale, conditional on $A$ and $X$. For a one-unit increase in $Y$, the conditional odds of response are multiplied by $\exp(\gamma)$. A nonzero $\gamma$ encodes outcome-dependent response; its value is generally not identified from observed data because $Y$ is unavailable when $R=0$.

What the Observed Data Identify

For a fixed treatment and covariate stratum, the observed data identify the response rate $r=P(R=1\mid A,X)$ and the outcome distribution among respondents $f(y\mid R=1,A,X)$. They do not, without further assumptions, identify the outcome distribution among nonrespondents or the response probabilities at each unobserved outcome value.

A selection model supplies the missing link. Given a chosen $\gamma$, the intercept is calibrated so that the model agrees with the observed response rate. The model then implies response probabilities for different outcome values and, consequently, a distribution for the outcomes that were not observed. This calibration is important: a sensitivity scenario should change the assumed outcome-dependence, not silently contradict the response rate in the data.

For a continuous outcome, one might specify a normal or other suitable distribution among respondents and combine it with the logistic response model. For a binary outcome, the observed respondent distribution is summarized by the proportion with $Y=1$. The following binary derivation makes the calibration particularly transparent.

Key relationship: Within a treatment-covariate stratum, write $r=P(R=1)$, $q=P(Y=1\mid R=1)$, $p=P(Y=1)$, and $\pi_y=P(R=1\mid Y=y)$. Then $$ p=\frac{q r}{\pi_1}, \qquad 1-p=\frac{(1-q)r}{\pi_0}, $$ and the logistic selection parameter satisfies $$ \gamma=\operatorname{logit}(\pi_1)-\operatorname{logit}(\pi_0). $$

The first two equalities follow from the joint probabilities $P(Y=1,R=1)=p\pi_1$ and $P(Y=0,R=1)=(1-p)\pi_0$. These probabilities are also $qr$ and $(1-q)r$, respectively. Thus, for a proposed $\gamma$, one can solve for response probabilities consistent with the observed $q$ and $r$, then recover the full outcome prevalence $p$. The resulting values must be valid probabilities; a chosen model that cannot match the observed data in a stratum is not a viable sensitivity scenario.

Worked Examples

Worked Example: Interpreting a Continuous-Outcome Sensitivity Parameter

Suppose $Y$ is a symptom score in points, and a fitted selection model is $\operatorname{logit}P(R=1\mid Y,A,X)=\alpha(A,X)-0.20Y$. Interpret the outcome-dependence term.

For each one-point increase in symptom score, the log odds of response decrease by $0.20$, holding treatment and covariates fixed. The corresponding response-odds ratio is $\exp(-0.20)=0.8187$. Thus, under this assumed model, a one-point higher score is associated with about $18.1\%$ lower odds of response, since $1-0.8187=0.1813$.

This is not an estimated causal effect of symptoms on response, nor does it establish that higher-scoring patients actually respond less often. It is the interpretation of a specified sensitivity value. The intercepts would still need to be calibrated to the observed response rates, and the assumed relationship would need to be considered over the outcome's relevant range.

Worked Example: Recovering Binary Outcome Risks Under an MNAR Model

Consider hypothetical data with no covariates, a binary outcome, and two treatment groups. In the treated group, $r=0.56$ of outcomes are observed and $q=0.24/0.56=0.4286$ of observed outcomes equal 1. In the control group, $r=0.78$ are observed and $q=0.24/0.78=0.3077$. Suppose the sensitivity parameter is $\gamma=-1.7918$, so the response odds for $Y=1$ are one-sixth the response odds for $Y=0$, because $\exp(-1.7918)\approx 1/6$.

For the treated group, response probabilities $\pi_1=0.40$ and $\pi_0=0.80$ have the required odds ratio: \[ \frac{0.40/(1-0.40)}{0.80/(1-0.80)} =\frac{2/3}{4}=\frac{1}{6}. \] They also reproduce the observed response rate when the implied full-data risk is $p=0.60$: \[ P(R=1)=0.60(0.40)+0.40(0.80)=0.56. \] Among respondents, the outcome risk is \[ P(Y=1\mid R=1)=\frac{0.60(0.40)}{0.56} =\frac{0.24}{0.56}=0.4286. \] Thus the model recovers a treated-group outcome risk of $0.60$.

For the control group, take $\pi_1=0.60$ and $\pi_0=0.90$. Their response-odds ratio is \[ \frac{0.60/(1-0.60)}{0.90/(1-0.90)} =\frac{1.5}{9}=\frac{1}{6}. \] With $p=0.40$, the response rate is $0.40(0.60)+0.60(0.90)=0.78$, and the respondent risk is $0.40(0.60)/0.78=0.3077$, matching the stated observed data. The implied risk difference is therefore $0.60-0.40=0.20$, or 20 percentage points. This contrast is conditional on the sensitivity model and the causal assumptions needed to interpret treatment-group risks as causal.

Worked Example: How the Sensitivity Value Changes a Causal Contrast

Use the same hypothetical observed data. For $\gamma=0$, response is independent of the binary outcome within each treatment group, so $\pi_1=\pi_0=r$ and the full-data risk equals the respondent risk $q$. The treated and control risks are then $0.4286$ and $0.3077$, giving a risk difference of \[ 0.4286-0.3077=0.1209, \] or 12.09 percentage points.

Now consider the opposite sensitivity value, $\gamma=1.7918$, for which response odds at $Y=1$ are six times those at $Y=0$. For the treated group, let $u=\pi_0$. The odds-ratio condition gives $\pi_1=6u/(1+5u)$. Calibration to $r=0.56$ and $q=3/7$ requires \[ r\left(\frac{q}{\pi_1}+\frac{1-q}{u}\right)=1. \] Substitution yields \[ 0.56\left(\frac{1+5u}{14u}+\frac{4}{7u}\right)=1, \] so $u=0.45$, $\pi_1=0.8308$, and $p=qr/\pi_1=0.24/0.8308=0.2889$.

For controls, $q=4/13$, $r=0.78$, and the same odds-ratio condition gives \[ 0.78\left(\frac{1+5u}{19.5u}+\frac{9}{13u}\right)=1. \] Solving gives $u=0.725$, $\pi_1=0.9405$, and $p=0.24/0.9405=0.2552$. The risk difference is $0.2889-0.2552=0.0337$, or 3.37 percentage points. These recalculations show how the same observed data can imply different causal contrasts under different, untestable outcome-response assumptions.

From Group Risks to Causal Contrasts

Let $Y(1)$ and $Y(0)$ denote potential outcomes under treatment and control. A common causal estimand is the average treatment effect $E[Y(1)-Y(0)]$; for a binary outcome, this can be expressed as a risk difference. If covariates $X$ are needed to control confounding, the selection model is fit or calibrated within covariate strata, and the resulting outcome distributions are standardized to a target covariate distribution. A crude difference between treatment groups is not generally a causal effect merely because missing outcomes have been modeled.

Causal interpretation still requires appropriate treatment-assignment assumptions. In particular, consistency links observed outcomes to the potential outcome under received treatment, treatment positivity requires both treatments to be possible in relevant covariate strata, and conditional exchangeability requires no residual confounding given $X$. The selection model addresses missing outcomes; it does not repair failures of these assumptions.

The selection model also needs a credible respondent-outcome model and response positivity over outcome values relevant to the analysis. If some outcome values have essentially zero probability of being observed, the data provide little or no information about their distribution, and model-based recovery relies heavily on extrapolation. With covariates, the assumed outcome-response relationship may also differ across strata; imposing a common $\gamma$ is a substantive restriction, not a consequence of the data.

Why the Sensitivity Analysis Matters

For MNAR outcomes, the observed data cannot determine $\gamma$. A single fitted estimate under an arbitrarily fixed value can therefore conceal the central uncertainty. A useful analysis reports the values considered, the rationale for their scale and direction, the resulting causal contrasts, and whether the substantive conclusion changes across plausible scenarios. The range should be clinically or scientifically interpretable: for a continuous outcome, state what a one-unit change means; for a binary outcome, describe the implied response-odds ratio.

A selection-model sensitivity analysis is not a test of which MNAR value is true. Model fit to observed outcomes and response rates cannot validate the assumed relationship with outcomes that are missing. The analysis instead makes the untestable assumption visible and quantifies its consequences for the estimand.

Specify the observed-data models.
Describe the outcome distribution among respondents and the response rates across treatment and relevant covariate strata.

Choose interpretable sensitivity values.
Define the scale and meaning of $\gamma$, and select values that represent plausible outcome-dependent response relationships.

Calibrate and recover.
For each value, calibrate response probabilities to observed response rates and derive the implied full outcome distributions.

Re-estimate and report.
Calculate the causal contrast under each scenario, using the same target population and confounding adjustment, and explain how conclusions vary.

Common Mistake

Wrong reasoning: “The response rate and observed outcome distribution are known, so the outcome-dependent response parameter can be estimated from the data.” The observed data do not include outcome values for nonrespondents; different values of the selection parameter can be compatible with the same observed response rate and respondent outcomes.

Correction: Treat the outcome-dependence parameter as a sensitivity assumption, calibrate the remaining model components to observed quantities, and report how the causal contrast changes across prespecified values. A precise estimate under one value does not remove uncertainty about that value.

Key takeaway: A selection model links the outcome distribution to the probability of observing the outcome. Its sensitivity parameter governs the unidentifiable outcome-response association; calibration makes each scenario compatible with observed response rates, while the resulting causal contrast remains conditional on both the sensitivity assumption and the usual causal identification assumptions.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following questions.

In a logistic selection model, what does $\exp(\gamma)$ represent for a one-unit increase in a continuous outcome?
Why can the observed response rate and respondent outcome distribution not, by themselves, identify the MNAR sensitivity parameter?
For a binary outcome, explain why $P(Y=1)=qr/\pi_1$.
What does $\gamma=0$ imply about response and outcome within a treatment-covariate stratum?
Name two treatment-effect identification assumptions that a selection model for missing outcomes does not replace.