Selection-Model Sensitivity Analysis for MNAR

Why Model the Chance of Being Observed?

When an outcome is missing, the observed cases may differ systematically from the unobserved cases. A selection model makes this possibility explicit by modeling the chance that an outcome is observed as a function of that outcome itself, as well as of recorded covariates. This is a model for missing not at random (MNAR) when, after conditioning on observed covariates, response still depends on an unobserved value.

The attraction and difficulty of this approach are the same: it gives a direct, interpretable description of outcome-dependent response, but the dependence on unobserved outcomes is generally not identified by the observed data alone. The practical aim is therefore not to pretend that the data determine this dependence. Instead, specify plausible fixed values for it, fit the remaining model under each value, and show how the conclusions change.

Definition: Let $Y$ denote a complete outcome vector, $X$ observed covariates, and $R$ a response-pattern indicator, with $R_j=1$ if outcome component $Y_j$ is observed and $R_j=0$ otherwise. A selection-model factorization specifies the joint distribution as $$ f_{\theta}(y\mid x)\,P_{\alpha,\lambda}(R=r\mid Y=y,X=x). $$ Here $\theta$ indexes the outcome model, $\alpha$ indexes response-model terms that may be estimated, and $\lambda$ denotes one or more sensitivity parameters governing dependence on unobserved outcomes. An MNAR selection model has response probabilities that depend on at least one outcome component that is missing for the response pattern under consideration.

Observed-Data Likelihood and the Sensitivity Parameter

For a particular observed pattern $r$, write $y_o$ for the observed components and $y_m$ for the missing components. The observed data include $x$, $r$, and $y_o$, but not $y_m$. Their likelihood contribution is the joint density of the observed values and the response pattern, with the missing values integrated out:

L_i(\theta,\alpha;\lambda) = \int f_{\theta}(y_o,y_m\mid x_i)\, P_{\alpha,\lambda}(R_i=r_i\mid y_o,y_m,x_i)\,dy_m .

For discrete missing components, the integral is replaced by a sum. The likelihood for independent subjects is the product of these contributions. This expression is the central computational distinction from simply fitting an outcome model to observed cases: the response probability remains inside the integral because it depends on the missing outcomes.

A common specification for a single binary response indicator is a logistic selection model:

\operatorname{logit}P(R=1\mid Y=y,X=x) = \alpha(x)+\lambda y .

For two outcome values $y=1$ and $y=0$, $\exp(\lambda)$ is the ratio of the response odds at $Y=1$ to the response odds at $Y=0$, conditional on $X$. Thus $\lambda=0$ means response does not depend on $Y$ after conditioning on $X$, while a positive $\lambda$ means higher outcome values have higher response odds. For a continuous outcome, the same coefficient describes the multiplicative change in response odds per one-unit increase in $Y$; the unit and scale must therefore be stated.

In a general model, $\lambda$ can be a vector and the outcome dependence can be expressed through specified functions of $Y$. The interpretation always depends on the particular link, scale, and functions used. In practice, the response intercept or other response-model terms may be fitted along with $\theta$, while $\lambda$ is held fixed at a chosen value.

Worked Example: A Binary Outcome and a Fixed Odds Ratio

Suppose $n=100$ people are enrolled. The outcome is observed for 80, and 20 of those 80 have $Y=1$. Thus the response rate is $q=0.80$, and the observed-case proportion is $p=P(Y=1\mid R=1)=20/80=0.25$. Consider the selection sensitivity value $\gamma=2$, where $\gamma$ is the response-odds ratio for $Y=1$ versus $Y=0$.

The observed-case odds of $Y=1$ are $p/(1-p)=0.25/0.75=1/3$. Under $\gamma=2$, the odds among nonrespondents are $(1/3)/2=1/6$. Therefore their outcome proportion is $m=(1/6)/(1+1/6)=1/7$, approximately $0.143$. The implied overall outcome proportion is $$ \widehat{\pi} = q p+(1-q)m = 0.80(0.25)+0.20(1/7) = 0.20+0.02857 = 0.22857 . $$

The observed cases alone have a proportion of $0.25$, but this sensitivity scenario implies a population proportion of about $0.229$. This is not a contradiction: they target different groups. The calculation is conditional on the chosen value $\gamma=2$; the observed counts did not determine that value.

Why the Selection Dependence Is Not Generally Identified

The binary case gives a direct demonstration. Let $q=P(R=1)$, $p=P(Y=1\mid R=1)$, and $m=P(Y=1\mid R=0)$. The observed data identify $q$ and $p$, but do not reveal $m$. The overall prevalence is $qp+(1-q)m$, and the response-odds ratio is determined by $p$ and $m$, not by the observed data alone.

Theorem: For a binary outcome with $0<q<1$ and $0<p<1$, the observed distribution of $(R, RY)$ does not, in general, identify the response-odds ratio $\gamma$. More precisely, there is an interval of distinct values of $m=P(Y=1\mid R=0)$ that yield the same observed distribution and different values of $\gamma$.

Proof. The observed distribution consists of the three probabilities $P(R=1,Y=1)=qp$, $P(R=1,Y=0)=q(1-p)$, and $P(R=0)=1-q$. It contains no separate information about the outcome among those with $R=0$. For any proposed $m$ in $(0,1)$, the complete joint distribution can be extended by assigning $P(R=0,Y=1)=(1-q)m$ and $P(R=0,Y=0)=(1-q)(1-m)$. These assignments leave all three observed probabilities unchanged.

By Bayes’ rule, the ratio of the outcome odds among respondents to the outcome odds among nonrespondents equals the response-odds ratio:

\gamma = \frac{P(R=1\mid Y=1)/P(R=0\mid Y=1)} {P(R=1\mid Y=0)/P(R=0\mid Y=0)} = \frac{p/(1-p)}{m/(1-m)}.

At $m=p$, this ratio is $1$. If $m\ne p$, it is different from $1$, while the observed probabilities remain the same. Moreover, for $m$ sufficiently close to $p$, all four joint cell probabilities are strictly positive: the respondent cells are positive by the conditions on $p$ and $q$, and the two nonrespondent cells are positive for $0<m<1$. Thus these choices correspond to valid distributions, not merely formal algebra. The observed distribution therefore permits distinct values of $m$ and $\gamma$, proving the claim. $\square$

This argument establishes a general warning, not a claim that every parametric selection model has exactly the same identification properties. Additional restrictions can sometimes identify parameters within a particular parametric model. In that case, conclusions depend on those restrictions. Without a credible source of information or assumptions that identify the outcome dependence, treating its estimate as if the observed data had established it is not justified.

Deriving and Using the Binary Sensitivity Formula

In the binary setting, define $\gamma$ as the ratio of response odds for $Y=1$ versus $Y=0$. The preceding odds identity can be rearranged to express the missing-outcome proportion as a function of the sensitivity value and the observed-case proportion:

\frac{m}{1-m} = \frac{1}{\gamma}\frac{p}{1-p}, \qquad m = \frac{p}{p+\gamma(1-p)}.

When $\gamma=1$, $m=p$, so the outcome prevalence is the same among respondents and nonrespondents. For $\gamma>1$, $m<p$: people with $Y=1$ are more likely to respond, so the observed-case proportion is higher than the missing-case proportion. For $0<\gamma<1$, the direction is reversed. The population prevalence under a fixed $\gamma$ is then

\pi(\gamma)=q p+(1-q)\frac{p}{p+\gamma(1-p)}.

Worked Example: Comparing Three Sensitivity Scenarios

Use the data from the previous calculation: $q=0.80$, $p=0.25$, with 80 responses among 100 people. Evaluate the implied prevalence at $\gamma=0.5$, $1$, and $2$.

For $\gamma=0.5$, $m=0.25/[0.25+0.5(0.75)]=0.25/0.625=0.40$, so $\pi=0.80(0.25)+0.20(0.40)=0.20+0.08=0.28$.

For $\gamma=1$, $m=0.25/[0.25+1(0.75)]=0.25$, so $\pi=0.80(0.25)+0.20(0.25)=0.25$.

For $\gamma=2$, $m=0.25/[0.25+2(0.75)]=0.25/1.75=1/7$, so $\pi=0.20+0.20/7\approx0.2286$.

The observed data are identical in all three calculations. What changes is the assumed relationship between response and outcome; that assumption changes the implied distribution of missing outcomes and hence the prevalence estimate.

Stratification and Model-Based Inference

When covariates predict both the outcome and response, sensitivity calculations should generally be conditional on those covariates. For binary $Y$, let $q_x=P(R=1\mid X=x)$ and $p_x=P(Y=1\mid R=1,X=x)$. If the conditional response-odds ratio is $\gamma_x$, then

m_x = P(Y=1\mid R=0,X=x) = \frac{p_x}{p_x+\gamma_x(1-p_x)}, \qquad \pi_x=q_xp_x+(1-q_x)m_x .

An overall prevalence is obtained by averaging the stratum-specific values using the target population’s covariate proportions. This standardization matters: a prevalence among respondents pooled across strata need not represent either the population prevalence or a common within-stratum value.

Worked Example: Standardizing Across Two Covariate Strata

Suppose two equally sized covariate strata each contain 100 people. In stratum A, $q_A=0.80$ and $p_A=0.20$; in stratum B, $q_B=0.50$ and $p_B=0.60$. Consider the specified sensitivity values $\gamma_A=2$ and $\gamma_B=0.5$.

In stratum A, $m_A=0.20/[0.20+2(0.80)]=0.20/1.80=1/9$. Thus $\pi_A=0.80(0.20)+0.20(1/9)=0.16+0.02222\approx0.18222$.

In stratum B, $m_B=0.60/[0.60+0.5(0.40)]=0.60/0.80=0.75$. Thus $\pi_B=0.50(0.60)+0.50(0.75)=0.30+0.375=0.675$.

Because the strata are equally sized, the standardized prevalence is $(\pi_A+\pi_B)/2=(0.18222+0.675)/2\approx0.42861$. For comparison, setting both sensitivity values to $1$ gives $\pi_A=0.20$ and $\pi_B=0.60$, and the standardized prevalence is $0.40$. The difference reflects the specified response-outcome associations within strata, not a change in the observed stratum data.

For more complex outcomes, the same principle applies through the observed-data likelihood. At each fixed $\lambda$, estimate the parameters that remain free by maximizing that likelihood or by an appropriate inferential procedure. For a target such as a mean or prevalence, calculate the target from the fitted complete-data model, including the implied distribution of missing outcomes. Repeating this process over a prespecified grid of $\lambda$ values reveals how the inference responds to the unidentifiable dependence.

A standard error calculated at one fixed sensitivity value describes sampling uncertainty conditional on that value and on the model. It does not account for uncertainty about which sensitivity value is appropriate. Report the fixed values considered, the conditional estimates and uncertainty, and the range of conclusions across the sensitivity analysis separately.

Common Mistakes and Interpretation

Estimating the sensitivity parameter as if the observed data identified it. A numerical optimizer may return a value, but an optimizer’s output does not establish identification. Explain what external information or restrictions support any estimated dependence on missing outcomes.
Calling a zero sensitivity coefficient “no missing data.” In the logistic model, $\lambda=0$ removes outcome dependence from response conditional on $X$; it does not make $R=1$ for everyone. Missing observations can still be numerous.
Confusing a response probability ratio with a response-odds ratio. In the logistic specification, $\exp(\lambda)$ is a ratio of odds, not a ratio of probabilities. For example, odds can double even though the probability does not double.
Reporting only the fit at the preferred sensitivity value. A single conditional estimate hides the dependence of the conclusion on an assumption that the observed data generally cannot verify. Show a defensible range of fixed values and its inferential consequences.
Ignoring covariate composition. Applying one pooled respondent proportion to all individuals can mix differences in covariate distributions with differences due to missingness. Model or stratify on relevant observed covariates and standardize to the target population.

Key takeaway: A selection model states how response depends on the full outcome, including unobserved values. The observed-data likelihood integrates over those values, but generally cannot determine the dependence parameter by itself. Fix that parameter at meaningful sensitivity values, fit the remaining model, and report how the substantive inference changes.

Check Your Understanding

Use the definitions and calculations above to answer the following.

In a selection model, what is integrated out of the likelihood contribution for a subject with missing outcome components?
For a binary outcome, if the response-odds ratio $\gamma$ is greater than one, is the implied outcome proportion among nonrespondents greater than or less than the observed-case proportion? Explain.
With $p=0.30$, calculate $m$ at the fixed sensitivity value $\gamma=1.5$.
Why can two different values of the response-odds ratio yield the same observed distribution?
What uncertainty does a standard error at a fixed sensitivity value describe, and what uncertainty does it not describe?