Sharp Bounds for a Missing Binary Outcome

What Can Be Learned Without Modeling Missing Outcomes?

When a binary outcome is not observed for everyone, the observed outcome rate among responders need not equal the population outcome probability. The difference depends on the outcomes among nonresponders, which are not directly observed. If we make no assumption about those outcomes, the data may not determine one population probability. They can, however, restrict it to a range.

This tutorial derives that range using only two observable quantities: the response rate and the probability of a positive outcome among responders. The resulting bounds are sharp: every value inside them can occur under some complete-data distribution consistent with those observable quantities, and no value outside them can. Thus the bounds describe exactly what those quantities alone establish—not a confidence interval and not a model-based guess for the missing outcomes.

The outcome \(Y\) is understood to be defined for every member of the target population, whether or not it is observed. We take \(q\) and \(p_R\) as the observable population quantities for the identification argument. In a sample, the response fraction and responder outcome fraction estimate these quantities, but sampling uncertainty is a separate issue.

The basic setup requires that response status is observed and that the outcome can be classified as 0 or 1 when it is observed. To use \(p_R\), there must be some responders, so \(q>0\). To define \(p_M\), there must be some nonresponders, so \(q<1\). No independence between \(Y\) and \(R\) is assumed. In particular, the outcome distribution among nonresponders may differ arbitrarily from that among responders.

Deriving the Bounds

The population consists of responders, who make up proportion \(q\), and nonresponders, who make up proportion \(1-q\). The probability of a positive outcome is therefore the weighted average of the positive-outcome probabilities in those two groups:

For \(0<q<1\), the only restriction imposed on the unobserved probability \(p_M\) is that it must lie between 0 and 1. Substituting the smallest possible value and the largest possible value into the mixture equation gives the bounds.

For the lower bound, \(p_M\geq 0\), so \((1-q)p_M\geq 0\) and \(p\geq q p_R\). For the upper bound, \(p_M\leq 1\), so \((1-q)p_M\leq 1-q\) and \(p\leq q p_R+(1-q)\). These inequalities show that no compatible population probability can fall outside the proposed range.

To verify sharpness rather than just validity, take any number \(t\) in the proposed interval. Since \(1-q>0\), set

The condition \(t\geq q p_R\) makes this value nonnegative. The condition \(t\leq q p_R+(1-q)\) makes it no greater than 1. Thus it is a valid nonresponder probability, and substituting it into the mixture gives \(p=t\). In particular, \(p_M=0\) and \(p_M=1\) produce the two endpoints. The observed quantities alone cannot rule out any point in the interval.

The width of the interval is \(1-q\), the nonresponse proportion. As response becomes more complete, the interval narrows. Its location also depends on \(p_R\): responders contribute \(q p_R\) to the population probability even in the lowest-outcome configuration for nonresponders.

Working From Sample Counts

In practice, the response rate and responder outcome probability are often estimated from counts. If \(n\) individuals are sampled, \(n_R\) respond, and \(y_R\) responders have \(Y=1\), the plug-in quantities are \(\hat q=n_R/n\) and \(\hat p_R=y_R/n_R\), provided \(n_R>0\). Substituting them into the formulas gives estimated identification limits. This arithmetic does not account for sampling uncertainty around either estimate.

Edge Cases and Interpretation

The conditional probabilities in the definitions require care at the extremes of the response rate. If \(q=1\), everyone responds, so there is no nonresponder group and \(p_M\) is undefined. In that case, the population probability is directly observed:

The interval formula also reduces algebraically to \(p=p_R\), but the direct statement is the appropriate interpretation: there are no missing outcomes to assign. If \(q=0\), there are no responders, so \(p_R=P(Y=1\mid R=1)\) is undefined. With no outcome observations, \(q\) alone gives only \(0\leq p\leq 1\). One must not insert an undefined \(p_R\) into the formula.

A bound is not a point estimate. Reporting its midpoint, for example, does not make that midpoint better supported by the observed data than other points in the interval. Nor is the interval a 95% confidence interval: it describes the range allowed by the stated identification assumptions, while a confidence interval would additionally quantify sampling uncertainty. In finite samples, uncertainty in \(\hat q\) and \(\hat p_R\) should be addressed separately when inferential coverage is required.

Why These Bounds Matter—and Where They Stop

The bounds make the role of missing outcomes transparent. The observed responders establish a contribution of \(q p_R\) to the population probability. The remaining contribution can range from zero to the full nonresponse proportion. This separates what follows from the observed distribution from what would require assumptions about unobserved outcomes.

One alternative is to report the responder proportion \(p_R\) as if it were the population probability. That is justified only under conditions that make the responder outcome probability equal to the population probability; the observed data alone do not establish those conditions. Another approach is to impose a model or restriction on \(p_M\). Such an approach may narrow the range or yield a point, but its conclusion then depends on the added restriction, not only on the response rate and responder outcomes.

The unassisted bounds can be wide when nonresponse is substantial. That width is not a defect in the calculation: it records how much the unobserved group could change the answer. These bounds also do not automatically accommodate covariates, multiple treatment groups, or restrictions linking outcomes across groups. Those settings require their own targets and assumptions. Here the target is one marginal population probability for one binary outcome.

Common Mistakes and Diagnostics

• Equating the responder rate with the population rate. The value \(p_R\) describes responders. The target is \(p=q p_R+(1-q)p_M\), which also depends on nonresponders unless \(q=1\) or a suitable assumption is introduced.
• Using the wrong denominator. The responder outcome probability is \(y_R/n_R\), not \(y_R/n\). The latter is the observed positive-outcome contribution to the whole sample and equals \(\hat q\hat p_R\).
• Calling the bounds a confidence interval. The endpoints arise from allowing \(p_M\) to range over all possible binary-outcome probabilities. Sampling uncertainty is not included merely by calculating these endpoints from sample proportions.
• Defining \(p_M\) when nobody is missing, or \(p_R\) when nobody responds. Conditional probabilities for empty groups are undefined. Handle \(q=1\) directly as \(p=p_R\); when \(q=0\), \(p_R\) is unavailable and the unrestricted range is \([0,1]\).
• Claiming the endpoints are only conservative approximations. Under the stated information and restrictions, they are attainable. Any narrower range must use information or assumptions beyond \(q\), \(p_R\), and the binary scale.

A useful diagnostic is to check the endpoints against the implied group assignments: the lower bound sets all missing outcomes to 0, and the upper bound sets all missing outcomes to 1. Also verify that the calculated interval lies within \([0,1]\), contains \(q p_R\), and has width \(1-q\) when \(0<q<1\). If these checks fail, inspect the denominators and arithmetic.