Why Bound an Effect Rather Than Fill In Missing Outcomes?
When some outcomes are missing, the observed data do not generally determine the average causal effect. A complete-case estimate or an imputation procedure supplies an answer by adding assumptions about the missing outcomes. Worst-case bounds take a different approach: they ask which treatment effects remain possible if missing outcomes may take any value allowed by the outcome scale. The result is an identified interval, not a point estimate.
For a binary outcome, the allowed values are 0 and 1. Without assumptions about why an outcome is missing, a missing outcome could be either value. This can leave a wide interval, but it makes the role of the data and of any extra assumptions explicit.
Assumptions and the Basic Binary-Outcome Bounds
Causal interpretation requires assumptions about treatment assignment, distinct from assumptions about missingness. In a randomized study, treatment assignment is independent of the potential outcomes. In an observational study, a common alternative is conditional exchangeability: \(A\) is independent of \((Y(0),Y(1))\) given measured covariates \(X\). Either setting also requires consistency, so the observed outcome equals \(Y(A)\) when it is observed, and positivity, so each treatment level occurs with positive probability in the population or covariate strata being analyzed.
These assumptions address treatment assignment; they do not make missingness ignorable. In particular, the bounds below allow the probability of observing an outcome to depend arbitrarily on its value. Let \(q_a=P(R=1\mid A=a)\) be the response rate in arm \(a\), and let \(m_a=E[Y\mid A=a,R=1]\) be the event rate among its respondents. In a randomized study, these arm quantities identify the corresponding observed-outcome summaries for the potential-outcome group. In an observational study, they need to be defined and used within covariate strata before standardizing.
The argument is a decomposition by response status. A fraction \(q_a\) has an observed mean \(m_a\); the remaining fraction \(1-q_a\) has an unknown binary-outcome mean between 0 and 1. Thus \[ \mu_a=q_a m_a+(1-q_a)u_a,\qquad 0\leq u_a\leq1, \] where \(u_a=E[Y(a)\mid R(a)=0]\) denotes the mean among those whose outcome would be missing under arm \(a\). Taking the smallest and largest possible values of \(u_a\) gives the arm-specific bounds. Subtracting the largest possible control mean from the smallest possible treated mean gives the lower bound for \(\tau\); subtracting the smallest control mean from the largest treated mean gives the upper bound.
These bounds are sharp under the stated assumptions: assigning all missing treated outcomes to 0 and all missing control outcomes to 1 attains the lower endpoint; reversing those assignments attains the upper endpoint. Intermediate assignments can produce effects between the endpoints. No assumption about the joint distribution of \(Y(1)\) and \(Y(0)\) is needed to bound their mean difference.
Worked Example: Worst-Case Bounds in a Randomized Study
Suppose a hypothetical randomized study assigns 100 people to each arm. In the treatment arm, 80 outcomes are observed and 48 are events. In the control arm, 90 outcomes are observed and 45 are events. Assume random assignment, consistency, and that the binary outcome is coded 1 for an event. Do not assume missingness is independent of the outcome.
The response rates and respondent event rates are \(q_1=80/100=0.80\), \(m_1=48/80=0.60\), \(q_0=90/100=0.90\), and \(m_0=45/90=0.50\). Therefore the treated mean is bounded by \[ 0.80(0.60)=0.48 \quad\text{and}\quad 0.80(0.60)+0.20=0.68. \] The control mean is bounded by \[ 0.90(0.50)=0.45 \quad\text{and}\quad 0.90(0.50)+0.10=0.55. \]
Subtracting across arms gives \[ 0.48-0.55=-0.07 \quad\text{and}\quad 0.68-0.45=0.23. \] Thus the worst-case ATE interval is \([-0.07,\,0.23]\), or from a 7-percentage-point decrease to a 23-percentage-point increase in event risk. It includes zero, so these data and assumptions do not determine even the direction of the effect.
Response Rates and Outcome Ranges
For an outcome known to lie in \([L,U]\), the same decomposition gives \[ q_a m_a+(1-q_a)L \leq \mu_a \leq q_a m_a+(1-q_a)U. \] The binary formula is the special case \(L=0\), \(U=1\). The width of an arm’s interval is \((1-q_a)(U-L)\). Higher response rates therefore narrow the worst-case interval, all else equal; if every outcome is observed, \(q_a=1\) and the arm mean is identified from the observed data. If no outcomes are observed, the responder mean is unavailable and the bound is simply \([L,U]\).
For binary outcomes, \([0,1]\) is the natural full outcome range. Narrower bounds require an additional, explicit restriction—not a more favorable calculation. For example, substantive knowledge might justify restricting the event risk among missing participants to a range \([\ell_a,u_a]\), where \(0\leq\ell_a\leq u_a\leq1\). This assumption changes the arm bounds to \(q_am_a+(1-q_a)\ell_a\) and \(q_am_a+(1-q_a)u_a\). The restriction is not learned from the observed outcomes alone and should be reported as an assumption.
Worked Example: Narrowing Bounds With a Missing-Outcome Restriction
Use the hypothetical study from the previous example. Suppose, in addition to its observed data, investigators judge that the event risk among missing treated participants is between 0.20 and 0.40, while the event risk among missing control participants is between 0.30 and 0.50. These are sensitivity assumptions about the missing groups, not consequences of randomization.
For treatment, \(q_1m_1=0.80(0.60)=0.48\) and \(1-q_1=0.20\). The restricted mean bounds are \[ 0.48+0.20(0.20)=0.52 \quad\text{and}\quad 0.48+0.20(0.40)=0.56. \] For control, \(q_0m_0=0.90(0.50)=0.45\) and \(1-q_0=0.10\). Its restricted mean bounds are \[ 0.45+0.10(0.30)=0.48 \quad\text{and}\quad 0.45+0.10(0.50)=0.50. \]
The resulting ATE interval is \[ 0.52-0.50=0.02 \quad\text{to}\quad 0.56-0.48=0.08. \] Under these added restrictions, the effect is between a 2- and an 8-percentage-point increase in event risk. The narrower interval is conditional on the missing-risk assumptions; it should not be presented as a conclusion from the observed data alone.
Covariate-Stratified Bounds for Observational Data
With observational treatment assignment, a crude comparison of arm-specific bounds may mix different covariate distributions. Under conditional exchangeability given \(X\), calculate bounds within each covariate stratum and average them over the target population’s distribution of \(X\). Define \(q_a(x)=P(R=1\mid A=a,X=x)\) and \(m_a(x)=E[Y\mid A=a,X=x,R=1]\). For binary outcomes, the standardized bounds are \[ \sum_x P(X=x)q_a(x)m_a(x) \leq E[Y(a)] \leq \sum_x P(X=x)\{q_a(x)m_a(x)+1-q_a(x)\}. \] For continuous \(X\), the corresponding sums are expectations over the target distribution of \(X\). Positivity is needed both for treatment assignment and for observing enough respondents to learn the conditional summaries in the strata used.
The target-population weights matter. The raw pooled quantities \(P(R=1\mid A=a)\) and \(E[Y\mid A=a,R=1]\) weight strata according to the distribution of \(X\) among people with \(A=a\). Those weights generally differ from the target distribution. Consequently, bounds averaged over the target distribution are not generally the same as bounds computed from raw pooled arm summaries.
Worked Example: Standardizing Rather Than Pooling
Consider a hypothetical observational population with two equally common strata, \(X=0\) and \(X=1\). Treatment probabilities are \(P(A=1\mid X=0)=0.90\) and \(P(A=1\mid X=1)=0.10\), so both strata have treatment positivity. Assume conditional exchangeability and consistency. In each stratum, half of outcomes are observed in either arm. Among treated respondents, event rates are \(m_1(0)=0\) and \(m_1(1)=1\). Among control respondents, they are \(m_0(0)=0.40\) and \(m_0(1)=0.60\).
For treatment, the stratum-specific intervals are \([0,0.50]\) at \(X=0\) and \([0.50,1]\) at \(X=1\). Averaging with the target weights \(0.50\) and \(0.50\) gives a standardized treated-mean interval \([0.25,0.75]\). For control, the stratum intervals are \([0.20,0.70]\) and \([0.30,0.80]\), giving the standardized control-mean interval \([0.25,0.75]\). The standardized ATE interval is therefore \([0.25-0.75,\,0.75-0.25]=[-0.50,0.50]\).
To see why raw pooling differs, among treated people the stratum weights are 0.90 and 0.10. The pooled response rate is 0.50 and the pooled respondent event rate is \(0.90(0)+0.10(1)=0.10\). The crude treated-mean bounds are consequently \([0.50(0.10),\,0.50(0.10)+0.50]=[0.05,0.55]\), not the target-standardized \([0.25,0.75]\). Among controls, the stratum weights are 0.10 and 0.90, so the pooled respondent event rate is \(0.10(0.40)+0.90(0.60)=0.58\); the crude control-mean bounds are \([0.29,0.79]\). These crude arm summaries do not target the equal-stratum population, and subtracting them would not give its standardized causal-effect bounds. Stratification alone does not guarantee a narrower interval; the essential point is to bound within strata and use the correct target-population weights.
How to Report and Interpret the Bounds
A worst-case identified interval is not a confidence interval. It describes uncertainty about the causal effect that remains even with the population observed-data quantities known exactly, under the stated assumptions. In finite samples, estimated response rates and respondent means also have sampling variability. An interval calculated by substituting sample proportions into the formulas is a plug-in estimate of the identification region; it does not by itself account for that sampling uncertainty.
Report the outcome coding, target population, treatment-assignment assumptions, response rates, and any restrictions imposed on missing outcomes. State the interval in meaningful units—for a binary outcome, usually percentage-point differences in risk—and distinguish assumptions supported by design or subject-matter knowledge from what the observed data identify.
Common Mistakes
- Treating missingness as harmless because treatment was randomized. Randomization balances treatment assignment, but it does not ensure that respondents represent everyone assigned to an arm. The bounds allow outcome-dependent missingness.
- Calling a plug-in bound a confidence interval. The bound describes identification under assumptions; sampling uncertainty needs separate treatment.
- Pooling before accounting for covariates. In observational data, raw arm summaries generally use arm-specific covariate weights. Standardize stratum-specific bounds to the target distribution instead.
- Reporting a narrower interval without its added assumptions. Restrictions on missing-group risks can tighten bounds, but the conclusion is conditional on those restrictions.
- Interpreting an interval that includes zero as proof of no effect. It means zero and nonzero effects remain compatible with the data and assumptions; it does not establish that the effect is zero.
Check Your Understanding
Use the definitions and bounds above to answer these questions.
- In a randomized arm, 70% of outcomes are observed and the respondent event rate is 0.30. What are the worst-case bounds for that arm’s event risk?
- Why does a higher response rate narrow a binary-outcome arm’s worst-case interval?
- What treatment-assignment assumptions support covariate-stratified causal bounds in an observational study?
- Why might a pooled bound based on raw arm summaries differ from a bound standardized to the target distribution of covariates?
- What does it mean if a worst-case ATE interval includes zero, and why is that interval not automatically a confidence interval?