Worst-Case Bounds For A Binary Outcome

Why Bound the Effect?

When outcomes are binary, the average treatment effect is a difference in probabilities: the average outcome under treatment minus the average outcome without treatment. But for each person, only one of those outcomes is observed. If treatment assignment may depend on the potential outcomes, the observed treatment-group means need not equal the population averages under treatment and no treatment. A useful first step is therefore to ask what the data alone can establish, without assuming that treatment assignment is unconfounded.

The answer is an interval rather than, in general, a single value. The interval uses the treatment prevalence and the observed outcome means in the two treatment groups. It allows the unobserved potential outcomes to take any value consistent with being binary. No random assignment, conditional exchangeability, or instrument is required for these bounds.

The outcome value 1 can represent an event, such as recovery, readmission, or disease. The ATE is then a difference in event probabilities, measured in risk-difference units. For example, an ATE of 0.05 represents an average increase of 5 percentage points in the probability of the outcome under treatment.

For the formulas below, assume treatment is well defined, the observed outcome equals the potential outcome under the treatment actually received, and there is no interference between individuals. These are consistency and no-interference conditions. Also assume \(0<p<1\), so both observed group means are defined. The bounds make no assumption about how treatment assignment relates to either potential outcome, and no assumption about the association between \(Y(1)\) and \(Y(0)\).

Deriving the Bounds

Consistency gives \(Y=Y(1)\) for treated individuals and \(Y=Y(0)\) for untreated individuals. Thus \(m_1\) reveals the mean of \(Y(1)\) among the treated, but it does not reveal the mean of \(Y(1)\) among the untreated. Since the missing potential outcome is binary, its mean must be between 0 and 1. Apply the law of total expectation to \(Y(1)\):

The unknown conditional mean \(E[Y(1)\mid D=0]\) lies in \([0,1]\). Multiplying by the nonnegative weight \(1-p\) and adding \(pm_1\) gives the bounds for the population mean under treatment. The same reasoning applies to \(Y(0)\): its mean among untreated individuals is observed as \(m_0\), while its mean among treated individuals is unknown but lies in \([0,1]\).

To bound the ATE, subtract the mean under no treatment from the mean under treatment. A lower bound results by taking the smallest allowed value of \(E[Y(1)]\) and subtracting the largest allowed value of \(E[Y(0)]\). An upper bound uses the largest allowed value of \(E[Y(1)]\) and the smallest allowed value of \(E[Y(0)]\). Therefore every ATE compatible with these restrictions must satisfy:

This derivation gives bounds that follow from the observed means and the binary range of the potential outcomes. It does not assume that the observed mean difference \(m_1-m_0\) is the ATE. That equality requires additional assumptions connecting treatment assignment to potential outcomes.

Understanding the Interval

The width of the interval has a simple form. Subtract the lower endpoint from the upper endpoint:

Thus, under these no-assumption bounds, the interval has width one risk-difference unit, regardless of the observed means or treatment prevalence. The prevalence changes where the interval sits, but not its width. The endpoints remain within the possible ATE range \([-1,1]\): the lower endpoint is at least \(-1\), and the upper endpoint is at most 1, because \(0\leq m_1,m_0\leq1\).

This fixed width helps explain why the bounds may be informative about the possible scale of the effect without resolving its direction. If the interval includes both negative and positive values, the observed data plus the binary outcome restriction do not determine whether treatment increases or decreases the outcome on average. More restrictions on the relationship between treatment and potential outcomes may narrow the interval, but they are substantive assumptions, not consequences of the observed group means alone.

Assumptions, Diagnostics, and Alternatives

The bounds depend on a small set of conditions. First, the outcome must genuinely be binary on the stated scale; if it is not restricted to 0 and 1, the missing-potential-outcome means need different bounds. Second, the treatment and outcome must be defined consistently so that the observed outcome corresponds to the potential outcome under the received treatment. Third, both treatment groups must be represented to use \(m_1\) and \(m_0\) as defined above. These are conditions on the setup and observed data, not tests that prove the bounds' causal interpretation.

Several checks can catch errors in applying the formulas. Confirm that \(p\), \(m_1\), and \(m_0\) are all proportions on the same population and that both observed means lie in \([0,1]\). Check that the lower endpoint does not exceed the upper endpoint and that both endpoints lie within \([-1,1]\). A result outside those ranges usually indicates a substitution or coding error. Also report whether an endpoint is a risk difference, not a relative risk or an odds ratio.

If the treatment prevalence is 0 or 1, one of the observed group means is undefined, so the displayed two-group formula cannot be used as written. The observed arm still identifies its own potential-outcome mean. For \(p=0\), if \(m_0=E[Y\mid D=0]\), then \(E[Y(0)]=m_0\) and \(E[Y(1)]\in[0,1]\), giving an ATE range \([-m_0,1-m_0]\). For \(p=1\), \(E[Y(1)]=m_1\) and \(E[Y(0)]\in[0,1]\), giving \([m_1-1,m_1]\).

The principal alternative is to make additional assumptions to obtain a narrower interval or a point estimate. For example, unconfoundedness asserts that, possibly conditional on measured covariates, treatment assignment is independent of the potential outcomes. Under suitable overlap and consistency conditions, adjustment methods can then identify an ATE. Such an estimate may be more precise, but its causal interpretation rests on the added assumptions. Instrumental-variable analyses use a different set of assumptions and may target a different effect. Neither approach is justified merely because its estimate is numerically narrower than the no-assumption interval.

These are population bounds. In an application, \(p\), \(m_1\), and \(m_0\) are usually estimated from a sample, so substituting sample proportions produces estimated endpoints, not a confidence interval for the ATE. Sampling uncertainty requires separate inferential methods. A narrow confidence interval around estimated endpoints would not remedy a failure of the causal assumptions, and a broad identification interval is not the same thing as sampling imprecision.

Common Mistakes

• Treating the observed difference as the causal effect. The reasoning “\(m_1-m_0\) is the ATE because those are the two observed means” ignores that the groups may differ before treatment. The correction is to distinguish factual outcomes from potential outcomes and use the stated bounds unless additional assumptions justify identification.
• Using the same missing-outcome contribution for both potential outcomes. \(Y(1)\) is observed for treated individuals, while \(Y(0)\) is observed for untreated individuals. The unknown treated-group contribution to \(E[Y(0)]\) is weighted by \(p\); the unknown untreated-group contribution to \(E[Y(1)]\) is weighted by \(1-p\).
• Calling the interval a confidence interval. The displayed interval describes values compatible with the population restrictions. Sample-based uncertainty around estimated endpoints is a separate issue.
• Concluding that an interval containing zero proves no effect. It says zero is among the values not ruled out by these restrictions; it does not establish that the true effect equals zero.