From Observed Rates to Causal Effect Bounds
With a binary outcome, observed treatment and control rates are easy to compare. But their difference is not automatically the average causal effect: for each person, only the outcome under the treatment actually received is observed. The other potential outcome is missing. Worst-case bounds make this missing information explicit by allowing each unobserved binary outcome to be either 0 or 1.
The calculation is useful when a point estimate would require assumptions, such as no unmeasured confounding, that are not justified. It gives a range of average treatment effects compatible with the observed rates and the basic outcome scale. The range can include effects in both directions; it need not settle whether treatment helps or harms.
Consistency means that the observed outcome is \(Y=Y(1)\) for a treated unit and \(Y=Y(0)\) for an untreated unit. Thus \(p_1\) reveals the treated group’s mean \(Y(1)\), and \(p_0\) reveals the untreated group’s mean \(Y(0)\). The other two group-specific potential-outcome means are unobserved: \(E[Y(1)\mid D=0]\) and \(E[Y(0)\mid D=1]\).
For the population bounds below, the required setup is a well-defined binary treatment and outcome, consistency, and observed treatment prevalence and outcome rates. No assumption that treatment is independent of potential outcomes is imposed. In data, the corresponding calculation substitutes the sample treatment fraction and sample outcome proportions. Those substitutions estimate the endpoints; sampling uncertainty around estimated endpoints is a separate issue.
Calculating Each Endpoint
The average treatment effect is a weighted average over the treated and untreated groups. Within each group, the observed outcome rate supplies one potential-outcome mean, while the unobserved mean can be anywhere from 0 to 1. For the lower endpoint, set the missing treated potential outcome among untreated units to 0 and the missing untreated potential outcome among treated units to 1. For the upper endpoint, reverse those choices.
The observed terms are \(E[Y(1)\mid D=1]=p_1\) and \(E[Y(0)\mid D=0]=p_0\). Because the unobserved means lie between 0 and 1, minimizing the expression uses \(E[Y(0)\mid D=1]=1\) and \(E[Y(1)\mid D=0]=0\). Maximizing it uses \(E[Y(0)\mid D=1]=0\) and \(E[Y(1)\mid D=0]=1\). Substitution gives:
These are worst-case bounds: each endpoint makes the most adverse or most favorable assignment of the missing potential outcomes for the effect, given the observed rates. The earlier material on why the binary outcome bounds are sharp establishes that these endpoints are attainable, not merely conservative inequalities. For hand calculation, note that the upper endpoint minus the lower endpoint is exactly \(q+(1-q)=1\). This width is a useful arithmetic check.
Worked Example: A Lower and Upper Endpoint From Counts
Suppose hypothetical data contain 200 people: 80 treated and 120 untreated. Among the treated, 24 have the outcome; among the untreated, 18 have it. Calculate and interpret the plug-in worst-case bounds.
Step 1: Calculate the inputs. The treatment fraction is \(q=80/200=0.40\). The treated outcome rate is \(p_1=24/80=0.30\), and the untreated outcome rate is \(p_0=18/120=0.15\).
Step 2: Calculate the lower endpoint. Substituting the observed rates gives:
This endpoint assigns outcome 0 under treatment to untreated people, and outcome 1 under no treatment to treated people, for their respective missing outcomes.
Step 3: Calculate the upper endpoint.
The estimated worst-case range is therefore \([-0.37,\,0.63]\), or a reduction of 37 percentage points through an increase of 63 percentage points in the outcome. It includes zero and effects of either sign. These data and assumptions alone do not determine whether the average causal effect is positive or negative.
Check. The endpoints are ordered, both lie between \(-1\) and \(1\), and their difference is \(0.63-(-0.37)=1.00\).
Work the Calculation Carefully
A reliable hand calculation keeps the inputs and the two endpoint recipes separate. First compute \(q\), \(p_1\), and \(p_0\) from their correct denominators. Then evaluate the lower and upper expressions, preserving the factor \(1-q\) on the untreated-group terms. Finally interpret the units as differences in outcome probabilities, or percentage points if the endpoints are multiplied by 100.
Divide the number treated by the total number of units: \(q=n_1/n\). The untreated fraction is \(1-q\).
Divide outcome-positive treated units by all treated units to obtain \(p_1\). Divide outcome-positive untreated units by all untreated units to obtain \(p_0\).
Calculate \(q p_1-q-(1-q)p_0\) for the lower endpoint and \(q p_1+(1-q)-(1-q)p_0\) for the upper endpoint.
Confirm the lower endpoint does not exceed the upper, the width is 1, and each is between \(-1\) and \(1\). State the range in outcome-probability difference units.
Worked Example: Checking the Effect of a Large Treated Group
In another hypothetical data set, 60 of 100 people are treated. There are 36 outcome-positive people among the treated and 20 among the 40 untreated people. Calculate the bounds.
The inputs are \(q=60/100=0.60\), \(p_1=36/60=0.60\), and \(p_0=20/40=0.50\). The untreated fraction is \(1-q=0.40\).
The lower endpoint is:
The upper endpoint is:
The bounds are \([-0.44,\,0.56]\), or \(-44\) to \(56\) percentage points. Again, zero lies inside the range, so the worst-case bounds do not establish the direction of the effect. The width check gives \(0.56-(-0.44)=1.00\).
Worked Example: Interpreting Bounds When the Observed Rates Differ
In a hypothetical population, one quarter of units are treated. The observed outcome rate is 0.20 among treated units and 0.40 among untreated units. Calculate the worst-case bounds and compare them with the observed rate difference.
Here \(q=0.25\), \(p_1=0.20\), \(p_0=0.40\), and \(1-q=0.75\). The observed rate difference is \(p_1-p_0=0.20-0.40=-0.20\), but this is not by itself the average causal effect.
The lower endpoint is:
The upper endpoint is:
The bound is \([-0.50,\,0.50]\), or \(-50\) to \(50\) percentage points. The observed rate difference is within this range, but it is not generally equal to the causal effect: the two rates refer to different groups, and treatment assignment may be related to their potential outcomes. The broad bounds reflect that the missing potential outcomes are unrestricted beyond being binary.
Diagnostics, Alternatives, and Limitations
The calculation has simple checks, but passing them does not make the causal range narrow or prove the assumptions. If \(q\), \(p_1\), or \(p_0\) was calculated with the wrong denominator, the endpoints can be wrong even if the arithmetic is internally consistent. In particular, \(p_1\) uses only treated people and \(p_0\) only untreated people; neither is a proportion of the full sample.
- Endpoint order: the lower endpoint must be less than or equal to the upper endpoint. For these formulas, their difference must equal 1.
- Scale: an average difference between binary outcomes must lie between \(-1\) and \(1\). Endpoints outside that range signal a formula or arithmetic error.
- Units: a value such as 0.20 is a 20-percentage-point difference in outcome probability, not a 20 percent relative change.
- Observed arms: both treated and untreated outcome rates require observations in their respective groups. If an arm is absent, its observed rate cannot be calculated from the data, and this plug-in procedure cannot be applied as written.
A frequent mistake is to subtract the observed rates and call \(p_1-p_0\) the average treatment effect. That difference compares the observed treated group with the observed untreated group; the groups may differ for reasons related to treatment assignment. Another mistake is to treat the bounds as a confidence interval. They describe the range of causal effects compatible with the observed rates under the stated worst-case restrictions, not the sampling uncertainty of an estimated parameter. A confidence interval for estimated endpoints would address sampling variability separately.
These bounds require less causal structure than a point-identifying approach that assumes conditional or unconditional exchangeability. Their cost is that the range can be wide and may include effects of opposite signs. If justified assumptions are available, other methods can produce more informative estimates, but those conclusions depend on the assumptions added. The purpose of the worst-case calculation is to show what the observed rates and the binary scale alone can support, not to select an effect within the range.
Check Your Understanding
Try each question, then compare your work with the solutions below.
- In hypothetical data, \(q=0.50\), \(p_1=0.40\), and \(p_0=0.20\). Calculate both endpoints and state the width.
- A student calculates \(p_0\) by dividing the number of outcome-positive untreated people by the full sample size. What is wrong, and what denominator should be used?
- For hypothetical data, \(q=0.20\), \(p_1=0.50\), and \(p_0=0.25\). Calculate the bounds and state whether they determine the sign of the effect.
- Explain why bounds \([-0.10,\,0.90]\) are not a 95% confidence interval.
- If calculated endpoints are \(-0.30\) and \(0.60\), name one arithmetic check that fails and give a likely implication.
Solutions. (1) Lower: \(0.50(0.40)-0.50-0.50(0.20)=0.20-0.50-0.10=-0.40\). Upper: \(0.20+0.50-0.10=0.60\). The width is \(0.60-(-0.40)=1.00\). (2) The control outcome rate is conditional on being untreated, so its denominator is the number of untreated people, not the full sample. (3) Lower: \(0.20(0.50)-0.20-0.80(0.25)=0.10-0.20-0.20=-0.30\). Upper: \(0.10+0.80-0.20=0.70\). Since the range includes zero and both signs, it does not determine the effect’s sign. (4) The endpoints describe effects compatible with the observed rates under worst-case missing potential outcomes; they do not encode a 95% sampling-coverage procedure. (5) The width is \(0.60-(-0.30)=0.90\), not 1.00. At least one input or endpoint calculation should be checked.