Treatment-Effect Bounds With Arm-Specific Response Rates

Why Arm-Specific Response Matters

When outcomes are missing, the observed mean in a treatment arm is generally the mean among respondents, not the mean for everyone assigned to that arm. If the response rates differ between arms, the amount of information about each arm’s full mean differs too. A worst-case bound makes this uncertainty explicit without assuming that responders and nonresponders have the same outcomes.

The target here is the population average treatment effect: the mean outcome under treatment minus the mean outcome under control. We use the outcome’s known limits and each arm’s own response rate and respondent mean to find the smallest and largest values of that contrast compatible with the observed data. These are identification bounds: they describe which target values the observed data and stated assumptions allow, not how much sampling uncertainty there is.

Definition: Let $Y(1)$ and $Y(0)$ be the potential outcomes under treatment and control, and let $R=1$ indicate that the outcome is observed. The average treatment effect is $\tau=E[Y(1)]-E[Y(0)]$. For arm $a\in\{0,1\}$, define the response rate $\pi_a=P(R=1\mid A=a)$ and respondent mean $\mu_{a,\mathrm{obs}}=E[Y\mid A=a,R=1]$. Assume the outcome in either arm lies within a known range $[L,U]$, where $L<U$.

Assumptions and the Arm-Level Bounds

For a causal interpretation, suppose treatment is randomized: assignment $A$ is independent of the potential outcomes and potential response indicators. Also assume consistency, so a person assigned to arm $a$ has observed outcome $Y=Y(a)$ whenever that outcome is observed. Response may depend on the outcome; no missing-at-random assumption is imposed. Randomization connects each assigned arm to the corresponding potential-outcome population, while the known range limits the outcomes of nonrespondents.

The response rates and respondent outcome distributions are observable from the study, subject to sampling variability. The mean among nonrespondents is not observed. Because every outcome is between $L$ and $U$, their mean must also be between $L$ and $U$. Decomposing the full arm mean into respondents and nonrespondents gives the bounds directly.

\mu_a = \pi_a\mu_{a,\mathrm{obs}}+(1-\pi_a)\mu_{a,\mathrm{mis}}, \qquad L\leq \mu_{a,\mathrm{mis}}\leq U.

The expression is a weighted average: a fraction $\pi_a$ of the arm contributes its observed respondent mean, and the remaining fraction contributes the unknown nonrespondent mean. The smallest possible full mean occurs when every nonrespondent has outcome $L$; the largest occurs when every nonrespondent has outcome $U$. Thus, for each arm,

\ell_a=\pi_a\mu_{a,\mathrm{obs}}+(1-\pi_a)L, \qquad u_a=\pi_a\mu_{a,\mathrm{obs}}+(1-\pi_a)U.

The treatment-effect contrast subtracts the control mean from the treatment mean. To make this contrast as small as possible, use the smallest compatible treatment mean and the largest compatible control mean. To make it as large as possible, reverse those choices.

Theorem: Under the stated assumptions, the sharp worst-case bounds for the average treatment effect are $$ \ell_1-u_0\leq \tau\leq u_1-\ell_0. $$ “Sharp” means that every value in this interval is compatible with the observed arm-specific response rates and respondent outcome distributions, given only the stated outcome limits and assumptions.

Why the Bounds Are Sharp

For a derivation, first consider an arm $a$. Since $\mu_{a,\mathrm{mis}}$ can be no smaller than $L$, multiplying by the nonnegative missing fraction $1-\pi_a$ and adding the respondent contribution gives $\mu_a\geq\ell_a$. The same argument using $\mu_{a,\mathrm{mis}}\leq U$ gives $\mu_a\leq u_a$. Both endpoints can be attained: set every missing outcome in that arm to $L$ for the lower endpoint, or to $U$ for the upper endpoint. Intermediate missing means produce intermediate full means.

For the contrast, these arm-specific inequalities imply $\mu_1-\mu_0\geq\ell_1-u_0$ and $\mu_1-\mu_0\leq u_1-\ell_0$. The lower endpoint is attained by assigning $L$ to treatment-arm nonrespondents and $U$ to control-arm nonrespondents. The upper endpoint is attained by the opposite assignments. No restriction on the joint distribution of $Y(1)$ and $Y(0)$ is needed to bound their mean difference: expectation is linear, so the target depends on the two marginal means. The endpoint constructions can be coupled across individuals in a way that preserves the observed arm-specific data. This establishes sharpness rather than merely showing that the interval is valid.

Worked Example: Different Response Rates on a Bounded Scale

Suppose a hypothetical randomized study measures an outcome on a scale known to range from 0 to 100. The treatment arm has response rate $\pi_1=0.80$ and respondent mean $\mu_{1,\mathrm{obs}}=70$. The control arm has response rate $\pi_0=0.60$ and respondent mean $\mu_{0,\mathrm{obs}}=60$. Assume randomization and consistency, and assume the stated 0-to-100 limits apply to all outcomes, including missing ones.

For treatment, the missing fraction is $1-0.80=0.20$. Its full mean is at least $0.80(70)+0.20(0)=56$ and at most $0.80(70)+0.20(100)=76$. Thus $\mu_1\in[56,76]$ points.

For control, the missing fraction is $1-0.60=0.40$. Its full mean is at least $0.60(60)+0.40(0)=36$ and at most $0.60(60)+0.40(100)=76$. Thus $\mu_0\in[36,76]$ points.

The treatment-effect bounds are $[56-76,\;76-36]=[-20,40]$ points. The observed data and assumptions therefore allow an average treatment effect as low as 20 points below zero or as high as 40 points above zero. The respondent mean difference, $70-60=10$ points, is not the full-population effect unless additional assumptions justify using respondent means for everyone.

Worked Calculations and Interpretation

Worked Example: An Outcome Scale With a Negative Lower Limit

In a hypothetical randomized study, a symptom-change outcome is known to lie between $-10$ and 30 points. Treatment response is 90%, with respondent mean 8 points. Control response is 50%, with respondent mean 10 points. Assume these limits are justified for all participants and that randomization and consistency hold.

The treatment-arm mean lies between $0.90(8)+0.10(-10)=7.2-1=6.2$ and $0.90(8)+0.10(30)=7.2+3=10.2$ points. The control-arm mean lies between $0.50(10)+0.50(-10)=5-5=0$ and $0.50(10)+0.50(30)=5+15=20$ points.

Subtracting the largest control mean from the smallest treatment mean gives the lower contrast bound, $6.2-20=-13.8$ points. Subtracting the smallest control mean from the largest treatment mean gives the upper bound, $10.2-0=10.2$ points. Therefore $\tau\in[-13.8,10.2]$ points. The interval includes zero, but this is an identification statement, not a significance test: it says the observed data do not determine the sign of the average effect without further assumptions.

Worked Example: One Arm Is Fully Observed

Consider a hypothetical randomized study with outcomes bounded from 0 to 10 points. Everyone in the control arm responds, and the control mean is 5 points. In the treatment arm, 70% respond and the respondent mean is 6 points. Assume the outcome limits, randomization, and consistency are appropriate.

The control mean is known exactly from the response data: $\ell_0=u_0=1(5)+0(0)=5$ points. The treatment mean lies between $0.70(6)+0.30(0)=4.2$ and $0.70(6)+0.30(10)=7.2$ points.

The contrast therefore lies between $4.2-5=-0.8$ and $7.2-5=2.2$ points. Full response in one arm removes missing-outcome uncertainty in that arm, but it does not remove uncertainty from missing outcomes in the other arm.

Diagnostics, Comparisons, and Limitations

Before calculating bounds, check that the outcome limits are genuinely defensible for every participant, not merely for the observed values. A scale’s theoretical minimum and maximum are often suitable. An observed sample minimum and maximum are not automatically valid limits for unobserved outcomes. If an asserted limit is too narrow, the resulting bounds can exclude plausible effects and are not valid worst-case bounds.

Check each response rate and respondent mean separately by treatment arm. Using a pooled response rate or a pooled respondent mean in place of the arm-specific quantities changes the calculation and generally does not bound the target. Also verify that the respondent mean is calculated on the same outcome scale used for $L$ and $U$. If an arm has no respondents, its respondent mean is unavailable; the data provide only the range $[L,U]$ for that arm mean, and the formula involving $\mu_{a,\mathrm{obs}}$ should not be used as if that mean were observed.

The principal alternative is a point estimate based on an assumption about the missing outcomes, such as treating respondents as representative of nonrespondents within an arm. Such an assumption can narrow uncertainty, but it is not implied by the bounds calculation and may be untestable from the observed outcomes alone. Complete-case comparisons likewise compare respondent means; they do not in general equal the full-population treatment effect. The worst-case bounds avoid those extrapolations, at the cost of potentially being wide.

Finite bounds require a finite, justified outcome range. If outcomes are unbounded and no further restrictions are imposed, these worst-case calculations need not yield finite limits. Even with bounded outcomes, low response rates can make bounds wide because a large share of each arm’s mean is unconstrained. Conversely, high response rates shrink each arm’s interval: its width is $(1-\pi_a)(U-L)$. The contrast interval’s width is the sum of the two arm widths, $[(1-\pi_1)+(1-\pi_0)](U-L)$, under a common range. This shows directly how response rates and outcome range govern worst-case precision.

In practice, $\pi_a$ and $\mu_{a,\mathrm{obs}}$ are estimated from a sample, so substituting sample quantities produces estimated identification bounds. Those endpoints are not, by themselves, a confidence interval; sampling uncertainty around the bounds is a separate issue. A bound containing zero does not prove there is no effect, and a bound entirely above or below zero indicates a sign shared by all effects compatible with these assumptions, not a probability statement about the effect.

Common pitfall: Subtracting the two respondent means and calling the result the treatment effect silently assumes that the respondent means represent the full arm means. Instead, bound each arm’s full mean using its own response rate and the outcome limits, then subtract the arm intervals in the order required for a lower and an upper contrast bound.

Check Your Understanding

Answer each question using the arm-specific formulas. Solutions are included below the questions.

Outcomes lie in $[0,20]$. In a hypothetical randomized study, treatment has $\pi_1=0.50$ and respondent mean 12; control has $\pi_0=0.80$ and respondent mean 10. Find the treatment-effect bounds.
Why is the lower contrast bound formed from the lower treatment mean and the upper control mean?
With a common outcome range of width 50, what is the width of the identification interval if response rates are 0.90 in both arms?
If outcomes have no known finite limits and no other restrictions are imposed, can this worst-case method guarantee finite bounds? Explain.

Solutions. (1) Treatment mean bounds are $0.50(12)+0.50(0)=6$ and $0.50(12)+0.50(20)=16$. Control bounds are $0.80(10)+0.20(0)=8$ and $0.80(10)+0.20(20)=12$. Hence $\tau\in[6-12,\;16-8]=[-6,8]$ points. (2) Since $\tau=\mu_1-\mu_0$, making the difference smallest requires minimizing $\mu_1$ and maximizing $\mu_0$. (3) Each arm’s interval has width $(1-0.90)(50)=5$, so the contrast interval has width $5+5=10$ units. (4) No. Without finite outcome limits or other restrictions, the missing outcomes may be arbitrarily extreme, so finite worst-case bounds are not guaranteed.