Sensitivity of Ethical Targets to Assumed Response Rates

Why Response-Rate Assumptions Matter

An ethical allocation target can depend on assumed response probabilities in two ways. Those probabilities determine the expected responses under each allocation, and they determine the variability of an estimated treatment-control difference. When a target is subject to a precision requirement, changing the response-rate assumptions can therefore change both the best allocation within the feasible set and the feasible set itself.

This tutorial studies a two-arm trial with a binary response. It treats the assumed response probabilities as inputs to a planning calculation, not as known truths. The aim is to make the sensitivity calculation explicit: specify the assumptions, recalculate the variance constraint, and then optimize the stated welfare objective over the allocations that satisfy it.

Model, Assumptions, and the Precision Constraint

For the planning calculation, assume each participant has a binary response, participants' responses are independent within and between arms, and the response probability is constant within each arm. For an allocation with \(n_T=Nx\) treatment participants and \(n_C=N(1-x)\) control participants, the sample response proportions estimate \(p_T\) and \(p_C\). Under this model, the variance of their difference is

This is a planning variance: it uses assumed probabilities in place of unknown probabilities. It presumes both arms receive participants, so \(0<x<1\). Suppose the design requires this variance to be no greater than a chosen bound \(V_{\max}\). A smaller bound demands more precision. The bound is a design choice that must be justified in the application; the calculations below use \(V_{\max}=0.0055\) as a hypothetical requirement.

To derive the feasible range, write \(a=p_T(1-p_T)/N\) and \(b=p_C(1-p_C)/N\). Since \(x(1-x)>0\), multiplying the variance inequality by this positive quantity preserves its direction:

The feasible allocations are the values between the two roots of this quadratic, when real roots in \((0,1)\) exist. The variance expression diverges as \(x\) approaches 0 or 1 when the corresponding arm has positive response variance. Thus an adequate precision constraint generally excludes allocations too close to either extreme.

Worked Calculations Under Alternative Assumptions

What the Sensitivity Analysis Shows

Across these hypothetical scenarios, the treatment targets are about 67.84%, 74.31%, and 30.60%. These are not three estimates of a single data-driven target. They are conditional planning results: each follows from its own assumed response probabilities, the same total sample size, the same variance bound, and the same expected-response objective.

The variance constraint changes with the response probabilities because the Bernoulli variance \(p(1-p)\) changes. For equal arm variances, a change in those variances changes the allowable range symmetrically around one-half. When the arm variances differ, the feasible range need not be symmetric. The objective can also change direction if the assumed ordering of \(p_T\) and \(p_C\) changes. Sensitivity analysis must recalculate both the variance-feasible range and the preferred endpoint.

The calculations depend on their assumptions. Independence, stable arm-specific response probabilities, a fixed total sample size, and the chosen variance cap are all part of the planning model. If response probabilities vary across participant subgroups, if outcomes are dependent, or if the estimator and variance model differ from the ones used here, the displayed variance formula may not apply. A planning target is only as defensible as its model and inputs.

Diagnostics, Alternatives, and Limitations

Several checks help identify a faulty sensitivity calculation. First, verify that every variance term is nonnegative and uses the correct arm's probability and allocation fraction. Second, substitute each reported boundary back into the original variance expression; rounding should produce a value close to, but not materially above, \(V_{\max}\). Third, check that the chosen endpoint maximizes the stated objective: use the upper endpoint when \(p_T>p_C\), the lower when \(p_T<p_C\), and acknowledge indifference in expected response count when the probabilities are equal. Finally, verify that expected counts are calculated as \(N[xp_T+(1-x)p_C]\), not by multiplying a response probability by an inconsistent allocation count.

A common mistake is to vary \(p_T\) and \(p_C\) in the welfare calculation while leaving the variance-feasible range unchanged. That is generally incorrect: the assumed response probabilities also determine the Bernoulli variances in the constraint. Another mistake is to describe an assumed rate as if it were observed evidence. Scenario calculations do not quantify how likely each scenario is, nor do they establish which assumed rates are correct.

Equal allocation is a simple alternative: it gives each arm half the participants and can be compared with the constrained target, but it is not automatically optimal for the stated welfare objective. Unconstrained welfare maximization would allocate everyone to the arm with the larger assumed response probability, which fails the variance requirement when that allocation leaves no comparison arm. The constrained target balances the specified welfare objective against the specified precision requirement. Other objective functions or constraints can yield different targets; conclusions should not be transferred to them without recalculation.

The method has limitations even when each calculation is correct. It treats assumed response probabilities as fixed inputs, whereas they are uncertain and may be misspecified. A small set of scenarios reveals how the target changes under those scenarios, but it is not a probability distribution over targets and does not guarantee robustness to values outside the chosen range. Report the assumptions, the constraint, and the resulting target together so that the conditional nature of the recommendation is clear.