Intervals With Adaptive Randomization

Why Adaptive Allocation Changes Interval Construction

When treatment assignment probabilities change in response to accumulating outcomes, the observed treatment groups are not formed by a fixed allocation rule. Participants assigned later may have different probabilities of receiving each treatment because earlier outcomes affected the design. An interval that treats the final allocation as if it had been fixed in advance can therefore misrepresent uncertainty.

For a continuous outcome, a useful starting point is an inverse-probability weighted score using the known probability of the treatment actually assigned. The score corrects for unequal assignment probabilities. Its values are dependent across participants through the adaptive rule, but its centered increments have a useful sequential structure: each increment has conditional mean zero given the information available before the next assignment. This structure gives a variance calculation that accounts for adaptation without pretending that the realized allocation probabilities were fixed independently of the outcomes.

The inferential target here is the difference in marginal mean outcomes in the population represented by the enrolled participants. This setup assumes participants' potential-outcome pairs are independent and identically distributed, assignment is randomized using the stated \(p_i\) given the recorded history, and outcomes are observed for assigned participants. It also assumes positivity: for some fixed \(\epsilon>0\), \(\epsilon\leq p_i\leq1-\epsilon\) at every assignment. Positivity prevents a treatment's weight from becoming arbitrarily large. The probabilities must be recorded correctly; an unknown or misrecorded allocation rule is not covered by the calculation below.

Variance From the Sequential Allocation Rule

Write \(m_{21}=E[Y_i(1)^2]\) and \(m_{20}=E[Y_i(0)^2]\). Conditional on \(\mathcal F_{i-1}\), the new participant's potential outcomes have the population distribution, and assignment uses probability \(p_i\). Consequently,

The first equality follows by averaging over the two possible assignments: the treatment-1 contribution has expectation \(p_i E[Y_i(1)]/p_i=\mu_1\), while the treatment-0 contribution has expectation \(-(1-p_i)E[Y_i(0)]/(1-p_i)=-\mu_0\). For the second equality, the conditional second moment of \(Z_i\) is \(m_{21}/p_i+m_{20}/(1-p_i)\); subtracting the square of its conditional mean gives the variance.

Set \(D_i=Z_i-\Delta\). The conditional mean of each \(D_i\) given prior information is zero. For \(j<i\), \(D_j\) is part of the information available before assignment \(i\), so \(E[D_jD_i]=E[D_jE(D_i\mid\mathcal F_{i-1})]=0\). Thus the increments are uncorrelated, even though the assignment probabilities depend on past outcomes. It follows that the unconditional variance of the estimator averages the expected conditional variances over the possible histories:

This is an unconditional variance: it averages over outcomes and the random adaptation process. It is generally not valid to condition on the entire realized sequence \(p_1,\ldots,p_N\) and then simply add the displayed conditional variances evaluated at those values. Later probabilities can encode earlier outcomes, so conditioning on the whole sequence can also condition on outcome information. The sequential variance argument instead uses the probability \(p_i\) as it was determined before participant \(i\)'s assignment.

A Plug-In Interval for a Continuous Outcome

The population second moments and \(\Delta\) are unknown. One plug-in approach estimates the moments using inverse-probability weighted squared outcomes:

Under the stated assignment rule, each summand for treatment 1 has conditional expectation \(m_{21}\), and each summand for treatment 0 has conditional expectation \(m_{20}\). A plug-in estimate of the variance of \(\hat\Delta\) is

With bounded allocation probabilities, suitable finite moments, and a large enough sample for a central limit approximation, a two-sided approximate 95% Wald interval is \(\hat\Delta\pm1.96\widehat{SE}\). Its interpretation is in the units of the outcome. The nominal coverage is over repeated outcomes and assignments generated by the adaptive design; it does not promise the same coverage after conditioning on a particular realized adaptation path.

Diagnostics, Alternatives, and Limitations

The allocation probabilities should be examined before interpreting an interval. Probabilities close to zero or one make the inverse weights large and can cause one or two observations to dominate both the estimate and its variance. Inspect the range of \(p_i\), the largest weights, and the contribution of each participant to \(\hat\Delta\). A probability outside the recorded design rule, a missing outcome, or an estimated variance that is negative indicates that the stated procedure cannot be applied as written or that the plug-in calculation is unstable.

A common mistake is to substitute the realized probabilities into the variance formula and call the result the variance conditional on the observed sequence of probabilities. That reasoning treats those probabilities as if they were independent of the outcomes that generated the adaptation. The correction is to distinguish predictable probabilities—known just before each assignment—from the full realized sequence. Use the sequential variance expression for unconditional inference over the design, and do not claim conditional coverage for a selected path without a method that establishes that guarantee.

Another tempting approach is to calculate a standard error from the final treatment-group means as though the participants had been assigned with fixed probabilities. Such a calculation does not, by itself, account for adaptive assignment or the unequal weights in the estimator above. Model-based outcome regression can be more precise when its assumptions are appropriate, but it relies on a correctly specified outcome model; a direct IPW calculation instead relies on the known allocation mechanism and positivity. Neither removes the need to check sample size, outcome tails, and influential weights. In particular, a normal approximation can be poor with few participants or highly variable outcomes even when the allocation rule is known.

The displayed interval is for a marginal continuous-outcome contrast under the stated population and design assumptions. It is not an interval conditional on selection of an adaptation path, a subgroup chosen after observing outcomes, or an unrecorded allocation mechanism. If the target population, outcome model, or assignment rule differs from these assumptions, the interval's coverage interpretation may not apply.