Martingale Difference Sequences in Adaptive Trials

Why Centering Matters in an Adaptive Trial

In an adaptive trial, the probability of assigning the next participant to an arm may depend on allocations and responses already observed. The next allocation is therefore not generally independent of the trial’s past. Yet two useful kinds of random variation still have a precise mean-zero property once that past is taken into account: the difference between an allocation indicator and its conditional probability, and the difference between a response and its conditional mean. These are martingale differences.

The point is not to claim that successive allocations or responses are independent. Instead, each increment has conditional expectation zero given the information available just before it occurs. That property makes martingale difference sequences a basic way to represent uncertainty under adaptive sampling. Here the focus is on establishing the property and checking its conditions, not on drawing asymptotic conclusions from it.

Trial Information and the Definition

Suppose participants enter in order \(i=1,2,\ldots\), and there are \(K\) treatment arms. Let \(\mathcal{F}_{i-1}\) denote the information available immediately before participant \(i\) is assigned, and let \(\mathcal{F}_i\) include that participant’s assignment and observed response as well as the earlier information. Thus the information sets grow over time: \(\mathcal{F}_{i-1}\subseteq\mathcal{F}_i\). This is the timing convention needed to say what is known before each new random increment.

Write \(A_i\) for participant \(i\)’s assigned arm and \(I(A_i=k)\) for the indicator that the assignment is to arm \(k\). The design’s conditional allocation probability is \(\pi_{i,k}=P(A_i=k\mid\mathcal{F}_{i-1})\). It is determined by the observed past before the assignment is drawn, although its value may change from one participant to the next.

The condition is conditional, not merely unconditional. It implies \(E(D_i)=0\) by taking expectations, but the reverse implication is not enough: an increment can have overall mean zero while its conditional mean varies with the history.

Allocation Innovations

For each arm \(k\), define the allocation innovation \(D^A_{i,k}=I(A_i=k)-\pi_{i,k}\). The allocation rule makes \(\pi_{i,k}\) the conditional probability of the indicator being one, given the pre-assignment information. Therefore \(E(I(A_i=k)\mid\mathcal{F}_{i-1})=\pi_{i,k}\), and subtracting \(\pi_{i,k}\) gives a conditional mean of zero. The innovation is known by time \(i\), and its absolute value is at most one, so it is integrable.

For a fixed participant, these arm-specific innovations are linked: summing them over all arms gives zero, because exactly one arm is assigned and the allocation probabilities sum to one. They are not independent across arms. This linkage does not prevent each component from being a martingale difference.

Response Residuals Centered at Conditional Means

Let \(Y_i\) be the response observed for participant \(i\). For each arm \(k\), suppose the conditional response mean is \(m_{i,k}=E(Y_i\mid\mathcal{F}_{i-1},A_i=k)\), whenever arm \(k\) has positive conditional assignment probability. This formulation allows the mean to depend on the information already observed. A common special case is a fixed arm mean, \(m_{i,k}=\mu_k\).

The observed response residual is \(D^Y_i=Y_i-m_{i,A_i}\). If response distributions are integrable and the conditional means are finite, then the residual is integrable. To verify its conditional mean, condition first on the possible assignments. For every arm that can be assigned, its response residual has conditional mean zero by the definition of \(m_{i,k}\). Averaging those zero means over the assignment probabilities also gives zero.

The same argument gives a useful arm-specific version: \(D^Y_{i,k}=I(A_i=k)(Y_i-m_{i,k})\). Its conditional expectation is the assignment probability multiplied by the within-arm conditional residual mean, so it is zero as well. If the response is integrable and the conditional mean is finite, the weighted residual is integrable. These statements require conditional centering at the appropriate arm-specific mean; they do not require identical responses across participants.

Worked Examples: Response Conditions

Why the Conditional-Mean Calculation Works

The response calculation uses the law of iterated expectations: condition on the assignment, whose probabilities are known given \(\mathcal{F}_{i-1}\), and then average over those assignments. This short derivation is important because adaptive allocation changes the weights \(\pi_{i,k}\) as the trial proceeds. It does not invalidate the calculation: at each step, the probabilities used are the probabilities conditional on the actual past.

This argument needs no independence between increments. It does require that the conditional means and assignment probabilities match the specified sequential model. For example, if the response mean depends on earlier observed history, a fixed arm mean is a valid centering value only if it still equals the conditional mean at each relevant history.

Comparison, Diagnostics, and Common Pitfalls

An ordinary independent, mean-zero sequence is one special case of a martingale difference sequence relative to a suitable past-information filtration. The martingale condition is more flexible: it allows the distribution of an increment to depend on the past, provided its conditional mean is zero. Conversely, an unconditional mean-zero calculation alone is weaker and does not establish the martingale difference property.

A frequent mistake is to center every response by one pooled mean. In the Bernoulli example above, the mean of the response given the history is \(0.60(0.70)+0.40(0.40)=0.58\). If the assignment probability changes to \(0.80\) for arm 1, the conditional response mean becomes \(0.80(0.70)+0.20(0.40)=0.64\). Subtracting the old pooled value \(0.58\) would then give conditional mean \(0.06\), not zero. Subtracting the assigned-arm mean instead gives zero within either arm, regardless of the current allocation probabilities.

Other useful diagnostics are direct checks of the timing and model assumptions. Is \(\pi_{i,k}\) calculated from information available before assignment, rather than from the observed assignment itself? Does the stated response mean condition on both the relevant history and the assigned arm? Is the response integrable? For bounded outcomes, integrability follows from the bound; for unbounded outcomes, it must be checked from the response model. A wrong filtration, a misspecified conditional mean, or a response distribution with no finite absolute expectation can invalidate the claimed sequence.

The martingale difference property does not assert that residuals are independent, that their variances are constant, or that an arm will be sampled often. Nor does it by itself establish a law of large numbers or a central limit theorem. Those require additional conditions. The practical value here is more basic: it separates variation that is unpredictable on average given the past from predictable changes in allocation or response means.