Consistency of Arm-Specific Means Under Adaptive Sampling

Why Adaptive Sampling Need Not Prevent Consistency

In a response-adaptive trial, the probability of assigning a participant to a treatment can depend on responses observed earlier. The resulting allocation count is random, and it may be associated with the observed history. That dependence does not, by itself, invalidate the sample mean within a treatment. The key distinction is between how often a treatment is selected and the conditional distribution of its response when selected.

This tutorial gives a sufficient condition for the mean of the responses observed on each arm to converge to that arm's response mean. The result is an almost-sure consistency statement: with probability one, the estimation error tends to zero as the number of assignments grows. It does not require an arm's fraction of all assignments to settle at a positive value. It does require that the arm continue to be sampled, and that its conditional response mean remain stable.

Setup and Assumptions

Consider a sequential trial with a fixed finite number \(K\) of treatments. At assignment \(t\), let \(A_t\in\{1,\ldots,K\}\) denote the selected treatment, and let \(Y_t\) be the response observed after that assignment. For arm \(i\), write \(I_{i,t}=1\) if \(A_t=i\), and \(I_{i,t}=0\) otherwise. The number assigned to arm \(i\) by time \(n\) and its observed sample mean are

Let \(\mathcal H_{i,k-1}\) denote the information available immediately before the response from the \(k\)th assignment to arm \(i\) is generated. This includes the past trial history and the fact that arm \(i\) has just been selected; it does not include that response. Write \(Y_{i,k}\) for this \(k\)th observed response on arm \(i\). A sufficient set of assumptions is:

• Stable conditional mean: for every \(k\), \(E(Y_{i,k}\mid\mathcal H_{i,k-1})=\mu_i\), where \(\mu_i\) is a fixed, finite arm-specific mean.
• Bounded conditional second moment: for some finite constant \(C_i\), \(E\{(Y_{i,k}-\mu_i)^2\mid\mathcal H_{i,k-1}\}\le C_i\) for every \(k\).
• Persistent sampling: \(N_i(n)\to\infty\) almost surely as \(n\to\infty\).

The response assumption says that, conditional on the information available before an arm-\(i\) response, its expected value is \(\mu_i\). Its conditional distribution may otherwise depend on the past. In the familiar special case where each arm has an independent, identically distributed response distribution with finite variance and mean \(\mu_i\), the first two conditions hold. The more general conditional formulation makes explicit what is needed when the trial is adaptive.

Why the Result Holds

The essential step is to index the observations by their order within arm \(i\), rather than by calendar assignment time. Define the centered response \(D_{i,k}=Y_{i,k}-\mu_i\). By the conditional-mean assumption, these centered responses have conditional expectation zero given the information before their respective responses. They are therefore martingale differences with respect to the history observed at successive arm-\(i\) responses. Their conditional second moments are bounded by \(C_i\).

For clarity, fix one arm and omit its subscript. Consider the weighted partial sums \(\sum_{k=1}^m D_k/k\). The martingale differences \(D_k/k\) have a finite total sum of expected conditional variances:

The martingale convergence theorem consequently gives convergence almost surely of \(\sum_{k=1}^m D_k/k\) as \(m\to\infty\). Kronecker's lemma states that if a series \(\sum_{k=1}^{\infty}x_k/k\) converges, then \(m^{-1}\sum_{k=1}^m x_k\to0\). Applying it to \(x_k=D_k\) yields

At calendar time \(n\), the arm-specific mean is exactly the mean of the first \(N_i(n)\) responses indexed within that arm. Persistent sampling says this random index tends to infinity almost surely. Substituting \(m=N_i(n)\) into the within-arm convergence therefore gives

This proof explains why adaptive assignment does not automatically spoil consistency. Assignment decisions can depend on past responses, but the next response on an arm still has the specified conditional mean, and enough responses from that arm eventually accumulate. The denominator is random, yet it is simply the number of terms in the within-arm average.

Infinite Sampling Is Not the Same as a Positive Allocation Fraction

Persistent sampling requires \(N_i(n)\to\infty\); it does not require \(N_i(n)/n\) to converge to a positive constant. An arm may receive an increasingly small share of assignments and still be sampled infinitely many times. Its mean can then be consistent, although this theorem gives no guarantee that convergence is fast or that a usual standard-error approximation is accurate.

Failure Modes and Diagnostics

The theorem is a sufficient result, not an automatic property of every adaptive trial. Its assumptions clarify what can go wrong and what should be checked before interpreting an arm-specific average as an estimate of a stable mean.

• Sampling stops or is too sparse. If the assignment rule can permanently exclude an arm, its count may remain finite, so its sample mean cannot be justified by this result. A design-level lower bound on assignment probabilities, or a direct argument that the count diverges, is more informative than observing that the arm was assigned several times early in a trial.
• The conditional mean changes. If later responses on an arm have a different conditional mean because of time trends, changing eligibility, or a response process altered by prior treatment, then there may be no fixed \(\mu_i\) to which the raw mean should converge. Check whether the response model makes the conditional mean stable for the actual sequence of participants and trial times.
• Conditional moments are too large. A bounded conditional second moment is a convenient sufficient condition. Extremely heavy-tailed responses, or variance that grows without control along the sequence, fall outside the stated proof. A few finite-sample summaries do not verify this assumption; it must be justified from the response model or supported by substantive knowledge.
• The response model changes under selection. The assumption concerns the response conditional on the history and the arm selection. If selection changes which response distribution is being sampled in a way that changes its conditional mean, the observed arm mean may target a changing mixture rather than one fixed arm mean.

A diagnostic review should therefore examine the allocation mechanism and the response model together. Plotting \(N_i(n)\) over time can reveal an arm whose accrual is slowing or stopping, but a plot cannot establish infinite sampling. Monitoring response averages can reveal drift, but apparent stability in a finite trial does not prove that the conditional mean is constant. The assumptions are primarily design and model conditions, not tests that can be conclusively certified from the observed data.

How This Result Differs from Related Methods

Under fixed allocation, the ordinary strong law of large numbers gives consistency for an arm mean when responses on that arm are identically distributed with finite mean (and suitable independence conditions). The adaptive result above uses a martingale strong-law argument instead: it permits dependence through the past, provided each response has the same conditional mean and the moment condition holds. In both settings, infinitely many observations from the arm are essential to the stated argument.

An inverse-probability-weighted estimator is a different construction: it reweights observations using their assignment probabilities and can target a population quantity under appropriate conditions. It is not needed merely to prove consistency of the raw within-arm sample mean under the conditional response assumptions stated here. Conversely, if those assumptions fail, this theorem does not establish that reweighting will repair the problem; that requires its own estimand and assumptions.

Consistency also does not specify the distribution of estimation error at a particular rate. A count that diverges very slowly can still yield almost-sure convergence while leaving substantial uncertainty at a finite sample size. A central limit theorem requires additional conditions and is a separate conclusion; it should not be inferred from consistency alone. Likewise, consistency of arm-specific means says nothing by itself about convergence of allocation proportions or the validity of a causal interpretation without an appropriate trial design and estimand.