Verifying a Martingale CLT for an Adaptive Trial

Why verify the martingale conditions?

In an adaptive trial, the probability of assigning the next participant to an arm can depend on the trial history. Consequently, the assignment indicators are generally not independent and identically distributed. A central limit theorem for independent observations cannot simply be applied to them. A martingale central limit theorem (martingale CLT) is useful because it allows each new random increment to have a distribution that depends on the past, provided its conditional mean, accumulated conditional variance, and large-increment behavior satisfy appropriate conditions.

This tutorial verifies those conditions for a two-arm Bernoulli trial and obtains a joint limit for the randomization noise and the two arms’ response noise. The key object is the predictable quadratic variation: the sum of the one-step conditional covariance matrices. It is not, in general, the conditional covariance of the entire sum.

Trial model and martingale increments

There are $n$ sequential participants and two treatments. Let $\mathcal{F}_{i-1}$ denote all observed trial information immediately before participant $i$ is assigned. The probability of assigning treatment 1 is $\pi_i$, where $\pi_i$ is measurable with respect to $\mathcal{F}_{i-1}$. Write $A_i=1$ for treatment 1 and $A_i=0$ for treatment 2. Given $\mathcal{F}_{i-1}$, the randomization satisfies $P(A_i=1\mid\mathcal{F}_{i-1})=\pi_i$.

Assume that, conditional on the assigned treatment and the past, the binary response $Y_i$ has success probability $\theta_1$ on treatment 1 and $\theta_2$ on treatment 2. Here $0<\theta_1,\theta_2<1$. The randomization is independent of the current participant’s response conditional on the assigned treatment and past. Let $\mathcal{F}_i$ include the assignment and response for participant $i$.

Definition: Define the three-dimensional increment $$ D_i= \begin{pmatrix} A_i-\pi_i\\ A_i(Y_i-\theta_1)\\ (1-A_i)(Y_i-\theta_2) \end{pmatrix}. $$ The first coordinate is allocation randomization noise. The second and third are the response residuals contributed by the assigned arm. With the stated model, $E(D_i\mid\mathcal{F}_{i-1})=0$, so the partial sums of these increments form a martingale with respect to $\{\mathcal{F}_i\}$.

For the response coordinates, the conditional-mean claim follows by conditioning first on the assignment. For example, when $A_i=1$, $E(Y_i-\theta_1\mid A_i=1,\mathcal{F}_{i-1})=0$; when $A_i=0$, the second coordinate is zero. The corresponding reasoning applies to the third coordinate. Predictability of $\pi_i$ gives $E(A_i-\pi_i\mid\mathcal{F}_{i-1})=0$.

One-step covariance and its accumulated limit

Put $v_1=\theta_1(1-\theta_1)$ and $v_2=\theta_2(1-\theta_2)$. Conditional on $\mathcal{F}_{i-1}$, the first coordinate has variance $\pi_i(1-\pi_i)$. The second has variance $\pi_i v_1$, and the third has variance $(1-\pi_i)v_2$. All conditional covariances between distinct coordinates are zero. For instance, the covariance between allocation noise and the treatment 1 residual is zero because the residual has conditional mean zero given assignment; the two arm-specific residual coordinates cannot both be nonzero.

E(D_iD_i^\mathsf{T}\mid\mathcal{F}_{i-1}) = \begin{pmatrix} \pi_i(1-\pi_i)&0&0\\ 0&\pi_i v_1&0\\ 0&0&(1-\pi_i)v_2 \end{pmatrix}.

For the normalized sum $n^{-1/2}\sum_{i=1}^nD_i$, the predictable quadratic variation is the sum of these one-step conditional covariance matrices divided by $n$. It records the variance accumulated conditionally along the trial’s history. It is this matrix, not the conditional covariance of the full sum, that the martingale CLT requires to converge. In general, conditional on the initial information, the covariance of the full sum is the conditional expectation of the accumulated conditional covariance; it need not equal the realized predictable quadratic variation.

The conditions must allow for two different allocation averages. Suppose, in probability,

\frac{1}{n}\sum_{i=1}^n\pi_i\longrightarrow q, \qquad \frac{1}{n}\sum_{i=1}^n\pi_i(1-\pi_i)\longrightarrow r,

where $0<q<1$ and $r>0$ are constants. The first average determines the limiting fraction assigned to treatment 1. The second determines the variance of allocation randomization noise. These limits are separate assumptions: knowing the average of $\pi_i$ alone does not generally determine the average of $\pi_i(1-\pi_i)$.

Worked Example: Accumulated Conditional Covariance

Consider a hypothetical four-step segment with predictable assignment probabilities $0.4, 0.5, 0.7, 0.6$, and suppose $\theta_1=0.60$ and $\theta_2=0.30$. These values illustrate the calculation; four observations alone do not establish an asymptotic convergence condition.

The response variances are $v_1=0.60(0.40)=0.24$ and $v_2=0.30(0.70)=0.21$. The average assignment probability is $$ \frac{0.4+0.5+0.7+0.6}{4}=0.55. $$ The average conditional allocation variance is $$ \frac{0.4(0.6)+0.5(0.5)+0.7(0.3)+0.6(0.4)}{4} =\frac{0.94}{4}=0.235. $$ The two average conditional response variances are $0.55(0.24)=0.132$ and $0.45(0.21)=0.0945$. Thus the predictable quadratic variation for the normalized four-step sum is $$ \begin{pmatrix} 0.235&0&0\\ 0&0.132&0\\ 0&0&0.0945 \end{pmatrix}. $$ This is the accumulated conditional covariance along the specified probability sequence, not a claim that the full-sum covariance conditional on the initial information equals this realized matrix.

Worked Example: Why the Allocation Variance Limit Needs Its Own Check

Suppose a hypothetical design alternates assignment probabilities $0.2$ and $0.8$, with each used half the time. The average probability of treatment 1 is $q=0.5$, but the average one-step allocation variance is $$ \frac{0.2(0.8)+0.8(0.2)}{2}=0.16. $$

If one incorrectly substituted $q$ into $q(1-q)$, the proposed allocation variance would be $0.5(0.5)=0.25$, not $0.16$. The discrepancy occurs because the probabilities vary across steps. For this design, the allocation-noise variance limit is $r=0.16$, while the response-noise limits use $q=0.5$ and $1-q=0.5$. This example shows why convergence of the average probability does not by itself verify convergence of the allocation part of the predictable quadratic variation.

Lindeberg negligibility and the joint limit

The martingale Lindeberg condition requires that increments of order $\sqrt n$ make a negligible contribution to the normalized accumulated variance. In vector form, for every $\varepsilon>0$, it is enough to have

\frac{1}{n}\sum_{i=1}^n E\!\left[ \|D_i\|^2 \mathbf{1}\{\|D_i\|>\varepsilon\sqrt n\} \mid\mathcal{F}_{i-1} \right]\longrightarrow 0 \quad\text{in probability}.

Here it holds automatically. Each coordinate of $D_i$ has absolute value at most 1, so $\|D_i\|\leq\sqrt{3}$. For any fixed $\varepsilon>0$, once $n>3/\varepsilon^2$, the event $\{\|D_i\|>\varepsilon\sqrt n\}$ is empty for every $i$. Thus every term in the Lindeberg sum is then zero.

Worked Example: Checking a Lindeberg Threshold

For a hypothetical check at $\varepsilon=0.1$ and $n=400$, the threshold is $\varepsilon\sqrt n=0.1(20)=2$. Since $\|D_i\|\leq\sqrt{3}\approx1.732<2$, the indicator in every Lindeberg term is zero. More generally, $0.1\sqrt n>\sqrt{3}$ when $n>300$, so the same argument applies for all sufficiently large $n$. This is a bound-based verification; it does not depend on which particular Bernoulli outcomes were observed.

The multivariate martingale CLT now applies: the increments have conditional mean zero, their normalized predictable quadratic variation converges in probability to a fixed matrix, and the Lindeberg condition holds. The limiting matrix is

\Sigma= \begin{pmatrix} r&0&0\\ 0&qv_1&0\\ 0&0&(1-q)v_2 \end{pmatrix}, \qquad \frac{1}{\sqrt n}\sum_{i=1}^nD_i \ \xrightarrow{d}\ N_3(0,\Sigma).

The zero off-diagonal entries mean that the three coordinates of this joint Gaussian limit are independent. This does not assert that the finite-sample components are independent. The result is a joint limit for allocation noise and arm-specific response noise, not by itself a limit for every statistic that includes the predictable allocation probabilities.

From response noise to arm-specific estimates

Let $N_1=\sum_{i=1}^n A_i$, $N_2=n-N_1$, and let $\hat{\theta}_k$ be the observed response proportion on arm $k$, when that arm has at least one assignment. The allocation martingale satisfies $$ N_1-\sum_{i=1}^n\pi_i=\sum_{i=1}^n(A_i-\pi_i). $$ Because the conditional variance of each summand is at most $1/4$, the second moment of the sum is at most $n/4$. Hence its value divided by $n$ converges to zero in probability. Together with $n^{-1}\sum_i\pi_i\to q$, this gives $N_1/n\to q$ and $N_2/n\to1-q$ in probability. In particular, both arms have sample sizes tending to infinity in probability.

The exact ratio identities are $$ \sqrt n(\hat{\theta}_1-\theta_1) =\frac{n^{-1/2}\sum_iA_i(Y_i-\theta_1)}{N_1/n}, \qquad \sqrt n(\hat{\theta}_2-\theta_2) =\frac{n^{-1/2}\sum_i(1-A_i)(Y_i-\theta_2)}{N_2/n}. $$

Applying the joint limit above and Slutsky’s theorem to these ratios yields

\begin{pmatrix} n^{-1/2}\sum_i(A_i-\pi_i)\\ \sqrt n(\hat{\theta}_1-\theta_1)\\ \sqrt n(\hat{\theta}_2-\theta_2) \end{pmatrix} \xrightarrow{d} N_3\!\left( 0, \begin{pmatrix} r&0&0\\ 0&v_1/q&0\\ 0&0&v_2/(1-q) \end{pmatrix} \right).

Thus, for example, the treatment 1 response proportion has asymptotic variance $v_1/q$ on the $\sqrt n$ scale, or approximately $v_1/(nq)$ on the original scale. The limiting response estimates and allocation noise are independent in the joint Gaussian limit under these assumptions.

Worked Example: Interpreting the Joint Limit Numerically

Suppose a hypothetical trial satisfies the convergence conditions with $q=0.60$ and $r=0.22$, and has response probabilities $\theta_1=0.65$ and $\theta_2=0.35$. Then $v_1=v_2=0.65(0.35)=0.2275$. The limiting covariance matrix for allocation noise and the two scaled response estimates is $$ \begin{pmatrix} 0.22&0&0\\ 0&0.2275/0.60&0\\ 0&0&0.2275/0.40 \end{pmatrix} = \begin{pmatrix} 0.22&0&0\\ 0&0.3792&0\\ 0&0&0.5688 \end{pmatrix}. $$

For $n=2000$, the approximate standard errors of the two response proportions are $\sqrt{0.3792/2000}\approx0.01377$ on treatment 1 and $\sqrt{0.5688/2000}\approx0.01686$ on treatment 2. These are hypothetical large-sample approximations; they do not imply that the arm-specific estimates have those exact finite-sample standard errors.

Diagnostics, alternatives, and limitations

A practical verification should check the assumptions as well as the algebra. Confirm that $\pi_i$ is determined by information available before participant $i$ is assigned, and that the response model gives the stated conditional means and variances. Check that the averages of $\pi_i$ and $\pi_i(1-\pi_i)$ have the required limits, rather than inferring the latter from the former. The arm-specific variance terms also depend on a stable Bernoulli response model with fixed $\theta_1$ and $\theta_2$. If response probabilities change with time or history, their conditional variances must be analyzed and their averages shown to converge to suitable limits.

The independent-observation CLT is a simpler alternative when observations really are independent with a common distribution, but adaptive assignment generally removes that premise. A martingale CLT accommodates the history-dependent assignment by using conditional means and predictable quadratic variation. It still requires those conditional variances to stabilize and large increments to be negligible. Bounded Bernoulli increments handle the latter requirement here; rare-arm designs with vanishing allocation fractions may fail to provide the diverging arm sample sizes needed for the estimator limits.

Finally, do not confuse the martingale allocation noise with the full allocation proportion. The identity $$ \frac{N_1}{n}-q = \left(\frac{1}{n}\sum_i\pi_i-q\right) +\frac{1}{n}\sum_i(A_i-\pi_i) $$ has a predictable-probability component as well as a randomization-noise component. The martingale CLT controls the second component on the $\sqrt n$ scale, but says nothing by itself about the first component on that scale. A CLT for $N_1/n$ centered at $q$ therefore needs an additional limit theory for the predictable component.

Common pitfall: Replacing $n^{-1}\sum_i\pi_i(1-\pi_i)$ by $q(1-q)$ merely because $n^{-1}\sum_i\pi_i\to q$ is generally wrong. The replacement is justified under stronger conditions, such as $\pi_i\to q$ in a manner that implies the variance average converges to $q(1-q)$. Otherwise, verify the allocation-variance limit $r$ separately.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Why must the assignment probability $\pi_i$ be measurable with respect to $\mathcal{F}_{i-1}$?
If the average assignment probability converges to $q=0.5$, must the allocation-noise variance limit equal $0.25$? Explain.
What feature of the increments makes the martingale Lindeberg condition especially simple in this Bernoulli model?
What additional component prevents the martingale CLT alone from establishing a centered-at-$q$ CLT for $N_1/n$?

Solutions. (1) Predictability ensures the assignment probability is known from the pre-assignment history, so $E(A_i-\pi_i\mid\mathcal{F}_{i-1})=0$. (2) No. Alternating probabilities $0.2$ and $0.8$ have average $0.5$, but average assignment variance $0.16$, not $0.25$. (3) Each vector increment has norm at most $\sqrt{3}$, so for any fixed positive Lindeberg threshold its indicator is eventually zero. (4) The predictable term $n^{-1}\sum_i\pi_i-q$ also contributes to the allocation proportion; its behavior on the $\sqrt n$ scale requires a separate result.