Martingale Scores For Adaptive Randomization

Why Center Assignment at Each Step?

In a response-adaptive trial, the probability of assigning the next participant to treatment can depend on the assignments and outcomes already observed. Thus a treatment indicator is generally not centered by one fixed allocation fraction. The relevant center at participant $i$ is the probability assigned by the design given the trial history immediately before that participant is randomized.

This sequential centering has a useful consequence: each centered assignment contribution has conditional expectation zero. Such contributions are called martingale differences. Their sum is a mean-zero score, even when the assignment probabilities change over time, provided the probabilities and score construction match the randomization mechanism and null hypothesis.

The result is a statement about the randomization distribution, not an assertion that the observed score must be near zero in every trial. A particular assignment path can produce a large score. The claim is that, under the stated conditions, averaging over the randomization at each step gives zero.

History, Assignment Probabilities, and Scores

Definition: Let $A_i=1$ indicate that participant $i$ is assigned to treatment and $A_i=0$ indicate control. Let $H_{i-1}$ denote all information available immediately before participant $i$'s assignment, including prior assignments and observed outcomes and any baseline information used by the algorithm. The sequential assignment probability is $$ \pi_i=P(A_i=1\mid H_{i-1}). $$ A sequence $D_i$ is a sequence of martingale differences with respect to the histories if $D_i$ is determined by the information available through step $i$, has finite expectation, and satisfies $E(D_i\mid H_{i-1})=0$ for every $i$.

For binary assignment, conditional on $H_{i-1}$, $A_i$ is Bernoulli with success probability $\pi_i$. Therefore $E(A_i-\pi_i\mid H_{i-1})=0$ and $\operatorname{Var}(A_i\mid H_{i-1})=\pi_i(1-\pi_i)$. These identities use the conditional probability actually implemented by the design. They do not require the $\pi_i$'s to be equal across participants.

A basic assignment-balance contribution is $(A_i-\pi_i)X_i$, where $X_i$ is a baseline covariate known before assignment. More generally, a predictable weight $W_i$—a quantity determined by $H_{i-1}$—gives $D_i=W_i(A_i-\pi_i)$, which has conditional mean zero. For an outcome-based treatment score, a null hypothesis is usually needed to make the outcome factor fixed under randomization.

Definition: Under the sharp null of no treatment effect for any participant, $Y_i(1)=Y_i(0)$ for every $i$, where $Y_i(1)$ and $Y_i(0)$ are the participant's potential outcomes under treatment and control. Write their common value as $y_i$. The observed outcome is then $Y_i=y_i$, regardless of $A_i$. A sharp-null score with fixed weights $c_i$ is $$ S_n=\sum_{i=1}^{n} c_i(A_i-\pi_i)y_i. $$ The potential outcomes and weights are held fixed in the randomization argument; the assignments, and possibly the probabilities determined by past assignments, vary over randomization paths.

The factors $c_i$ can all equal one, or can encode a prespecified contrast or scale. They may also be predictable weights. Under the sharp null, $y_i$ is fixed across assignment paths, so the current centered assignment still has conditional mean zero when multiplied by it. In an actual analysis, $y_i$ is obtained under this null by using the observed outcome; the null makes that outcome the same under either assignment.

Deriving the Mean-Zero Property

The key calculation is local to one randomization step. Suppose $c_i$ and $y_i$ are fixed given the potential-outcome schedule, or more generally are measurable before assignment. Then

E\{c_i(A_i-\pi_i)y_i\mid H_{i-1}\} =c_i y_i\{E(A_i\mid H_{i-1})-\pi_i\} =c_i y_i(\pi_i-\pi_i)=0.

If $D_i=c_i(A_i-\pi_i)y_i$, each $D_i$ is therefore a martingale difference. Summing and applying the law of iterated expectations gives

E(S_n)=\sum_{i=1}^{n}E(D_i) =\sum_{i=1}^{n}E\{E(D_i\mid H_{i-1})\}=0.

This derivation does not say that the increments are independent. They need not be: later $\pi_i$'s may depend on earlier assignments and outcomes. Instead, the increments are conditionally centered at each step. This is enough to make distinct increments uncorrelated, under finite second moments. If $i<j$, then $D_i$ is known by the time $H_{j-1}$ is observed, so

E(D_iD_j)=E\{D_i E(D_j\mid H_{j-1})\}=0.

Consequently, the variance of the raw sum is the sum of the increment variances:

\operatorname{Var}(S_n)=\sum_{i=1}^{n}E(D_i^2) =\sum_{i=1}^{n}E\{c_i^2y_i^2\pi_i(1-\pi_i)\}.

The quantity $\sum_i c_i^2y_i^2\pi_i(1-\pi_i)$ is the sum of the conditional increment variances, often called the predictable quadratic variation. In an adaptive design it can itself be random, because the probabilities can depend on the realized history. It is not generally correct to treat its realized value as the unconditional variance of $S_n$.

Worked Example: A Three-Participant Sharp-Null Score

Consider a hypothetical trial with three participants. Under the sharp null, their outcomes are fixed at 10, 8, and 12 points. The design uses treatment probabilities 0.50, 0.70, and 0.40, respectively; for this illustration, these probabilities are the ones specified along the realized history. The observed assignments are treatment, control, treatment, so $A=(1,0,1)$. Use $c_i=1$.

The score is $$ S_3=\sum_{i=1}^{3}(A_i-\pi_i)y_i. $$ Substituting the values gives $$ S_3=(1-0.50)(10)+(0-0.70)(8)+(1-0.40)(12) =5.00-5.60+7.20=6.60\text{ points}. $$

This realized score is positive, but that does not contradict its mean-zero property. At each step, conditional on the history, the expected centered assignment is zero. For example, the first contribution has conditional expectation $10\{0.50(1-0.50)+(1-0.50)(0-0.50)\}=0$; equivalently, it is $10E(A_1-0.50)=0$. The same conditional-centering argument applies at steps two and three.

The conditional variance contribution at step $i$ is $y_i^2\pi_i(1-\pi_i)$. Along the stated probability sequence, these are $25.0$, $13.44$, and $34.56$ points squared, for a total of $73.0$ points squared. If the later probabilities varied across histories, this sum would vary too; the unconditional variance would be its expectation over histories.

Predictable Covariate Scores and Outcome Scores

Not every useful martingale score needs an outcome null hypothesis. If $X_i$ is a baseline quantity known before assignment, then $D_i=(A_i-\pi_i)X_i$ has conditional mean zero under the randomization mechanism, regardless of treatment effects on outcomes. This can describe assignment balance along the trial. By contrast, multiplying a centered assignment by a response observed after assignment does not in general preserve conditional mean zero. For a treatment-effect score, the sharp null can supply the needed fixed response through null imputation.

Worked Example: A Predictable Baseline-Covariate Score

Suppose three hypothetical participants have baseline covariate values $X=(2,1,3)$, and the design's sequential treatment probabilities along a history are $(0.50,0.60,0.30)$. The realized assignments are $A=(1,1,0)$. Consider the balance score $B=\sum_i(A_i-\pi_i)X_i$.

Substitution and arithmetic give $$ B=(1-0.50)(2)+(1-0.60)(1)+(0-0.30)(3) =1.00+0.40-0.90=0.50. $$ The score is $0.50$ covariate units for this path. At each step $X_i$ is already known, so $E\{(A_i-\pi_i)X_i\mid H_{i-1}\}=X_i(\pi_i-\pi_i)=0$. Thus $E(B)=0$ over randomization, even if treatment changes outcomes. The realized score need not be exactly zero because randomization produces imbalance in particular samples.

For an outcome score under a constant-effect sharp null $Y_i(1)-Y_i(0)=\tau$ for every participant, define the null-adjusted response $R_i=Y_i-\tau A_i$. Under that null, $R_i=Y_i(0)$ for either assignment. Thus a score such as $\sum_i(A_i-\pi_i)R_i$ has the same conditional-centering justification, with the response imputed under the null. This is a sharp-null argument; a weak null stating only that an average effect is zero does not make each participant's adjusted response fixed under assignment.

Worked Example: Why a Raw Outcome Score Can Fail

Consider one hypothetical participant with treatment probability $\pi=0.60$, control potential outcome $Y(0)=10$ points, and treatment potential outcome $Y(1)=14$ points. The sharp null is false for this participant. If the score contribution is formed using the observed raw outcome, $D=(A-\pi)Y(A)$, then its conditional expectation is $$ E(D)=0.60(1-0.60)(14)+0.40(0-0.60)(10) =3.36-2.40=0.96\text{ points}. $$ Equivalently, $E\{(A-\pi)Y(A)\}=\pi(1-\pi)\{Y(1)-Y(0)\}=0.24(4)=0.96$. The contribution is not mean zero, because the response changes with the current assignment.

For the constant-effect value $\tau=4$ points, null adjustment gives $R=Y-4A$. If assigned control, $R=10-0=10$; if assigned treatment, $R=14-4=10$. Therefore $E\{(A-0.60)R\}=10\{0.60(0.40)+0.40(-0.60)\}=0$. The adjustment works because the hypothesized sharp effect makes the adjusted response invariant to assignment.

When the Construction Is Reliable

Before using a martingale score, check the conditions that make the conditional expectation calculation valid:

Correct history: $H_{i-1}$ includes the information on which the assignment algorithm actually conditions, and excludes information unavailable before assignment.
Correct probability: $\pi_i$ is the probability used by the design conditional on that history, not a final allocation fraction or a marginal inclusion probability substituted for it.
Valid centering: the assignment is conditionally Bernoulli with mean $\pi_i$, or the score is derived using the appropriate conditional mean for a different assignment variable.
Valid multiplier: a multiplier is predictable, or an outcome multiplier is fixed across assignment paths under the null being tested.
Finite moments: the contributions have finite expectations for the mean-zero argument and finite second moments for the variance decomposition.

A probability of zero or one still gives a centered contribution of zero, though it contributes no randomization variance. Probabilities strictly between zero and one are needed when the score is to contain random assignment variation at that step. Inverse-probability weights can require stronger positivity conditions, but they are not needed for the basic score defined here.

Diagnostics, Alternatives, and Limits

A useful diagnostic is to audit the score one assignment step at a time: reconstruct $H_{i-1}$, confirm the design's $\pi_i$, and verify that the multiplier is fixed under the null or known before $A_i$ is drawn. If the calculated conditional mean of a contribution is not zero, the martingale argument does not apply. Frequent causes include using a marginal treatment probability, omitting information used by the adaptive algorithm, or using an unadjusted post-assignment outcome under a non-sharp null.

Martingale scores complement, rather than replace, the design-based variance calculations from joint assignment probabilities. Joint probabilities describe dependence among assignments and can be used for variance calculations of a range of estimators. Martingale scores instead organize a statistic into sequential contributions centered at their conditional means. Their advantage is that conditional centering directly yields zero mean and orthogonality of increments. Their limitation is that the result depends on correctly specifying the sequential mechanism and the score's null or predictability conditions.

A mean-zero score is not automatically a standard normal statistic, an exact randomization $p$-value, or a valid score under a weak average-effect null. Nor does dividing by a random estimate of its scale automatically preserve mean zero. Those conclusions require additional arguments about the reference distribution, variance estimation, or the null hypothesis. Here the guaranteed result is the conditional mean-zero property and its resulting variance decomposition.

Common mistake: “The treatment probability averaged over the whole trial was 0.50, so I can center every assignment by 0.50 and the score is a martingale.” This is generally wrong when the design adapts. The correction is to center $A_i$ by $\pi_i=P(A_i=1\mid H_{i-1})$ at each step. Also check that an outcome multiplier is fixed under the null; conditional centering of assignment alone cannot repair an assignment-dependent multiplier.

Check Your Understanding

Answer each question, then compare with the solutions.

A design assigns participant $i$ to treatment with probability $\pi_i$ given $H_{i-1}$. Show that $W_i(A_i-\pi_i)$ is conditionally mean zero when $W_i$ is known before assignment.
Under a sharp null, $y=5$, $\pi=0.40$, and the observed assignment is control. Calculate the score contribution and its conditional variance.
Why are martingale-difference contributions uncorrelated across distinct times even when the assignment probabilities adapt?
A researcher uses $(A_i-\pi_i)Y_i$ as a treatment score under only a weak average-effect null. Is the mean-zero conclusion guaranteed? Explain.
Why is the realized sum of conditional variance contributions not necessarily the unconditional variance of the score in an adaptive design?

Solutions. (1) Since $W_i$ is known given $H_{i-1}$, $E\{W_i(A_i-\pi_i)\mid H_{i-1}\}=W_i\{\pi_i-\pi_i\}=0$. (2) The contribution is $(0-0.40)(5)=-2.00$ outcome units. Its conditional variance is $5^2(0.40)(0.60)=6.00$ squared outcome units. (3) For $i<j$, $D_i$ is known by $H_{j-1}$, so $E(D_iD_j)=E\{D_iE(D_j\mid H_{j-1})\}=0$. Adaptation does not change this calculation. (4) No. A weak average-effect null does not imply each participant's outcome is unchanged by assignment, so $Y_i$ may depend on $A_i$, preventing the required conditional-centering step. (5) The conditional probabilities, and hence the conditional variance contributions, may vary with the realized history. The unconditional variance is the expectation of their sum, not necessarily the sum observed on one path.