Confidence Sequences Under Adaptive Assignment

Why Use a Confidence Sequence?

In an adaptively randomized trial, the probability of assigning treatment may change as outcomes accrue. Analysts may also inspect results repeatedly. A confidence interval calculated for one prespecified final sample size does not, by itself, provide valid coverage after repeated looks or data-dependent stopping. A confidence sequence addresses this by providing a sequence of intervals whose coverage holds simultaneously over time, under the stated design and outcome assumptions.

This tutorial constructs such a sequence for the average treatment effect among the units accrued by each time $t$. The construction uses inverse-probability weighting and a time-uniform test-martingale bound. It allows treatment effects to vary across units; it does not assume a constant effect.

Setup and Assumptions

For each unit $i$, let $Y_i(1)$ and $Y_i(0)$ be the potential outcomes under treatment and control, and let $A_i$ equal 1 for treatment and 0 for control. The observed outcome is $Y_i=A_iY_i(1)+(1-A_i)Y_i(0)$. Write the unit-level treatment effect as $\tau_i=Y_i(1)-Y_i(0)$, and define the target after $t$ assignments as the average effect among the first $t$ accrued units.

\bar{\tau}_t=\frac{1}{t}\sum_{i=1}^{t}\tau_i.

The target is a finite-sample average over the accrued units, not automatically an effect for a broader population. The units’ potential outcomes are fixed before their own assignments, and the unit sequence is fixed in advance or accrued by a rule based only on information available before each assignment. Assume consistency and no interference: each unit receives its assigned treatment, and one unit’s assignment does not change another unit’s potential outcomes.

Let $\mathcal{H}_{i-1}$ denote the observed history available just before assigning unit $i$, including earlier assignments and outcomes. The adaptive design must use a known, predictable treatment probability $p_i=P(A_i=1\mid\mathcal{H}_{i-1})$, with $0<p_i<1$. Predictable means $p_i$ is determined from the history before $A_i$ is drawn. For a finite, useful bound, assume the probabilities are bounded away from 0 and 1, for example $\epsilon\leq p_i\leq1-\epsilon$ for a fixed $\epsilon>0$.

Assume also that both potential outcomes satisfy a known bound $L\leq Y_i(a)\leq U$, for $a\in\{0,1\}$ and every unit. Set $R=U-L$. The interval $[L,U]$ and its units must be specified in advance; the bound is a substantive assumption, not a limit estimated from the observed outcomes. Subtracting $L$ from both potential outcomes leaves each treatment effect unchanged, while placing the shifted outcomes in $[0,R]$.

Definition: For unit $i$, define the inverse-probability weighted contrast $$ Z_i=\frac{A_i(Y_i-L)}{p_i}-\frac{(1-A_i)(Y_i-L)}{1-p_i}. $$ Under the stated randomization and consistency assumptions, its conditional expectation given the pre-assignment history is $\tau_i$. The cumulative estimation error is $M_t=\sum_{i=1}^{t}(Z_i-\tau_i)$.

From the Randomization to a Martingale Bound

Conditional on $\mathcal{H}_{i-1}$, the potential outcomes and $p_i$ are fixed, while $A_i$ is drawn with treatment probability $p_i$. Thus $Z_i=(Y_i(1)-L)/p_i$ if $A_i=1$, and $Z_i=-(Y_i(0)-L)/(1-p_i)$ if $A_i=0$. Taking the conditional expectation gives $p_i(Y_i(1)-L)/p_i-(1-p_i)(Y_i(0)-L)/(1-p_i)=\tau_i$. Consequently $Z_i-\tau_i$ has conditional mean zero, and $M_t$ is a martingale.

The same outcome bounds control the increment’s range. The two possible values of $Z_i$ differ by $(Y_i(1)-L)/p_i+(Y_i(0)-L)/(1-p_i)$, which is nonnegative and at most $R/p_i+R/(1-p_i)=R/[p_i(1-p_i)]$. Subtracting $\tau_i$ shifts both possible values equally, so the range length of $Z_i-\tau_i$ is at most $c_i=R/[p_i(1-p_i)]$. This bound is known at the time of assignment.

For a conditionally mean-zero random variable with range length at most $c_i$, Hoeffding’s lemma gives a conditional exponential-moment bound. Therefore, for every fixed real $\lambda$,

E_{t}(\lambda)=\exp\left\{\lambda M_t-\frac{\lambda^2}{8}V_t\right\}, \qquad V_t=\sum_{i=1}^{t}c_i^2,

is a nonnegative supermartingale starting at 1. Here the notation $E_t(\lambda)$ names a process, not an expectation. Ville’s inequality states that a nonnegative supermartingale starting at 1 has probability at most $1/k$ of ever reaching $k$. Thus a large value of this process is evidence against the martingale-centering model, with an error bound that holds over all times, not only at a chosen final time.

A convenient two-sided bound follows by mixing these processes over $\lambda$ with a normal density of mean 0 and variance $1/\rho$, where $\rho>0$ is a chosen scale in squared outcome units. A mixture of nonnegative supermartingales is again a nonnegative supermartingale. Evaluating the Gaussian integral gives

\mathcal{E}_t= \sqrt{\frac{\rho}{\rho+V_t/4}} \exp\left\{\frac{M_t^2}{2(\rho+V_t/4)}\right\}.

Since $\mathcal{E}_0=1$, Ville’s inequality implies $P(\text{there exists }t\geq1:\mathcal{E}_t\geq1/\alpha)\leq\alpha$. Solving $\mathcal{E}_t<1/\alpha$ for $|M_t|$ yields the time-uniform boundary

b_t(\alpha)= \sqrt{2\left(\rho+\frac{V_t}{4}\right) \left[ \log\left(\frac{1}{\alpha}\right) +\frac{1}{2}\log\left(1+\frac{V_t}{4\rho}\right) \right]}.

The logarithmic term involving $V_t$ allows the boundary to hold over an unlimited sequence of looks. The choice of $\rho$ affects the width of the bound, not its stated coverage guarantee.

Inverting the Bound

The observable cumulative contrast is $\sum_{i=1}^{t}Z_i$, and its difference from the cumulative treatment effect is exactly $M_t$. Replacing the unknown $\sum_i\tau_i$ by a candidate value $t\theta$, the bound $|M_t|\leq b_t(\alpha)$ gives a set of plausible values of $\bar{\tau}_t$.

Theorem: Under the assumptions above, the intervals $$ C_t= \left[ \frac{1}{t}\sum_{i=1}^{t}Z_i-\frac{b_t(\alpha)}{t}, \quad \frac{1}{t}\sum_{i=1}^{t}Z_i+\frac{b_t(\alpha)}{t} \right] $$ satisfy $$ P\left(\bar{\tau}_t\in C_t\text{ for every }t\geq1\right)\geq1-\alpha. $$ The probability refers to the random assignments under the design, with the potential outcomes held fixed. If the effect is known to lie in a narrower range, the interval may be intersected with that range without reducing coverage.

The justification is direct: $\bar{\tau}_t\in C_t$ exactly when $\left|\sum_{i=1}^{t}Z_i-\sum_{i=1}^{t}\tau_i\right|\leq b_t(\alpha)$, or $|M_t|\leq b_t(\alpha)$. The test-martingale bound guarantees this inequality simultaneously for all $t$ with probability at least $1-\alpha$. Inverting the bound therefore produces simultaneous coverage, rather than a collection of unrelated pointwise intervals.

Worked Example: A Sequence With Equal Assignment Probabilities

Suppose outcomes are binary, so $L=0$, $U=1$, and $R=1$. In a hypothetical trial, four units have $p_i=0.5$. The observed assignments and outcomes are: treated with outcome 1; control with outcome 0; treated with outcome 0; control with outcome 1. These are the only observed outcomes; the unobserved potential outcomes remain unknown.

For $p_i=0.5$, $Z_i=2Y_i$ for a treated unit and $Z_i=-2Y_i$ for a control unit. The four contributions are therefore $2,0,0,-2$, so $\sum_i Z_i=0$ and the estimated average effect is $0/4=0$ outcome units.

Each range length is $c_i=1/[0.5(0.5)]=4$, so $V_4=4(4^2)=64$. Take $\rho=1$ squared outcome unit and $\alpha=0.05$. Then

b_4(0.05)= \sqrt{2(1+64/4) \left[\log(20)+\tfrac12\log(17)\right]} =\sqrt{34(2.9957+1.4166)} \approx12.25.

The resulting interval is $0\pm12.25/4$, or approximately $[-3.062,3.062]$ outcome units. Since binary potential outcomes imply $-1\leq\bar{\tau}_4\leq1$, intersecting with this known range gives $[-1,1]$. The example illustrates both the validity and the conservatism of a generic bounded-increment guarantee when only four assignments are available.

Worked Example: Unequal Adaptive Probabilities

Consider three hypothetical units with binary outcomes and assignment probabilities $p_1=0.4$, $p_2=0.6$, and $p_3=0.5$. Suppose the observed data are: treated with outcome 1, control with outcome 0, and control with outcome 1, respectively. These probabilities may have been selected from the preceding history; the calculation requires that each was known before its assignment.

The contributions are $1/0.4=2.5$, $0$, and $-1/(1-0.5)=-2$. Thus $\sum_i Z_i=0.5$, and the estimated average effect is $0.5/3=0.1667$ outcome units.

The range lengths are $c_1=1/[0.4(0.6)]=4.1667$, $c_2=4.1667$, and $c_3=4$. Hence $V_3\approx4.1667^2+4.1667^2+4^2=50.72$. With $\rho=1$ and $\alpha=0.10$,

b_3(0.10)= \sqrt{2(1+50.72/4) \left[\log(10)+\tfrac12\log(1+50.72/4)\right]} \approx9.94.

The confidence sequence interval at $t=3$ is approximately $0.1667\pm9.94/3=[-3.147,3.480]$. Intersecting with the binary-outcome effect range $[-1,1]$ gives $[-1,1]$. The calculation also shows why assignment probabilities near the extremes inflate the bound: the inverse-probability weights have a larger range.

Diagnostics, Alternatives, and Limitations

Before interpreting a sequence, check the inputs and design records. Confirm that each $p_i$ is the probability actually used, conditional on the information available before assignment; that neither probability is zero; and that the outcome bound applies to both potential outcomes, not merely to the observed values. Check that assignments follow the recorded randomization mechanism and that enrollment of each unit precedes its own assignment. A violation can invalidate the martingale argument, not merely make the interval less efficient.

A confidence sequence is different from repeatedly recalculating an ordinary fixed-time 95% interval. Such fixed-time intervals generally guarantee coverage at one specified time under their own assumptions; using them at many times does not automatically preserve 95% coverage for the whole sequence. A sharp-null randomization test, discussed in the earlier material on randomization inference, answers a different question: it tests a fully specified sharp null that determines all missing potential outcomes. The sequence here targets an average effect and permits heterogeneous effects, but pays for that generality with a potentially wide bound.

A key pitfall is to claim that $Z_i-\theta$ is a mean-zero increment whenever the cumulative average effect equals $\theta$. Under heterogeneous effects, its conditional expectation is $\tau_i-\theta$, which need not be zero for each unit—even if the average of the $\tau_i$'s at a particular time equals $\theta$. The valid construction centers each increment by its own fixed $\tau_i$ in the proof, obtains a simultaneous bound on the total estimation error, and then inverts that bound for the average effect. This does not make the unobserved $\tau_i$'s available for computation.

The interval can be conservative, especially with few units, wide outcome bounds, or extreme assignment probabilities. It is also tied to the stated finite-sample target and design assumptions; it does not automatically account for interference, outcome-bound violations, or a change in the enrollment population. More specialized bounds may improve precision, but require additional justified assumptions. The main advantage here is transparent validity across repeated looks, including a stopping time chosen using the observed history, provided the design and bounds continue to hold.

Worked Example: Outcome Units and the Meaning of the Target

Suppose in a hypothetical study both potential outcomes are bounded between 2 and 5 hours of symptom-free time. Then $R=3$ hours, and subtracting $L=2$ before weighting does not change the treatment-effect scale. If a unit is treated with observed outcome 5 hours and $p_i=0.5$, its contribution is $Z_i=(5-2)/0.5=6$ hours. If instead it is a control unit with observed outcome 2 hours and $p_i=0.5$, its contribution is $Z_i=-(2-2)/0.5=0$ hours.

For $p_i=0.5$, the increment range length is $c_i=R/[p_i(1-p_i)]=3/0.25=12$ hours. The weighted contributions need not themselves lie between $-3$ and 3 hours; inverse-probability weighting can enlarge their range. The parameter $\bar{\tau}_t$, however, remains an average of differences between outcomes each in $[2,5]$, so $-3\leq\bar{\tau}_t\leq3$ hours. Any reported interval should identify its units and may be intersected with that logical effect range.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why must $p_i$ be known before assignment, and what does “bounded away from 0 and 1” accomplish?
If $R=2$ outcome units and $p_i=0.5$, calculate the increment range length $c_i$.
Why does the confidence sequence cover the accrued-sample average effect at all times rather than only at one selected time?
Explain why a zero average effect at time $t$ does not imply that every $Z_i$ is conditionally centered at zero.
What should be done if the known outcome range is $[2,5]$ hours and the computed confidence interval extends below $-3$ hours?

Solutions. (1) Predictability ensures the conditional assignment probability used in the weighting calculation is the one governing the next random assignment. Staying away from 0 and 1 prevents unbounded inverse-probability weights and gives finite increment bounds. (2) $c_i=R/[p_i(1-p_i)]=2/(0.5\cdot0.5)=8$ outcome units. (3) The nonnegative test-martingale and Ville’s inequality bound the probability of crossing the boundary at any time, so the inverted inequalities hold simultaneously. (4) Conditional centering is by the unit effect $\tau_i$; under heterogeneity, $\tau_i$ can differ from the average even when the average is zero. (5) Intersect the computed interval with the logically possible effect range $[-3,3]$ hours, while reporting the outcome units and the assumptions that justify the bound.