Variance Estimation for Adaptive Treatment Contrasts

Why Variance Has Two Sources

In an adaptive trial, assignment probabilities can change as outcomes accumulate. That creates a variance question beyond the usual one: uncertainty comes from the outcomes, but also from which treatment each participant is randomized to and with what probability. When some assignments become unlikely, inverse-probability weights can amplify both sources.

This tutorial studies a two-arm inverse-probability contrast. Its variance decomposition makes the role of assignment probabilities explicit. The result is not a universal standard error for every adaptive estimator: it relies on a defined target, a specified randomization mechanism, and stable outcome distributions. Different estimators, including stabilized weighted estimators, require variance calculations that match their own formulas.

Definition: For participant $i$, let $A_i=1$ indicate assignment to treatment 1 and $A_i=0$ assignment to treatment 0. Let $Y_i(1)$ and $Y_i(0)$ be that participant’s potential outcomes, and let $Y_i=A_iY_i(1)+(1-A_i)Y_i(0)$ be the observed outcome. Write $H_{i-1}$ for the information available before participant $i$ is assigned, and let $\pi_i=P(A_i=1\mid H_{i-1})$ be the known randomization probability. The target is the superpopulation mean contrast $\Delta=\mu_1-\mu_0$, where $\mu_a=E[Y_i(a)]$. The inverse-probability contribution and estimator are $$ D_i=\frac{A_iY_i}{\pi_i}-\frac{(1-A_i)Y_i}{1-\pi_i}, \qquad \hat{\Delta}_{HT}=\frac{1}{n}\sum_{i=1}^{n}D_i. $$

The subscript $HT$ denotes the Horvitz–Thompson form. Each observed outcome is weighted by the inverse probability of the assignment that revealed it. The estimator’s target is a marginal treatment contrast, not the difference between whichever groups happen to have been observed. If assignment probabilities depend on recorded history, they must be the probabilities actually used by the randomization policy.

Assumptions for the Decomposition

The basic result uses sequential randomization: conditional on $H_{i-1}$ and the current participant’s potential outcomes, the assignment is generated with probability $\pi_i$, independently of those potential outcomes. It also assumes consistency (the observed outcome is the potential outcome under the assigned arm) and no interference between participants. For the simple formula below, potential outcomes are drawn from a stable superpopulation: their arm-specific means and variances do not change with $i$ or depend on the past history. Define $\sigma_a^2=\operatorname{Var}(Y_i(a))$.

Positivity is essential: probabilities must remain away from zero and one. A convenient sufficient condition is $\epsilon\leq\pi_i\leq1-\epsilon$ for some fixed $\epsilon>0$. Without it, a rare assignment can create a very large weight, and the variance can become enormous or even fail to be finite. The formula also presumes the probabilities are known from the design; if they are estimated rather than logged exactly, that estimation uncertainty needs separate attention.

Deriving the Two Contributions

Given the past history, $\pi_i$ is known. The contribution’s conditional expectation is $\mu_1-\mu_0=\Delta$. To derive its conditional variance, first condition on the assigned arm. If $A_i=1$, then $D_i=Y_i(1)/\pi_i$; if $A_i=0$, then $D_i=-Y_i(0)/(1-\pi_i)$. The law of total variance separates variation in outcomes within an assigned arm from variation in the arm-specific conditional expectations across assignments.

\begin{aligned} \operatorname{Var}(D_i\mid H_{i-1}) &=E\{\operatorname{Var}(D_i\mid A_i,H_{i-1})\mid H_{i-1}\}\\ &\quad+\operatorname{Var}\{E(D_i\mid A_i,H_{i-1})\mid H_{i-1}\}\\ &=\frac{\sigma_1^2}{\pi_i}+\frac{\sigma_0^2}{1-\pi_i} +\pi_i(1-\pi_i) \left(\frac{\mu_1}{\pi_i}+\frac{\mu_0}{1-\pi_i}\right)^2. \end{aligned}

For the first term, the conditional outcome variance is $\sigma_1^2/\pi_i^2$ in arm 1 and $\sigma_0^2/(1-\pi_i)^2$ in arm 0. Weighting by their assignment probabilities gives $\sigma_1^2/\pi_i+\sigma_0^2/(1-\pi_i)$. For the second term, the two possible conditional expectations are $\mu_1/\pi_i$ and $-\mu_0/(1-\pi_i)$. A two-point random variable with probabilities $\pi_i$ and $1-\pi_i$ has variance equal to the product of those probabilities times the squared difference between its values. That yields the final term.

Key result: The outcome contribution is $\sigma_1^2/\pi_i+\sigma_0^2/(1-\pi_i)$. The assignment contribution is $\pi_i(1-\pi_i)(\mu_1/\pi_i+\mu_0/(1-\pi_i))^2$. For a fixed, nonadaptive probability $\pi$ and independent participants, $\operatorname{Var}(\hat{\Delta}_{HT})$ is this per-participant variance divided by $n$. With adaptive probabilities, the corresponding variance is $$ \operatorname{Var}(\hat{\Delta}_{HT}) =\frac{1}{n^2}\sum_{i=1}^{n}E\!\left[ \frac{\sigma_1^2}{\pi_i}+\frac{\sigma_0^2}{1-\pi_i} +\pi_i(1-\pi_i) \left(\frac{\mu_1}{\pi_i}+\frac{\mu_0}{1-\pi_i}\right)^2 \right], $$ under the stated stable-outcome assumptions and sequential randomization.

The adaptive formula averages over the random histories that determine future assignment probabilities. The contributions $D_i-\Delta$ have conditional mean zero given the past, so distinct contributions are uncorrelated: for $j<i$, $D_j-\Delta$ is part of the past and $E(D_i-\Delta\mid H_{i-1})=0$. Thus the variance of their sum is the sum of their variances. In a realized trial, the sequence of probabilities is observed, but the unconditional variance averages over the possible sequences the policy could have generated.

Worked Variance Calculations

Worked Example: A Low Probability for Treatment 1

Suppose a hypothetical trial uses $\pi=0.25$ for each participant. The outcome means are $\mu_1=8$ and $\mu_0=5$ points, and the outcome variances are $\sigma_1^2=4$ and $\sigma_0^2=9$ points squared. These values satisfy the stable-outcome assumptions; the fixed probability is positive and below one. Calculate the variance of one weighted contribution.

The outcome contribution is $$ \frac{4}{0.25}+\frac{9}{0.75}=16+12=28 $$ points squared.

The assignment contribution is $$ 0.25(0.75)\left(\frac{8}{0.25}+\frac{5}{0.75}\right)^2 =0.1875(32+6.6667)^2 =0.1875(38.6667)^2 \approx 280.3 $$ points squared. Therefore the total variance per contribution is $28+280.3\approx308.3$ points squared. For $n=100$ independent participants with this fixed probability, the estimator’s variance is approximately $308.3/100=3.083$ points squared, giving a standard error of $\sqrt{3.083}\approx1.756$ points.

Here the assignment component is much larger than the outcome component. The weighted contrast can fluctuate substantially because a low-probability treatment arm receives large weights when it is assigned. The calculation does not imply that every adaptive trial has this variance; it describes this specified probability and outcome model.

Worked Example: Unequal Outcome Variances and a Different Allocation

Consider another hypothetical setting with $\pi=0.4$, $\mu_1=6$, $\mu_0=2$, $\sigma_1^2=1$, and $\sigma_0^2=4$. The probability is strictly between zero and one, and the arm-specific moments are assumed stable. Compute both contributions.

The outcome contribution is $$ \frac{1}{0.4}+\frac{4}{0.6}=2.5+6.6667=9.1667 $$ outcome units squared.

The assignment contribution is $$ 0.4(0.6)\left(\frac{6}{0.4}+\frac{2}{0.6}\right)^2 =0.24(15+3.3333)^2 =0.24(18.3333)^2 \approx80.67 $$ outcome units squared. The total is $9.1667+80.6667\approx89.83$ outcome units squared. For $n=100$ participants with this constant probability, the standard error is $\sqrt{89.8333/100}\approx0.9478$ outcome units.

If one retained only the outcome contribution, the resulting standard error would be $\sqrt{9.1667/100}\approx0.3028$ units. That calculation omits the assignment contribution and is not the standard error for this inverse-probability estimator under this model.

Estimating Variance From Observed Data

The formula involving $\mu_a$ and $\sigma_a^2$ is useful for understanding the design, but these population quantities are usually unknown. A design-aware empirical alternative uses the observed weighted contributions. Define $$ \hat{V}_{mart}=\frac{1}{n^2}\sum_{i=1}^{n}(D_i-\hat{\Delta}_{HT})^2. $$ Under bounded probabilities, finite moments, and conditions ensuring the normalized sum of conditional variances converges to a finite positive limit, this estimates the asymptotic variance associated with the realized sequence of contributions. It is a large-sample variance estimator, not an exact finite-sample identity. In particular, a trial with few participants or extreme weights can make it unstable.

For a trial with only a small number of assignments, a model-based plug-in calculation may also be considered, but it must estimate the arm-specific outcome moments and preserve the assignment term. Simple observed-arm means and variances need not estimate those moments reliably when allocation is adaptive. Any model-based method should state how those quantities are estimated and should respect the actual probabilities used by the policy.

Worked Example: An Empirical Contribution-Based Estimate

Suppose four hypothetical participants have logged probabilities $(0.5,0.25,0.5,0.75)$, assignments $(1,0,1,0)$, and observed outcomes $(6,3,4,2)$ points. The logged probabilities are all strictly between zero and one. Calculate the inverse-probability contributions and the empirical variance estimate.

The contributions are $6/0.5=12$, $-3/0.75=-4$, $4/0.5=8$, and $-2/0.25=-8$. Thus $$ \hat{\Delta}_{HT}=\frac{12-4+8-8}{4}=2 $$ points. The squared deviations from this estimate are $100,36,36,100$, respectively. Hence $$ \hat{V}_{mart}=\frac{100+36+36+100}{4^2} =\frac{272}{16}=17 $$ points squared, and the corresponding estimated standard error is $\sqrt{17}\approx4.123$ points.

This is a calculation of the estimator, not evidence that the asymptotic approximation is accurate with only four participants. The large standard error relative to the estimate also illustrates how weighted contributions can be highly variable.

Conventional Standard Errors, Diagnostics, and Limits

A conventional standard error for a difference of arm means, often written $s_1^2/N_1+s_0^2/N_0$, is derived for that arm-means estimator under its own sampling assumptions. It is not automatically a variance estimate for $\hat{\Delta}_{HT}$: the latter has weighted contributions and an assignment component. Even when the estimand is the same, changing the estimator changes the variance calculation. Under adaptive allocation, the realized arm counts may also be related to earlier outcomes, so treating them as if they arose from a fixed allocation can be unjustified.

Several diagnostics help identify problems before interpreting a standard error:

Inspect the probability range. Report the smallest and largest $\pi_i$, and examine the largest inverse-probability weights. Probabilities near zero or one indicate weak practical positivity even if no probability is exactly zero.
Check contribution concentration. If a few $D_i$ values dominate the sum of squared deviations, the variance estimate may be highly sensitive to individual participants.
Check stability of the outcome model. If arm-specific means or variances change over calendar time or with past outcomes, the stable-moment formula may not apply as written. A time-varying target or conditional outcome model may be needed.
Verify the recorded policy probabilities. Rounded, missing, or incorrectly reconstructed probabilities can make weighted contributions and their variance estimates wrong.

The decomposition also clarifies why an assignment contribution should not be treated as a universal penalty. It depends on the outcome means as well as on $\pi_i$. For the uncentered Horvitz–Thompson contrast, adding the same constant to both arm outcomes can change this component, even though $\Delta$ itself is unchanged. Centering, covariate adjustment, or a stabilized estimator can alter the variance, but each changes the estimator or its contribution formula. The variance result here should not be transferred mechanically to those alternatives.

A practical comparison is therefore: use the design-aware contribution-based variance for the stated inverse-probability estimator when its assumptions and large-sample conditions are credible; use a model-based variance only when its outcome and assignment models are defensible; and use the conventional arm-means formula only when the estimator and its assumptions warrant it. No standard error repairs positivity failure, an incorrect target, or a misspecified randomization record.

Common pitfall: “The arm sample variances are small, so the adaptive contrast must be precise.” This ignores variation from random assignment probabilities and inverse weighting. For the Horvitz–Thompson contrast, calculate or estimate variance from its weighted contributions, and check whether rare assignments create influential weights.

Check Your Understanding

Answer each question, then compare with the solutions below.

With $\pi=0.5$, $\mu_1=4$, $\mu_0=2$, $\sigma_1^2=2$, and $\sigma_0^2=3$, calculate the outcome and assignment contributions to the variance of one weighted contribution.
Why does the law of total variance produce two contributions in this setting?
What is the main problem if an adaptive policy assigns treatment 1 with probability $0.01$ for some participants?
Why is $s_1^2/N_1+s_0^2/N_0$ not automatically the standard error for the inverse-probability contrast?

Solutions. (1) The outcome component is $2/0.5+3/0.5=10$ outcome units squared. The assignment component is $0.5(0.5)(4/0.5+2/0.5)^2=0.25(12)^2=36$ units squared, for a total of $46$ units squared. (2) Conditional on the assigned arm, outcomes vary; across assignments, the conditional expected weighted contribution also varies. The law of total variance adds these within-arm and between-assignment terms. (3) The inverse weight for treatment 1 is $1/0.01=100$, so rare assignments can produce extreme contributions and unstable variance estimates. (4) That formula concerns a difference of arm means under its own assumptions, whereas the inverse-probability contrast uses differently weighted contributions and includes assignment-probability variation.