Positivity Under Strong Temporal Allocation Shifts

Why Temporal Positivity Matters

Adaptive allocation can change substantially as a trial proceeds. If the probability of assigning a participant to one treatment becomes very small during a particular enrollment period, that period may contribute few participants on that treatment. This matters even when the allocation probabilities are known exactly: weighting can correct the unequal probabilities in expectation, but it cannot create information that the trial did not collect.

The issue is not simply whether both treatments were assigned at some point during the trial. A time-averaged treatment effect relies on comparisons within the enrollment periods represented in its target population. When one arm is rare in a period, its outcomes there are weakly represented; when an arm has no participants in a period, a period-specific comparison cannot be calculated from the observed outcomes alone.

For a target distribution assigning proportion \(q_t\) to period \(t\), with \(\sum_t q_t=1\), a common target is the time-standardized effect \(\tau_Q=\sum_t q_t\{E[Y(1)\mid T=t]-E[Y(0)\mid T=t]\}\). Here \(Y(1)\) and \(Y(0)\) are the potential outcomes under treatment and control. The target weights \(q_t\) describe the population or time distribution for which the effect is wanted; they need not equal the proportions enrolled in the trial. This tutorial focuses on whether the observed allocation supports estimating this target, not on choosing a trend model.

In an adaptive trial, \(e_i\) may depend on prior assignments or outcomes as well as enrollment time. A period-level average probability is therefore a useful first summary, but it can hide near-zero probabilities for particular histories within that period. Use the actual probabilities generated by the randomization procedure, at the level relevant to the analysis.

What Near-Zero Probabilities Do to Weighting

Inverse-probability weighting gives more influence to participants whose observed assignment was unlikely under the design. A treated participant receives weight \(1/e_i\), while a control participant receives weight \(1/(1-e_i)\). Under the randomization probabilities and appropriate sequential randomization assumptions, this corrects the unequal representation induced by assignment in expectation. The basic reason follows from the treatment indicator:

The corresponding control identity is \(E[(1-A_i)Y_i/(1-e_i)\mid T_i,H_i,Y_i(1),Y_i(0)]=Y_i(0)\). These equalities require the stated assignment probabilities and that assignment is randomized conditional on the history used to define them. They explain unbiased representation in repeated randomization; they do not guarantee a stable estimate in one realized trial.

When \(e_i\) is small, the treated weight \(1/e_i\) is large. A few rare treated participants can then dominate the weighted estimate. This usually increases sampling variability; if the probability is exactly zero in a relevant part of the target, the corresponding potential outcome is not represented at all. No finite weight can recover information from an arm that was never assigned there.

A simple practical check is to compare period-specific expected counts, \(n_t\bar e_t\) and \(n_t(1-\bar e_t)\), with the realized counts in each arm. Here \(n_t\) is the number enrolled in period \(t\), and \(\bar e_t\) is the period’s average treatment probability. Expected counts are a planning diagnostic, not a guarantee of realized counts. Also examine the actual assignment probabilities and the distribution of weights, rather than relying only on an overall treatment proportion.

Weights, Effective Information, and a Short Calculation

The effective sample size summarizes how uneven weights reduce the information in a weighted group. For weights \(w_1,\ldots,w_m\), define \(ESS=(\sum_j w_j)^2/\sum_j w_j^2\). It equals \(m\) when all weights are equal and is smaller when a few observations carry much of the total weight. It is a diagnostic rather than a replacement for a variance estimate.

A useful justification for using the effective sample size is immediate from its formula: if all \(m\) weights equal \(c\), then \(ESS=(mc)^2/(mc^2)=m\). If one weight dominates, the denominator grows relative to the squared sum, so the effective sample size falls. This diagnostic does not measure bias, and it does not account for outcome variability or dependence; it should be read alongside period-specific counts and the outcome analysis.

Does the Target Effect Have Support?

Support is a property of the combination of the target and the observed allocation, not a single trial-wide number. Ask which periods receive positive target weight, whether both treatments have nonzero assignment probabilities there, and whether realized data provide enough observations in each arm to estimate the period-specific contrast credibly. If the target places substantial weight on a period with almost no treatment observations, the effect may be formally identifiable under a model but weakly supported by direct comparisons.

Outcome regression and weighting address the problem differently. Weighting explicitly rebalances participants according to their assignment probabilities and can become unstable when those probabilities are extreme. Outcome regression models outcomes as a function of treatment, time, and possibly other variables; it may produce predictions in a sparsely represented arm, but those predictions depend heavily on the model’s assumptions. Neither approach can turn absent overlap into direct evidence. With good overlap, the two approaches can offer complementary checks; with poor overlap, agreement between them is not proof that extrapolation is reliable.

Diagnostics and Common Mistakes

For a practical assessment, inspect allocation probabilities and data together. Plot or tabulate the probabilities over enrollment time; report period-specific arm counts; identify any zero counts; and summarize the treatment and control weight distributions, including their largest values and effective sample sizes. If probabilities vary with pre-assignment history, check that variation rather than relying on period averages alone. A probability plot can reveal an abrupt temporal shift, while the arm-by-period table shows whether the shift translated into sparse realized data.

A common mistake is to reason, “Both treatments were assigned somewhere in the trial, so the overall positivity assumption holds.” This confuses trial-wide representation with support in the periods contributing to the target. The correction is to examine each relevant period (and relevant assignment history): a large early control group cannot supply direct treated outcomes for a late period when treatment allocation there was nearly absent.

• Do not equate a nonzero probability with adequate information. A probability of 0.01 is positive, but it may yield very few treated participants.
• Do not interpret large weights as extra data. They expand representation in the weighted calculation but do not increase the number of independently observed outcomes.
• Do not treat a model prediction as observed overlap. Regression may fill a gap by extrapolation; state that dependence clearly.
• Do not silently change the target. Removing a poorly supported period changes the population or time distribution to which the effect refers.

The central limitation is that diagnosis does not itself repair a weak design. Extreme weights may produce unstable estimates, while model-based estimates in unsupported periods rely on assumptions that the data cannot check well. The appropriate interpretation distinguishes what is estimable with direct support from what is inferred through modeling.