Inverse-Probability Weighting for Changing Allocation Probabilities

Why Weight for Changing Allocation Probabilities?

In a response-adaptive trial, the probability of assigning treatment can change as enrollment proceeds. For example, an algorithm may assign treatment more often during one period than another because earlier trial outcomes altered its allocation target. If prognosis also changes over calendar time, the participants assigned to each arm may represent different mixes of enrollment times. A raw difference in observed arm means can then reflect both treatment and those differing mixes.

Inverse-probability weighting uses the probability recorded immediately before each assignment to reweight the observed outcome. A participant assigned to an arm with a low probability receives a larger weight: that observation stands in, in expectation, for similar participants who could have been assigned to that arm but were not. This is a probability-based correction, not a guarantee that the realized arms will have identical time distributions.

The conditioning on \(H_i\) matters: assignments need not be independent across participants. The allocation algorithm may adapt to earlier assignments or outcomes. What matters for the calculation is the actual conditional probability used for participant \(i\), not an overall treatment fraction or an estimate from the realized assignments.

Estimands and the Weighted Estimators

Suppose the trial has a fixed cohort of \(n\) participants. Write \(Y_i(1)\) and \(Y_i(0)\) for participant \(i\)'s potential outcomes under treatment and control, respectively, and let \(Y_i\) be the observed outcome. The finite-cohort treatment-specific means are \(\mu_1=n^{-1}\sum_iY_i(1)\) and \(\mu_0=n^{-1}\sum_iY_i(0)\). Their difference \(\tau=\mu_1-\mu_0\) is the average treatment effect for these enrolled participants. The observed outcome is \(Y_i=A_iY_i(1)+(1-A_i)Y_i(0)\), under consistency: the outcome observed under the assigned treatment is the corresponding potential outcome.

The Horvitz–Thompson (HT) estimates use the cohort size \(n\) as the denominator. Define \(p_{i1}=\pi_i\) and \(p_{i0}=1-\pi_i\). Then:

Here \(I(A_i=a)\) equals 1 when participant \(i\) received arm \(a\), and 0 otherwise. A normalized alternative, often called the Hájek estimate, divides each weighted outcome total by the corresponding total weight:

The HT form has a direct finite-cohort unbiasedness result under the assumptions below. The Hájek form normalizes the weights separately within each arm; it can be more stable in some samples, but its random denominator means it is not generally exactly unbiased in finite samples. Both target a cohort-average contrast when the weights are correctly specified; neither should be interpreted as automatically producing exact time balance in the observed data.

Why the HT Weight Works

Assume treatment assignment is randomized at each enrollment, with the stated probability conditional on the pre-assignment history, and that this random draw is independent of the current participant’s potential outcomes conditional on that history. Then, for either arm \(a\), the assignment indicator has conditional expectation \(p_{ia}\). Consequently:

Averaging this equality over participants gives \(E(\widehat{\mu}_a^{HT})=\mu_a\), and subtracting the arm-specific equalities gives \(E(\widehat{\tau}^{HT})=\tau\). The reasoning uses conditional assignment probabilities participant by participant, so adaptation based on prior information does not itself invalidate the result. It does require the probabilities actually used by the randomization mechanism, not probabilities reconstructed incorrectly from the final allocation totals.

The assumptions have substantive content. The treatment and control outcomes must be well defined and observed for the assigned arm; there must be no interference between participants; and the probability supplied to the weight must be the probability of the assignment that was genuinely made. Most importantly, for every participant in the target cohort, both arm probabilities must be positive: \(0<\pi_i<1\). If outcomes are missing after assignment, these formulas alone do not correct for that missingness.

Worked Calculations

Diagnostics, Alternatives, and Limitations

Before interpreting a weighted contrast, check the probability record and the resulting weights. Confirm that each value is the pre-assignment conditional probability for that participant, including any dependence on earlier trial history. Then examine the largest weights, arm-specific weight distributions, and how the weighted enrollment-time distribution compares across arms and with the full cohort. The effective sample size for weights \(w_i\) in an arm is \((\sum_i w_i)^2/\sum_i w_i^2\); a value much smaller than the number of observed participants in that arm indicates that a few observations carry much of the weighted estimate.

The essential positivity condition is that treatment and control each remain possible for every participant in the target cohort. Probabilities close to zero or one technically satisfy positivity but can still create very large weights and high variance, as the preceding example illustrates. If an arm has probability zero for some participants, its potential outcomes for those participants cannot be recovered by these weights from the observed assignments. Truncating or capping extreme weights may reduce variability, but changes the estimator and can introduce bias; it is not a way to restore information where an arm was impossible.

A raw arm-mean contrast uses no probability weights, so it may be appealing for its simplicity, but changing allocation combined with time-related prognosis can leave the observed arm means comparing different enrollment-time mixes. Time-stratified standardization, temporal blocking, and regression adjustment are alternatives with different requirements and finite-sample behavior. Weighting is especially transparent when the sequential randomization probabilities are known and correctly recorded. It can nevertheless be noisy, and weighting alone does not promise exact balance in a small realized trial. The target here is the average over the enrolled cohort; a different target population requires a corresponding change in the estimand and weighting strategy.

A common mistake is to say, “The trial assigned treatment to 60% of participants, so I will use 0.60 as every participant’s treatment probability.” That substitutes an overall realized fraction for each participant’s conditional randomization probability. The correction is to use the recorded \(\pi_i\) and \(1-\pi_i\) at each assignment. Another mistake is to assume that correct weights force exact balance in calendar time. They balance in the relevant randomization expectation; observed weighted time distributions still require inspection.