Outcome Regression and Weighting for Temporal Balance

Why Combine Outcome Regression and Weighting?

When allocation probabilities change during enrollment, the treatment and control groups can contain different proportions of early and late participants. If prognosis changes over calendar time, a raw difference in arm averages can then mix treatment effects with differences in enrollment timing. Earlier material on time-stratified standardization and inverse-probability weighting provides two ways to address this problem. This tutorial shows how they relate and how to combine them.

The goal is to compare treatment and control outcomes for a common distribution of enrollment times, rather than to compare the time distributions that happened to occur in the two arms. The choice of that common distribution is part of the estimand: a natural target is the distribution of enrollment times among all participants in the analyzed trial cohort.

For the formulas below, write \(p_i=P(A_i=1\mid W_i)\) for the known probability of treatment assignment at participant \(i\)'s randomization. In an adaptive trial, this probability can change over enrollment. The information \(W_i\) must be sufficient for the conditional randomization statement being used; if the assignment probability depends on recorded pre-assignment history beyond calendar time, that history belongs in the conditioning information. In a simple time-only illustration, \(W_i\) is just the enrollment period.

The key assumptions are consistency (the observed outcome is \(Y_i(A_i)\)), no interference between participants, and sequential randomization: conditional on the recorded pre-assignment information, current assignment is independent of the participant's potential outcomes. Also, \(0<p_i<1\) for the target participants, so each has a nonzero chance of either assignment. These conditions support identification; they do not ensure that a chosen outcome model is adequate or that a finite-sample estimate is precise.

Three Ways to Target the Same Temporal Distribution

Outcome-regression standardization estimates the conditional mean outcome under each assignment, then averages predicted contrasts over the target covariate or time distribution. If \(\hat m_a(W_i)\) is a fitted outcome model, the empirical estimator is

This approach can be efficient when the outcome model captures relevant prognostic patterns, including temporal drift. A wrong functional form, omitted time pattern, or inadequately modeled treatment-by-time variation can distort the standardized contrast.

Inverse-probability weighting (IPW) uses each observed outcome with a weight inversely proportional to the probability of receiving its observed assignment. A simple Horvitz–Thompson estimator of the same target is

The factor \(1/p_i\) gives more representation to treated participants whose observed assignment was less likely; \(1/(1-p_i)\) does the analogous job for controls. Under the stated randomization assumptions, averaging over assignments recovers the target potential-outcome means. IPW does not require a correct outcome regression, but highly variable weights can make its estimate unstable.

A third approach adds weighted outcome residuals to the regression-standardized estimate. This is the augmented inverse-probability-weighted (AIPW) estimator:

The first part standardizes fitted outcomes. The residual terms correct the fitted means using observed outcomes, with assignment-probability weights. In a randomized adaptive trial, the allocation probabilities are ordinarily known by design; using the recorded probabilities is important, rather than reconstructing them from the realized arm counts.

Why the Augmentation Works

The double-robustness claim can be seen conditionally at a fixed \(W\). Let \(p=P(A=1\mid W)\), and write \(m_a(W)\) for the true conditional mean. Consider the AIPW expression using any fitted functions \(\tilde m_a(W)\) and a probability \(q(W)\). If \(q(W)=p\), then the conditional expectation of the treated residual contribution is \(m_1(W)-\tilde m_1(W)\); that of the subtracted control residual is \(-\{m_0(W)-\tilde m_0(W)\}\). Adding the regression contrast gives the true contrast:

Alternatively, if both outcome regressions are correct, the residuals have conditional mean zero within each assignment arm. Their weighted expectations are therefore zero when the denominators are valid, even if a fitted assignment probability \(q(W)\) is not the true probability. The fitted contrast alone then has the correct conditional expectation. Thus the estimator is consistent if either the assignment-probability model or the outcome regression is correct, subject to the identification assumptions and appropriate estimation conditions.

In response-adaptive randomization, this result should be interpreted carefully: the assignment probability is usually known, not an uncertain nuisance model. With correctly recorded probabilities, the AIPW estimator is protected against outcome-model misspecification in the expectation argument above. “Doubly robust” does not mean that the estimator is immune to incorrect recorded probabilities, lack of overlap, violations of randomization assumptions, or poor finite-sample behavior.

Diagnostics, Alternatives, and Limits

Diagnostics should check both the probability mechanism and the outcome model. Compare recorded probabilities with the trial's allocation rules and enrollment history; inspect treatment counts and outcome summaries by time period; and examine whether outcome residuals still show systematic temporal patterns. For IPW and AIPW, inspect the distribution of weights and identify participants whose weights dominate the estimate. A useful descriptive summary is the effective sample size within an arm, \((\sum_i w_i)^2/\sum_i w_i^2\), where \(w_i\) are that arm's positive weights. A much smaller effective sample size than the arm count signals that a few observations carry much of the weighted information.

Outcome regression and IPW have different vulnerabilities. Regression can extrapolate into periods or covariate profiles with little information in one arm, and its answer may depend on whether time was modeled flexibly enough. IPW relies less on outcome-model form but can be imprecise when probabilities are close to zero or one. AIPW combines fitted contrasts with residual correction; it often offers protection against one nuisance-model error, but it does not remove the need for overlap or reliable data. A direct time-stratified contrast is a transparent alternative when the periods are few and adequately represented. A raw arm comparison does not target the same temporal distribution when enrollment patterns differ.

These estimates also depend on the target distribution. Averaging over the enrolled cohort answers a different question from standardizing to a planned calendar-time distribution or to a specified external population. State the target explicitly. With a continuous time trend, the outcome model must represent that trend adequately; dividing time into too few categories can leave residual drift, while too many sparse periods can make cell estimates unstable. Finally, a point estimate alone does not establish the precision of the result: uncertainty calculations must respect the adaptive randomization design and the estimator used.

Common Mistake