When Recorded Enrollment Time Is Not Enough

Why Recorded Time May Not Capture What Changed

Calendar time is easy to record, and in an adaptive trial it can be related to both treatment allocation and outcomes. It is therefore useful to account for enrollment time when estimating treatment effects. But a date or time variable is usually a proxy for the circumstances that changed during enrollment, not a complete description of those circumstances. The same month can contain patients with different eligibility profiles, sites with different care, or outcome measurements made under different procedures.

The distinction matters for interpretation. A model that includes recorded time may control differences associated with that recorded variable, subject to its specification and the available data. It does not automatically control unrecorded factors that vary within the recorded time periods or that are related to treatment after conditioning on time. Nor does time adjustment repair an outcome measurement process that changed in an unrecorded, treatment-related way.

Definition: Recorded enrollment time is not enough when, after conditioning on the time variable used in the analysis, an unmeasured factor still predicts both treatment received (or the treatment contrast being estimated) and the outcome, or when the outcome is measured differently across groups in a way not represented by the model. Time adjustment alone then does not generally identify the intended treatment effect.

This is not a claim that response-adaptive randomization necessarily creates confounding. When treatment is assigned by a correctly implemented randomization scheme, the assignment is randomized according to its specified probabilities, conditional on the design history. A finite sample can nevertheless have chance imbalances. Concerns about confounding are especially direct for as-treated comparisons, deviations from assigned treatment, nonrandom entry or retention, and analyses that fail to reflect the allocation mechanism. It is important to say which comparison is being interpreted.

When Time Adjustment Would Be Sufficient

For an observational or as-treated comparison, a causal interpretation requires, among other conditions, that treatment groups be exchangeable after conditioning on the variables in the analysis, that treatment options have positive probability for the relevant covariate patterns, and that treatment and outcome be defined consistently. In notation, if \(C\) is recorded time and \(U\) is an unmeasured prognostic factor, adjustment for \(C\) alone requires that \(U\) not leave a treatment-outcome association after conditioning on \(C\), or that \(U\) not affect the outcome. A flexible time curve cannot substitute for that condition.

For example, if all relevant changes are completely captured by exact recorded time, the time-outcome and time-treatment relationships are modeled adequately, and treatment options overlap at each relevant time, then time adjustment can address time-related differences. In practice, those conditions can fail if time is grouped coarsely, if patients’ eligibility changes within a period, if site or care changes are not recorded, or if the model forces a trend that does not describe the data. Even a correctly specified time trend cannot adjust for a missing variable whose within-time distribution differs by treatment.

A randomized trial has a different protection: the randomization mechanism, rather than regression adjustment alone, supports the assigned-treatment comparison in expectation. Time adjustment can improve precision or address realized temporal imbalance, but it does not replace valid randomization-based inference. If the analysis instead compares treatment received, or if allocation is not implemented as specified, the required exchangeability must be considered for that comparison. The earlier material on uncertainty estimates after calendar-time adjustment concerns how to quantify uncertainty; uncertainty estimates do not themselves remove systematic bias.

A Short Derivation: The Residual Bias Term

A simple linear model shows why adding time may leave bias. Suppose the outcome obeys \(Y=\beta_0+\beta_A A+\gamma U+\varepsilon\), where \(A\) indicates treatment, \(U\) is an unmeasured prognostic factor, and \(\varepsilon\) has conditional mean zero. Suppose the analysis includes recorded time \(C\) but omits \(U\). Let \(A^*\) and \(U^*\) denote the residuals from regressing \(A\) and \(U\), respectively, on the included time terms. By the partial-regression identity, the coefficient of \(A\) in the model adjusted for \(C\) is the regression slope of the outcome residual on \(A^*\). Substitution gives:

\hat{\beta}_{A,\text{adjusted}} \ \approx\ \beta_A+ \gamma\frac{\operatorname{Cov}(A^*,U^*)}{\operatorname{Var}(A^*)}.

The approximation describes the population regression coefficient (or its large-sample target), not a guarantee about every finite-sample estimate. The second term is zero if the residualized treatment and \(U\) are unassociated, or if \(U\) has no outcome association (\(\gamma=0\)). Otherwise, adjusting for \(C\) alone leaves a component of the \(U\)-related association in the treatment coefficient. This derivation assumes the stated linear conditional-mean model and positive residual variation in treatment; it does not establish that a linear model is suitable for every outcome or design.

Worked Example: Eligibility Changes Within Recorded Periods

Consider a hypothetical as-treated comparison with two recorded enrollment periods. In each period there are 50 patients receiving treatment A and 50 receiving treatment B. Eligibility severity is recorded poorly and is unavailable for adjustment. Severe patients have a 0.60 event risk and nonsevere patients a 0.10 risk under either treatment; thus, by construction, treatment has no effect within severity groups. The severe-patient counts are 30 for A and 10 for B in the early period, and 40 for A and 20 for B in the late period.

In the early period, the expected event risks are:

\begin{aligned} \text{Risk}_A&=\frac{30(0.60)+20(0.10)}{50} =\frac{20}{50}=0.40,\\ \text{Risk}_B&=\frac{10(0.60)+40(0.10)}{50} =\frac{10}{50}=0.20. \end{aligned}

In the late period:

\begin{aligned} \text{Risk}_A&=\frac{40(0.60)+10(0.10)}{50} =\frac{25}{50}=0.50,\\ \text{Risk}_B&=\frac{20(0.60)+30(0.10)}{50} =\frac{15}{50}=0.30. \end{aligned}

The treatment A minus B risk difference is 0.20 in each period. Standardizing both groups to equal weights for the two periods gives risks of \((0.40+0.50)/2=0.45\) for A and \((0.20+0.30)/2=0.25\) for B, a difference of 0.20, or 20 percentage points. Yet the stipulated treatment effect is zero within severity groups. Time standardization has balanced the recorded periods, but it has not balanced severity within them. This example illustrates the bias risk in an as-treated comparison; it does not show that a correctly implemented randomized assignment has a systematic treatment effect in expectation.

Three Ways the Proxy Can Fail

Worked Example: Care Changes Within a Month

Suppose, hypothetically, that two sites enroll patients during the same recorded month. One site provides enhanced supportive care, under which the event risk is 0.10 for either treatment; the other provides routine care, under which it is 0.30 for either treatment. In an as-treated data set, 60 of 100 patients receiving A and 20 of 100 receiving B are at the enhanced-care site. The care type is not recorded.

The observed risks are \(0.60(0.10)+0.40(0.30)=0.18\) for A and \(0.20(0.10)+0.80(0.30)=0.26\) for B. The apparent A-minus-B difference is \(0.18-0.26=-0.08\), or 8 percentage points lower risk under A. In this constructed example treatment has no effect at either site: the difference arises from care composition. Including month cannot remove a site-care difference that occurs within the month. Recording and adjusting for site or relevant care variables may help if the model and overlap are adequate. In an actual randomized trial, such an imbalance may instead be chance variation; its interpretation should follow the randomization design.

Worked Example: Outcome Measurement Changes Within a Period

Suppose the true event risk is 0.20 under either treatment in a hypothetical trial. In the same recorded period, events among A patients are assessed with a procedure having sensitivity 0.95, while events among B patients are assessed with a procedure having sensitivity 0.70. Assume specificity is 1.00 for both procedures, so no nonevents are incorrectly recorded as events. The observed risks are \(0.20(0.95)=0.190\) for A and \(0.20(0.70)=0.140\) for B. The observed risk difference is \(0.190-0.140=0.050\), or 5 percentage points.

A time term cannot correct this differential measurement because both groups are in the same period and the model does not represent the assessment procedure. This example assumes the stated sensitivities and perfect specificity; real measurement error may have a more complicated pattern. Standardized, blinded outcome assessment and recording the measurement procedure address the underlying problem more directly than adding a time curve.

The examples represent distinct mechanisms. Eligibility changes alter who enters or remains eligible; care changes alter prognosis or the treatment context; measurement changes alter the recorded outcome rather than necessarily altering the underlying health outcome. Calendar time may track any of these on average, but the relevant factor may vary within a recorded period, differ by site, or be related to treatment in a way time alone does not encode.

Diagnostics, Alternatives, and Limits

Warning signs include changing eligibility criteria or screening patterns, shifts in baseline severity within time periods, changing site participation or care procedures, differential follow-up, and changes in outcome definitions or assessment tools. Compare these patterns by treatment and over time using variables that were actually recorded. Ask whether the time variable is a date, a broad block, or a fitted trend, and whether important changes could occur inside its resolution. A smooth-looking adjusted trend is not evidence that unmeasured changes have been controlled.

The main alternatives are not simply “more time adjustment” versus “no adjustment.” Depending on the problem, useful steps include preserving concurrent randomized controls, recording and adjusting for eligibility and site characteristics, standardizing eligibility and outcome measurement, and analyzing assigned treatment according to the randomization design. For observational or as-treated comparisons, adjustment requires measured variables that adequately capture the relevant common causes, a defensible model, and treatment overlap. If those conditions are not credible, report the limitation rather than presenting time adjustment as a cure.

Each remedy has limits. Adjustment cannot recover information about a factor that was never measured without additional assumptions. Restricting to a stable eligibility or measurement regime may narrow the population to which results apply. A randomized assigned-treatment analysis protects the comparison specified by the randomization, but does not automatically answer a different question about treatment received, nor does it guarantee that outcomes were measured comparably. Finally, temporal adjustment can increase uncertainty when treatment allocation and enrollment time are strongly related; a precise-looking estimate is not necessarily an unbiased one.

Common Mistake

Common mistake: “The regression includes enrollment time, so all temporal confounding is controlled.” This reasoning treats a recorded proxy as if it measured every changing factor. The correction is to name the underlying eligibility, care, or measurement process; assess whether it is represented by recorded variables; and distinguish the randomized assigned-treatment comparison from an as-treated or otherwise nonrandomized comparison. If a relevant within-time factor remains unmeasured, time adjustment alone does not establish exchangeability.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In what condition would adjustment for recorded time alone remove confounding by an unmeasured factor \(U\)?
Why can a flexible time trend fail to control a care change that occurs between sites in the same month?
A randomized trial has an observed severity imbalance between assigned groups. Does that by itself prove that randomization failed?
In the measurement example, what observed A-minus-B risk difference results from sensitivities 0.95 and 0.70 when true risks are both 0.20 and specificity is 1.00?

Solutions. (1) Time adjustment alone is sufficient for this source of confounding if, conditional on recorded time, \(U\) is not associated with treatment, or \(U\) does not affect the outcome, alongside the other identification conditions. (2) The month indicator or trend does not distinguish sites within that month; if site care predicts outcomes and is associated with treatment, the within-month association remains. (3) No. A finite randomized sample can have chance imbalance; assess the allocation mechanism and use inference appropriate to the design rather than treating imbalance alone as proof of failure. (4) The observed risks are \(0.20(0.95)=0.190\) and \(0.20(0.70)=0.140\), so the difference is \(0.190-0.140=0.050\), or 5 percentage points.