Why Calendar Time Matters
In a clinical trial, the patients enrolled early may differ prognostically from those enrolled later. Standard care can change, eligibility criteria can evolve, or referral patterns and disease severity can shift. If the prognosis of enrolled patients changes while response-adaptive randomization changes the probability of assignment to each arm, the arms can end up containing different mixtures of early and late participants. Their crude outcome averages may then differ partly because of calendar time, not because of treatment.
Response-adaptive randomization uses accumulating trial information to update assignment probabilities. This can make later allocation probabilities differ from earlier ones. The assignment at each stage remains randomized according to the specified rule, but the resulting allocation path can align one arm more heavily with one part of a changing enrollment period. The relevant diagnostic is not simply whether the final arm counts are unequal; it is whether prognosis and allocation vary together over time.
The issue is clearest when calendar period is prognostic. Let \(A\) and \(B\) denote two treatment arms, and let \(t\) index calendar periods. Write \(n_{At}\) and \(n_{Bt}\) for the numbers assigned to each arm in period \(t\), and \(\bar{Y}_{At}\) and \(\bar{Y}_{Bt}\) for their observed mean outcomes in that period. The arm-wide means are weighted averages of these period-specific means, with weights determined by each arm’s own period composition.
An Exact Decomposition of the Crude Difference
When both arms have observations in every period being compared, define \(w_{At}=n_{At}/n_A\) and \(w_{Bt}=n_{Bt}/n_B\), where \(n_A=\sum_t n_{At}\) and \(n_B=\sum_t n_{Bt}\). Then the observed crude difference has an exact decomposition:
The first term averages within-period differences using arm \(B\)’s period weights. The second is a composition term, expressed relative to arm \(A\)’s period means. This identity is an arithmetic decomposition of the observed data, not a causal result. Other reference choices give different-looking but equivalent decompositions. The identity also shows why unequal allocation alone is not enough to diagnose distortion: if prognosis does not vary across periods, the composition term is zero even when the period weights differ.
A common alternative descriptive summary uses the same set of period weights \(q_t\) for both arms:
This puts both arms on a common calendar-time mix. It can reveal whether a crude difference is driven by differing period composition, but it does not by itself solve every problem in an adaptive trial. It requires observations from both arms in the periods receiving positive weight, and it does not remove within-period chance imbalance, outcome-driven selection, or unmeasured changes in patient mix.
Worked Example: A Crude Difference Created by Period Composition
Suppose a hypothetical trial enrolls participants in an early and a late period. In both arms, the observed mean outcome is 50 points early and 70 points late. The outcome units are points, and higher values indicate a better outcome. Arm \(A\) has 8 early and 2 late participants; arm \(B\) has 2 early and 8 late participants. Assume the period-specific means summarize the observed outcomes exactly as stated.
The crude mean for arm \(A\) is
The crude mean for arm \(B\) is
Thus the observed crude difference, \(B-A\), is \(66-54=12\) points. Yet the within-period differences are both \(0\) points. The decomposition attributes the entire crude difference to composition:
With common weights of one-half for each period, both standardized means are \((50+70)/2=60\) points, so their standardized difference is 0. This example demonstrates a possible distortion; it does not establish why the arms acquired these period proportions or imply that every adaptive allocation path will produce such a difference.
How Adaptation Can Amplify the Problem
Suppose an early cohort produces an encouraging, but noisy, response in arm \(B\). A response-adaptive rule may then increase the probability of assigning later participants to \(B\). If prognosis also improves over calendar time, \(B\) may receive a larger share of participants from the later, better-prognosis period. A crude arm comparison then reflects both the evolving allocation and the changing enrollment population.
The allocation probabilities are part of the design; the realized arm counts and outcomes are random. In particular, replacing random counts in a pooled-mean calculation with their expectations does not generally yield the exact expected pooled mean. The denominator of a pooled mean is random, and in general the expectation of a ratio is not the ratio of expectations. Calculations using expected counts can illustrate expected composition, but should be labelled as such—not as exact expected arm means.
Worked Example: A Realized Adaptive Path With Improving Prognosis
Consider a hypothetical trial with three enrollment periods, each containing 10 participants. The design starts with equal allocation. After accumulating earlier outcome information, the rule later favors arm \(B\); in the third period, 8 of 10 participants are assigned to \(B\). The table gives a possible realized allocation path and observed period-specific means. It is not a table of expected counts.
| Period | Arm \(A\): count | Arm \(A\): mean (points) | Arm \(B\): count | Arm \(B\): mean (points) |
|---|---|---|---|---|
| 1 | 5 | 40 | 5 | 44 |
| 2 | 5 | 50 | 5 | 50 |
| 3 | 2 | 60 | 8 | 60 |
The crude arm means are
The crude difference is \(5.28\) points in favor of \(B\). The observed within-period differences are \(4\), \(0\), and \(0\) points. With equal weights for the three periods, their common-weight average is \((4+0+0)/3=1.33\) points. The difference between this summary and the crude contrast illustrates the influence of unequal period composition. The period 1 difference is an observed difference, not proof of a treatment effect; it could reflect chance or other features of that period.
For an exact check, use the decomposition with arm \(A\) as the reference. The within-period component weighted by \(B\)’s proportions is \((5/18)(4)=1.11\) points. The composition component is
Together, \(1.11+4.17=5.28\) points, matching the crude difference up to rounding. This partition describes the realized data. It does not claim that the composition term is the exact expected contribution across repeated adaptive trials.
Why Expected Counts Do Not Give an Exact Expected Mean
A small calculation makes the random-denominator issue concrete. Imagine two hypothetical participants with fixed outcomes of 0 points in an early period and 10 points in a late period. Their probabilities of assignment to \(B\) are 0.2 and 0.8, respectively, as could occur under a rule whose allocation probabilities vary by time. For illustration, suppose assignments are independent conditional on those probabilities. The expected numbers assigned to \(B\) are 0.2 and 0.8.
Worked Example: Checking a Plug-In Pooled Mean
Plugging those expected counts into a weighted-mean formula gives
This is the mean calculated from the expected composition, not the exact expected observed mean in \(B\). The probability that neither participant is assigned to \(B\) is \(0.8(0.2)=0.16\), so the observed \(B\) mean is defined with probability \(0.84\). Conditional on at least one assignment to \(B\), the probabilities of the three possible nonempty patterns are: early participant only, \(0.04/0.84\); late participant only, \(0.64/0.84\); and both participants, \(0.16/0.84\). Their corresponding means are 0, 10, and 5 points. Therefore,
The exact conditional expectation, 8.571 points, differs from the plug-in value of 8 points. The assumptions here are deliberately simple, and the example is not an analysis procedure for trial outcomes. It demonstrates why expected counts should not be used to assert an exact expectation for a pooled sample mean.
Assumptions, Diagnostics, and Alternatives
The composition explanation is most relevant when calendar period predicts outcome and treatment allocation varies across periods. To interpret a within-period comparison as a randomized contrast, assignment must still be randomized within the period under the trial’s specified rule, with adequate probability of receiving each arm. The outcome for a participant must not determine that participant’s own assignment unless the design explicitly handles that feature. For a simple period adjustment, also assume the period labels capture the temporal changes being considered and that outcomes are measured comparably across periods. These conditions support a diagnostic comparison; they do not guarantee that a crude or adjusted contrast is unbiased for every target.
Useful diagnostics include plotting enrollment and allocation probabilities over calendar time, tabulating arm counts by period, and examining baseline prognostic variables and relevant changes in care by period. Plot outcomes by arm and period rather than relying only on a single pooled summary. Sparse or empty arm-period cells are a warning: there may be little or no direct within-period information for comparing arms. Check whether period boundaries reflect genuine operational or clinical changes, and whether a broad period category hides important changes within that category.
Several summaries can be compared, with different purposes and limitations:
| Approach | What it helps show | Main caution |
|---|---|---|
| Crude arm means | The observed overall outcome difference in the enrolled arms | Mixes within-period differences with potentially different period composition |
| Period-specific contrasts | How the observed arm difference varies over calendar time | Can be unstable when period-specific counts are small |
| Common-weight period summary | A contrast for a shared, stated period mix | Requires overlap and depends on the chosen weights |
| Regression including calendar time | Can describe outcomes conditional on measured time and other covariates | Results depend on model form; adjustment cannot recover information absent from empty cells |
A common mistake is to say, “Arm \(B\) has the higher mean, so the adaptive rule must have found a better treatment.” That reasoning confuses an observed pooled difference with a within-period treatment comparison. The correction is to inspect arm outcomes and prognostic composition over time, then state what the data support. Conversely, finding a temporal imbalance does not prove that the entire observed difference is due to drift; within-period differences and other sources of variation still matter.
Time adjustment has limits. Broad period categories can leave residual temporal confounding if prognosis changes within periods. Adjusting for time also does not automatically address outcome-driven selection in the adaptive rule, delayed outcome information, changes in eligibility, or unmeasured changes in care. If an arm is rarely or never assigned during a period, no modelling choice can create direct randomized comparisons there. These concerns motivate careful design and analysis, but the exact treatment-effect estimation strategy after adaptive assignment is a separate question.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- If period-specific outcomes are identical between arms but prognosis improves over time, can the crude arm means still differ? Why?
- In the decomposition, what does the composition component represent?
- Why is a mean computed using expected arm counts not generally the exact expected pooled sample mean?
- What does an empty arm-period cell imply for a within-period comparison?
Solutions. (1) Yes. If the arms have different proportions from early and late periods, their crude means can differ even though their observed means match within each period. (2) It is the part of the crude difference associated with different period weights, given the chosen reference arm’s period means; it is an exact arithmetic component, not by itself a causal effect. (3) The pooled mean has a random denominator, so the expectation of the ratio need not equal a ratio formed from expected counts. (4) There is no observed direct comparison between the arms in that period; a summary requiring that comparison needs different weights or additional assumptions, and modelling does not supply the missing randomized observations.