Two Different Changes Over Calendar Time
A trial unfolds across calendar time. During enrollment, clinical practice, eligibility mix, outcome assessment, or other conditions may change. At the same time, a response-adaptive randomization procedure may change the probability of assigning a participant to a particular treatment. These are related features of a trial timeline, but they are not the same quantity.
Calendar-time drift in outcomes means that the distribution of an outcome changes with enrollment time, when the treatment condition and outcome measurement are held fixed for the comparison. Calendar-time variation in allocation probabilities means that the randomization probability assigned to a treatment changes over enrollment time. The first concerns outcomes; the second concerns the randomization procedure.
In response-adaptive randomization, probabilities may be updated as the trial proceeds, so the sequence of assignment probabilities can change over enrollment time. The sequence need not be a smooth or monotone trend: it may change in steps, rise and fall, or vary irregularly. The important point is to describe the probabilities used at each enrollment or period, rather than assume that one probability applied throughout.
A probability is not an observed allocation fraction. If \(\pi_i=0.60\), the design assigns participant \(i\) to treatment 1 with probability 0.60, conditional on the information used by the procedure. It does not require exactly 60% of a group of participants to receive that treatment. Conversely, a high observed treatment fraction in a period does not by itself show that the assignment probabilities were high.
Defining and Describing Outcome Drift
A drift statement needs a reference condition. For example, one might compare the distribution of a binary outcome under a specified standard treatment across enrollment months, using the same eligibility definition, follow-up window, and ascertainment method. A change in the observed outcome rate could reflect a change in the patients enrolled, clinical practice, outcome measurement, or other calendar-time conditions. A descriptive drift pattern does not identify which cause produced the change.
For a binary outcome, period-specific proportions offer a direct description. For a quantitative outcome, period-specific means or distributions may be informative. The summary should name the time periods, outcome, units or scale, and treatment condition. A comparison of two periods can be written as a difference in rates or means, but it is descriptive unless additional design and modeling assumptions justify a broader inference.
Worked Example: Describing a Change in a Reference Outcome Rate
Hypothetical data: Suppose participants receiving the same standard treatment are enrolled in two calendar periods. The outcome is recovery by day 30, measured in the same way in both periods. In period 1, 5 of 25 participants recover; in period 2, 12 of 25 recover. Calculate and interpret the observed rate change.
The period-specific rates are \(5/25=0.20\) and \(12/25=0.48\). The difference, later period minus earlier period, is \(0.48-0.20=0.28\), or 28 percentage points.
In this hypothetical comparison, the observed recovery rate under the specified standard treatment is 28 percentage points higher in period 2 than in period 1. This is evidence of a descriptive calendar-time change in the measured outcome rate. It does not establish why the rate changed, and it does not by itself estimate a treatment effect.
The comparison is interpretable as a drift description only if the outcome definition and follow-up window are consistent. If day-30 recovery were recorded differently in the two periods, the rate difference could reflect measurement changes rather than a change in outcomes.
Representing Allocation Probabilities Over Enrollment
To describe the randomization process, record the probability actually used for each participant or for each enrollment period. In a two-treatment trial, \(A_i\) is an indicator: it equals 1 when participant \(i\) is assigned to the treatment of interest and 0 otherwise. The probability \(\pi_i\) is conditional on the information available just before that assignment. Since an adaptive rule can use accumulated trial information, the probability sequence can itself be random across possible trial histories.
For a block of enrollments \(B\), the average assignment probability is \(\bar{\pi}_B=|B|^{-1}\sum_{i\in B}\pi_i\). Comparing these averages across prespecified calendar periods gives a concise description of how allocation probabilities differed over time. If probabilities vary within a period, the period average should not hide that variation; reporting the range or the participant-level sequence may be useful.
The justification follows from conditional expectation. By definition, \(E(A_i\mid\mathcal{F}_{i-1},t_i)=\pi_i\). Taking expectations on both sides gives \(E(A_i)=E(\pi_i)\). Linearity of expectation then gives \(E(\sum_i A_i)=\sum_i E(A_i)\). Thus expected allocation counts can be calculated from the probabilities even when the procedure updates over time.
Worked Example: Expected Allocation Across Two Periods
Hypothetical design: In the first period, 12 participants each have probability 0.35 of assignment to treatment 1. In the second period, 12 participants each have probability 0.65. Calculate the expected number assigned to treatment 1 in each period and overall.
For period 1, the expected count is \(12(0.35)=4.2\). For period 2, it is \(12(0.65)=7.8\). Summing the period expectations gives \(4.2+7.8=12\) expected assignments to treatment 1 among 24 participants. The overall average probability is \((12(0.35)+12(0.65))/24=12/24=0.50\).
The probabilities increased by 0.30 between periods, while the average probability over all 24 enrollments was 0.50. The expected count of 4.2 in the first period is not a possible realized count; it is an expectation. The actual counts must be integers and can differ from their expectations.
Separating a Probability Pattern From Random Variation
A changing sequence of observed allocations is not enough to establish that the assignment probabilities changed. Even under a constant probability, random assignment can produce different treatment fractions in different periods. To describe a probability trend, use the recorded randomization probabilities or reconstruct them from the protocol and the information available at each decision. Then summarize changes in those probabilities over enrollment time.
Likewise, a sequence of probabilities that differs across participants does not automatically imply a smooth time trend. The probabilities may move irregularly, and a period average may conceal local changes. Useful checks include confirming the enrollment-time order, checking that recorded probabilities match the protocol, and comparing period summaries with the underlying sequence.
Worked Example: Comparing Probability Averages Across Four Periods
Hypothetical probability record: Each of four consecutive enrollment blocks contains 10 participants. The recorded probability of treatment 1 is constant within each block and equals 0.30, 0.40, 0.60, and 0.70, respectively. Calculate expected counts and compare the early and late blocks.
The expected counts by block are \(10(0.30)=3\), \(10(0.40)=4\), \(10(0.60)=6\), and \(10(0.70)=7\). The expected total across 40 participants is \(3+4+6+7=20\), corresponding to an overall average probability of \(20/40=0.50\).
The average probability in the first two blocks is \((0.30+0.40)/2=0.35\), because the blocks are the same size. In the last two blocks it is \((0.60+0.70)/2=0.65\). The late-minus-early difference is \(0.65-0.35=0.30\). This probability record shows a clear increase across the four blocks, even though the overall average is 0.50.
If the realized treatment-1 counts were 2, 5, 7, and 6, they would not equal the expected counts block by block. That discrepancy is compatible with random assignment; the probability record, not those realized counts, defines the allocation-probability pattern.
Assumptions, Comparisons, and Failure Modes
These descriptions rely on accurate records of enrollment time, treatment assignment, and the probability used at each randomization. Describing outcome drift additionally requires a consistent outcome definition and a clearly stated reference treatment condition. If eligibility or measurement changes over time, those changes should be reported rather than silently treated as comparable periods.
| Feature | Fixed-probability randomization | Response-adaptive randomization |
|---|---|---|
| Assignment probability | Set to the same value under the stated design across enrollments | May change as the trial proceeds according to the adaptive procedure |
| What to summarize over time | Protocol probability and realized allocation counts | Probability used at each enrollment or period, as well as realized counts |
| What a probability trend describes | Usually no planned change in probability | Change in the randomization probabilities; not, by itself, outcome drift |
Fixed-probability randomization provides a useful contrast because the intended allocation probability is stable. Response-adaptive randomization can instead produce changing probabilities. Neither description substitutes for an outcome summary: the allocation record answers how assignment probabilities changed, while an outcome record under a specified condition addresses whether outcomes drifted.
Several failures can make a drift description misleading. Missing or incorrectly recorded probabilities prevent a reliable account of the randomization process. Changed outcome definitions can create an apparent outcome shift. Broad time bins can mask within-period probability changes, while very small bins can produce noisy outcome rates. Finally, a pattern across periods may be real as a description but still leave its cause unresolved.
- Confusing probability with allocation fraction. A period with 7 of 10 assignments to treatment 1 does not prove that the probability was 0.70. Check the probability assigned by the randomization procedure.
- Calling every rate difference drift. A difference in observed rates can occur by chance. Describe the observed pattern and avoid claiming a systematic change without adequate evidence and a suitable analysis.
- Assuming randomization forces equal time-period counts. Randomization governs probabilities, not exact counts. A response-adaptive rule may use different probabilities at different enrollments.
- Treating drift as a causal explanation. Calendar time labels when outcomes were observed; it does not identify the reason for a change.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A hypothetical binary outcome is observed in 8 of 20 participants in one period and 11 of 20 in a later period under the same reference treatment. What is the observed rate difference, later minus earlier?
- Eight participants each have assignment probability 0.25, and the next eight each have probability 0.75. What is the expected total assigned to treatment 1?
- Why is an observed treatment fraction of 0.70 in a period not sufficient to conclude that the assignment probability was 0.70?
- Name two conditions that should be checked before interpreting a change in outcome rates across calendar periods as outcome drift.
Solutions. (1) The rates are \(8/20=0.40\) and \(11/20=0.55\); the difference is \(0.55-0.40=0.15\), or 15 percentage points. (2) The expected total is \(8(0.25)+8(0.75)=2+6=8\). (3) The realized fraction is a random outcome of assignment; the probability used by the procedure must be obtained from its record or protocol. (4) For example, check that the treatment condition is comparable and that the outcome definition, follow-up window, and ascertainment method are consistent across periods.