Why Plot Allocation Probabilities Over Enrollment Time?
In a response-adaptive trial, the chance of assignment to a treatment can change as information accumulates. A timeline of those chances makes the randomization rule's behavior visible: it can reveal a gradual movement toward one arm, a sudden shift, or a period in which an arm has very little chance of being assigned. These are useful signals when assessing whether the trial's allocation process is changing over enrollment.
An allocation-probability timeline is a diagnostic of the randomization mechanism, not a plot of patient outcomes. A shift in probabilities may arise because the adaptive rule responds to accumulated outcomes, because the rule has an explicit time-dependent component, or because of a protocol or implementation change. The plot alone does not identify which explanation applies, and it does not establish that outcomes have drifted.
Definition and Construction
The probability must be the one in force before the assignment is made. It is not the proportion of participants who have so far received that treatment. In a two-arm trial, if arm 1 has probability \(p_j\), arm 2 has probability \(1-p_j\). With more arms, the probabilities at each enrollment satisfy \(\sum_{k=1}^{K}\pi_{jk}=1\).
Enrollment order is often the clearest horizontal axis because it records the sequence in which the randomization rule was applied. Calendar dates can also be useful when enrollment pace varies or when a known protocol change occurred on a date. If the plot uses calendar time, preserve the order of randomization and make gaps or periods with no enrollment visible. For either axis, the vertical scale for probabilities should run from 0 to 1 so that the magnitude of changes is not visually exaggerated.
Reading the Timeline
Inspect the unsmoothed probabilities first. A steadily rising or falling series suggests a sustained change in the rule's allocation preference along the observed enrollment sequence. A large jump between adjacent enrollments suggests an abrupt change that merits checking against the allocation algorithm, the information available at that time, and the trial record. A run of probabilities near 0 for an arm signals a sparse probability region: participants enrolled in that region had little chance of receiving that arm.
A rolling average can help make a sustained pattern easier to see, but it should supplement rather than replace the individual probabilities. For a window of \(w\) consecutive enrollments, a simple descriptive average for arm \(k\), ending at enrollment \(j\), is
This average summarizes probabilities recorded in that window. It is not a new allocation probability, nor does smoothing show that the underlying process changed smoothly. Choose and report the window width; overlapping windows are descriptive and do not constitute independent observations.
Worked Example: A Gradual Shift Across Eight Enrollments
Suppose a hypothetical two-arm trial logs the following probabilities for arm 1, in enrollment order: 0.50, 0.50, 0.55, 0.60, 0.65, 0.70, 0.72, and 0.75. Arm 2's probabilities are the complements: 0.50, 0.50, 0.45, 0.40, 0.35, 0.30, 0.28, and 0.25. At each enrollment, the two probabilities sum to 1.
The change in arm 1's probability from enrollment 1 to enrollment 8 is \(0.75-0.50=0.25\), or 25 percentage points. The intermediate values mostly rise rather than changing only at the end, so the timeline describes a sustained shift toward arm 1 in the recorded allocation rule. It does not, by itself, tell us why that shift occurred.
For a three-enrollment rolling average ending at enrollment 3, the calculation is
For the window ending at enrollment 6, it is
And for the window ending at enrollment 8, it is \((0.70+0.72+0.75)/3=0.7233\). These summaries make the direction of the pattern easier to compare, while the original points show the enrollment-level changes.
Worked Example: Locating an Abrupt Change
In another hypothetical trial, the logged arm 1 probabilities for eight consecutive participants are 0.50, 0.50, 0.50, 0.50, 0.80, 0.80, 0.80, and 0.80. The arm 2 probabilities are the complements, so they change from 0.50 to 0.20 at enrollment 5.
The adjacent difference across the transition is \(0.80-0.50=0.30\), a 30-percentage-point jump. Before enrollment 5, the four-point average is \((4\times0.50)/4=0.50\); from enrollment 5 through 8, it is \((4\times0.80)/4=0.80\). The timeline therefore shows an abrupt change followed by a sustained period at the new level.
This pattern is a prompt to inspect what information and rule were in effect at enrollment 5. It is not proof of an error: a planned update could produce the same shape. Nor can the plot alone establish whether the change reflects outcome feedback, a scheduled rule, or an implementation event.
Worked Example: Recognizing a Sparse Probability Region
Suppose a hypothetical three-arm trial records the following probabilities in a late enrollment period: at enrollment 21, \((0.90,0.08,0.02)\); at 22, \((0.92,0.06,0.02)\); at 23, \((0.91,0.07,0.02)\); and at 24, \((0.93,0.05,0.02)\). Each vector sums to 1. The timeline for arm 3 is a flat run at 0.02, while arm 1's probability is high and arm 2's is also relatively low.
The arm 3 series shows that each participant in this period had a 2% chance of assignment to arm 3 under the logged rule. This is a sparse probability region for that arm, even though the series is not changing within these four enrollments. The appropriate interpretation is limited to the randomization opportunities represented in the plot; it does not say how many participants were actually assigned to arm 3, or whether the probability is acceptable under the protocol.
A Short Justification: Why the Probabilities Matter
The allocation probability has a direct interpretation for the next assignment. For a two-arm trial, define \(I_j=1\) if participant \(j\) is assigned to arm 1 and \(I_j=0\) otherwise. Conditional on the pre-assignment information, \(I_j\) is a Bernoulli random variable with success probability \(p_j\). Therefore,
Thus, the plotted value \(p_j\) is the chance of arm 1 assignment for that participant, given the information available to the randomization rule. It is not a guarantee about the assignment, and a sequence of probabilities is not the same thing as a sequence of observed assignments. This distinction is why the timeline can diagnose changing allocation pressure even when the realized assignment sequence happens to look different.
Practical Checks and Failure Modes
Use the probability vector calculated immediately before each randomization, and verify its arm labels and enrollment identifier.
For each enrollment, confirm that every probability is between 0 and 1 and that the arm probabilities sum to 1, allowing only for documented rounding.
For a multi-arm trial, show each treatment-specific series. If a rolling average is added, label its window and keep the unsmoothed values visible.
Compare the timing of a jump or sustained movement with the protocol, recorded algorithm inputs, and implementation logs before attributing a cause.
A common mistake is to plot the cumulative fraction assigned to an arm and call it that arm's allocation probability. The fraction is based on assignments already made; the allocation probability is the conditional chance used before each assignment. The two can differ substantially, especially over a short period. Another mistake is to smooth first and then conclude there were no abrupt changes: a wide window can conceal a real jump, so inspect the raw timeline.
Probability timelines also have limits. They show the probabilities the design recorded, not whether the design's model or rationale was appropriate. A probability near zero may warrant attention because it gives little randomization opportunity to that arm at that time, but there is no universal cutoff that defines an unacceptable region. Interpretation depends on the protocol and scientific question. A plot cannot separate prognostic drift from treatment-effect drift, establish a cause for an allocation change, or determine the consequences for an outcome analysis.
How This Diagnostic Differs From Other Displays
| Display | What it summarizes | What it does not establish by itself |
|---|---|---|
| Allocation-probability timeline | The conditional probabilities used by the randomization rule over enrollment | Observed treatment counts, outcome drift, or the cause of a probability change |
| Assignment sequence or treatment frequency | Which arms participants actually received | The probability assigned to each participant before randomization |
| Outcome plot over time | Observed outcomes indexed by time | Whether a change in outcomes was caused by the allocation rule or by temporal drift |
The probability timeline is especially useful as an audit and design-behavior display. It complements, rather than substitutes for, summaries of assignments or analyses of outcomes. Its main advantage is that it makes the randomization opportunities explicit at the enrollment level; its main limitation is that it does not explain their causes or resolve outcome comparisons.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- In a two-arm trial, arm 1 probabilities at three consecutive enrollments are 0.40, 0.45, and 0.50. What are the arm 2 probabilities, and what pattern does the arm 1 timeline show?
- A plot shows that arm 1 received 60% of participants enrolled in a period. Does this establish that its allocation probability was 0.60 for each participant? Explain.
- For probabilities 0.30, 0.45, and 0.60, calculate the three-enrollment average and state what it summarizes.
- A treatment-specific probability jumps from 0.70 to 0.20 between adjacent enrollments. What can the plot justify, and what should be checked before explaining the jump?
- Why should the raw points remain visible when a rolling average is displayed?
Solutions. (1) The arm 2 probabilities are 0.60, 0.55, and 0.50, respectively. The arm 1 series rises by 0.05 at each step, showing a gradual shift in recorded allocation preference. (2) No. The 60% is an observed frequency; individual conditional probabilities may have varied, and random assignments need not match their probabilities exactly. (3) \((0.30+0.45+0.60)/3=0.45\). It summarizes the average recorded probability over those three enrollments, not an assignment probability for a new participant. (4) The plot supports describing a 50-percentage-point abrupt decrease in the logged probability. Check the pre-assignment probability records, arm labels, algorithm inputs, protocol, and implementation logs before attributing a cause. (5) Smoothing can conceal abrupt changes and local extremes, so raw points are needed to inspect enrollment-level behavior.