Why Decision-Time Records Matter
In delayed-feedback randomization, an assignment is made using the information available at a particular moment, not all the information that will eventually be collected. An outcome can already have occurred but still be unavailable to the allocation algorithm. If a later audit reconstructs the decision using the final, completed data set, it may inadvertently give the algorithm information it did not have at the time. A reliable record therefore preserves both what was known at the decision and how that information led to the assignment.
This tutorial concerns the decision-time record for each assignment. It does not specify which allocation rule a study should use. Instead, it explains how to retain enough information to check what the rule used and to reproduce the realized assignment when the randomization mechanism permits exact replay.
Define the Decision-Time State
Let \(t_i\) denote the time at which participant \(i\)'s allocation decision is made. For each previously enrolled participant, relevant information may include treatment assignment, outcome value, outcome resolution status, and whether that information was available to the allocation procedure by \(t_i\). These are not interchangeable facts. In particular, an outcome's resolution time is when its value became determined under the study's outcome definition; its availability time is when the value was recorded and accessible to the allocation process.
A record should identify at least: the decision and participant; the decision timestamp and the relevant time zone or clock convention; each prior participant's treatment and decision-time outcome status; all outcomes actually available to the algorithm, with values and availability times; the allocation probabilities and the ordered treatment-arm labels to which they apply; and the randomization draw used to select the assignment. It should also preserve the allocation algorithm's version and any settings or inputs that affect its calculation, such as covariates, strata, eligibility rules, or tie-breaking conventions.
Status needs to be explicit. A prior participant whose outcome has not yet become available is pending for the allocation decision, even if the outcome is subsequently observed. A known dropout should be recorded as such rather than left indistinguishable from a pending outcome. Depending on the outcome definition and data system, other states may include observed, administratively censored, or status not yet ascertained. Record the relevant status as of \(t_i\), and retain the time at which a status or outcome later changed. This makes it possible to distinguish “not available then” from “never observed.”
Probabilities, Draws, and Reproduction
Suppose there are \(K\) treatment arms in a fixed, recorded order, with allocation probabilities \(p_{i1},\ldots,p_{iK}\) at decision \(i\). These probabilities must satisfy \(p_{ik}\geq 0\) and \(\sum_{k=1}^K p_{ik}=1\). Under inverse-transform randomization, a draw \(U_i\) from the uniform distribution on \([0,1)\) selects the first arm whose cumulative probability exceeds \(U_i\). For arm \(k\), define \(c_{ik}=\sum_{j=1}^k p_{ij}\), with \(c_{i0}=0\); the selected arm is \(k\) when \(c_{i,k-1}\leq U_i<c_{ik}\).
The boundary convention matters: if \(U_i\) equals a cumulative probability exactly, the half-open intervals above assign it to the next arm. A different implementation can use a different convention, so record the actual rule. If the randomization uses a different mechanism, preserve its corresponding random variate(s) and selection rule rather than assuming inverse-transform sampling.
The probabilities and realized treatment are enough to audit whether the assignment was possible under the stated randomization rule, but generally not enough to reproduce why that particular arm was selected. Exact replay requires the draw \(U_i\), the arm ordering, and the mapping rule. If a pseudorandom number generator is used, a stored draw can support replay of that decision; replaying the full computational stream may additionally require the generator and software version, initial state or seed, stream/substream and draw order, and all intervening random-number calls. A seed alone is not necessarily sufficient if software or call order changes.
Worked Example: Replaying a Two-Arm Assignment
Suppose a hypothetical trial has arms A and B, ordered A then B. At one decision, the algorithm records probabilities \(P(A)=0.65\) and \(P(B)=0.35\), and its recorded uniform draw is \(U=0.72\). Assume the implementation uses the half-open inverse-transform intervals defined above.
The cumulative probabilities are \(c_1=0.65\) and \(c_2=1.00\). Thus A is selected when \(0\leq U<0.65\), and B when \(0.65\leq U<1.00\). Since \(0.65\leq 0.72<1.00\), the assignment is B. The arithmetic reproduces the recorded treatment.
If the record contained only the probabilities and the observed assignment B, an auditor could verify that B had positive probability, but could not recover the particular draw that selected B. Many possible draws, from \(0.65\) up to but not including \(1\), would yield B. If the recorded draw were instead \(U=0.60\), the claimed assignment B would be inconsistent with the recorded order and rule.
Keep the Information Boundary Visible
A decision-time record should make it possible to determine which records were eligible for the calculation and why. For each outcome used, retain its value, the participant and outcome definition it refers to, and the time it became available to the allocation procedure. For each pending participant, retain the pending status as of the decision, not a value later filled in from the final analysis data. A timestamped history or immutable snapshot is preferable to a current database view that silently overwrites earlier statuses.
A useful consistency check is temporal: every outcome listed as available at \(t_i\) must have an availability time no later than \(t_i\), under the study's stated timestamp convention. A value whose availability time is later than the decision cannot be part of that decision-time state. Resolution time alone does not establish availability; a result resolved at 09:00 but entered at 14:00 was not available to a decision made at 11:00 if the allocation system could not access it before entry.
Worked Example: A Resolved but Unavailable Outcome
Consider a hypothetical allocation decision made at 11:00 on 4 May. Participant 17's outcome is later determined to have resolved at 09:00, but the result is entered into the system at 14:00. Participant 18's outcome was entered and accessible at 10:30. A third participant, 19, is known to have withdrawn before outcome collection.
At 11:00, participant 17's result is not available to the algorithm. The decision-time record must mark that outcome pending or otherwise unavailable, while preserving the later resolution and availability times in the event history. It must not place the eventual outcome value among the inputs to the 11:00 calculation.
Participant 18's outcome is available at 11:00 and can be included if the allocation rule uses it. Participant 19 should be recorded as a known dropout, not as an ordinary unresolved outcome. That distinction preserves the information actually known at the decision and avoids treating a withdrawal as a feedback delay. The rule's eligibility for these statuses depends on the allocation method; the record documents them but does not decide that policy.
This classification follows the information boundary: participant 17's availability time, 14:00, is later than the decision time, 11:00; participant 18's availability time, 10:30, is earlier; and participant 19's withdrawal status was known. Reconstructing the decision from final outcomes without these times would not establish what the algorithm could have used.
A Record That Supports an Audit
A compact record can be organized as an immutable decision header plus a linked snapshot of the relevant participant states. The header identifies the decision, timestamp, participant being assigned, algorithm version, ordered arms, allocation probabilities, random draw information, and realized assignment. The linked snapshot identifies the prior participants and the statuses or outcome values that were available to the calculation. If a calculation uses summaries rather than individual-level records, retain both the summaries and sufficient underlying inputs to verify how those summaries were obtained.
| Record component | What it establishes | Typical failure if omitted |
|---|---|---|
| Decision time and participant identifier | Which allocation event is being reconstructed | Decision records may be matched to the wrong participant or sequence |
| Outcome values and availability times | What response information could enter the calculation | Later feedback may be mistaken for information available earlier |
| Pending, dropout, and other known statuses | Which participants had unresolved or unavailable data, and why | Missing feedback and withdrawal become indistinguishable |
| Algorithm inputs, version, and settings | How the recorded probabilities should be checked | Probability calculations cannot be independently traced |
| Arm order, probabilities, and draw | How the realized assignment follows from randomization | A valid probability vector may still not reproduce the observed arm |
An aggregate log that stores only the number of responses per arm may be convenient, but it can hide whether the correct responses were included at a particular decision. Conversely, storing an entire data set without decision-time status and availability timestamps does not guarantee reproducibility: the final data may contain information unavailable at the time. A decision-time snapshot or a faithful event history is the important complement.
Worked Audit of a Three-Arm Record
Worked Example: Checking a Recorded Assignment
In a hypothetical three-arm trial, the recorded arm order is Control, Dose 1, Dose 2. The record gives probabilities \(0.20\), \(0.50\), and \(0.30\), respectively, and the uniform draw is \(U=0.68\). The assignment is recorded as Dose 1. Check the probability vector and the assignment.
First, the probabilities are nonnegative and sum to one: \(0.20+0.50+0.30=1.00\). The cumulative probabilities are \(c_1=0.20\), \(c_2=0.20+0.50=0.70\), and \(c_3=1.00\). The intervals are \([0,0.20)\) for Control, \([0.20,0.70)\) for Dose 1, and \([0.70,1.00)\) for Dose 2.
Because \(0.20\leq 0.68<0.70\), the draw selects Dose 1. The recorded assignment is consistent with the recorded probabilities, order, and draw. If the arm order had been Control, Dose 2, Dose 1, the same probability list and draw would instead select Dose 2. This is why an arm order is part of the record, not an incidental software detail.
Diagnostics, Alternatives, and Limitations
A basic audit compares the recorded assignment with the selection implied by the recorded probabilities, arm order, draw, and mapping convention. It also checks that probabilities are finite, nonnegative, and sum to one within a stated numerical tolerance; that the assigned arm has positive probability; and that no input outcome has an availability time after the decision. Mismatches suggest an implementation, timestamp, transcription, or version-control problem and should be investigated rather than silently repaired.
Common warning signs include repeated missing draws, changing historical statuses with no event timestamps, a probability vector that cannot be traced to its inputs, and an assignment that does not fall in the draw's interval. Another warning sign is using final outcome data to recreate an earlier state. These checks assess the integrity of the record; they do not by themselves prove that the allocation rule was scientifically appropriate or that the trial's outcomes are unbiased.
There are several possible documentation strategies. A decision-time snapshot is direct and convenient for replay but can require substantial storage. An event log recording each enrollment, outcome availability, withdrawal, and status change can be more compact and flexible, but reconstruction depends on correctly ordering events and applying the exact timestamp convention. Aggregate summaries are easiest to inspect but may omit detail needed to check inclusion decisions. A robust implementation may retain an event history alongside a snapshot or a verifiable summary for each decision.
No record can recover information that was never captured. Logging the assignment and probability vector without the draw allows a partial audit but usually not exact replay. Logging a random seed without preserving the generator, software version, and call sequence may also fail to reproduce the same draw. Finally, a perfect record supports traceability, not a guarantee of correct outcome measurement, correct model assumptions, or appropriate treatment allocation. Those questions require separate scientific and statistical assessment.
Common Mistake
Record the assignment time and the participant being allocated, using a consistent clock and timestamp convention.
Record the prior participants' relevant treatment, outcome, availability, and pending or dropout statuses as of that time.
Store algorithm version and inputs, ordered arms, allocation probabilities, random draw or draws, mapping convention, and realized assignment.
Verify the probabilities, timestamps, and assignment-draw mapping; retain later updates as new events rather than overwriting the historical state.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- An outcome resolved at 08:00, became accessible to the algorithm at 12:00, and the allocation decision occurred at 10:00. Was it available for that decision? What should the record show?
- Two arms are ordered A then B, with probabilities \(0.40\) and \(0.60\). Under the half-open inverse-transform convention, which arm is selected by \(U=0.40\)?
- What can be checked from the probabilities and observed assignment alone, and what additional item is needed to replay an inverse-transform assignment?
- Why should a known dropout not be recorded simply as pending?
Solutions. 1. No. Its availability time, 12:00, is later than the 10:00 decision, even though it resolved earlier. Record it as unavailable or pending at 10:00, retaining the later resolution and availability times. 2. B, because A's interval is \([0,0.40)\) and B's is \([0.40,1.00)\); the boundary value \(0.40\) belongs to B. 3. The probabilities and assignment can establish whether the assigned arm had positive probability, but they do not identify the draw that produced it. Replaying inverse-transform selection requires the draw, along with the arm order and mapping convention. 4. A known dropout is a different observed status from an outcome still awaiting feedback. Keeping them distinct prevents the record from misrepresenting what was known and allows later audits to identify the reason an outcome is absent.