Why Record Each Allocation?
In response-adaptive randomization, an allocation is made using information accrued before that participant is assigned. A useful audit record must therefore preserve more than the treatment selected: it should show what information the algorithm used, what allocation probabilities it produced, and how its randomization selected the treatment. That record supports checks for implementation errors and helps distinguish the intended algorithm from what actually happened.
Let \(t\) index allocation occasions, \(j=1,\ldots,K\) index treatment arms, and \(\mathcal{D}_{t-1}\) denote the trial information available immediately before allocation \(t\). The posterior state is the information needed by the specified model to determine the posterior distribution at that time. It may include the accrued response data, model parameters or sufficient statistics, and the model and prior settings. An audit should identify the actual state used, not a state reconstructed later from data that may have been updated or processed differently.
The distinction matters when the algorithm imposes minimum allocation floors, uses an estimated rather than exact posterior probability, or otherwise transforms the best-arm probabilities. Record both the posterior quantities and the final probabilities actually used for randomization. A vector of probabilities calculated for reporting is not necessarily the vector used by the allocation software.
What an Allocation Record Should Contain
A reproducible record links the allocation to a specific pre-allocation state and a specific execution of the randomization procedure. The following fields provide a practical audit structure. Exact storage formats depend on the implementation, but the underlying information should be unambiguous.
| Record item | What it establishes |
|---|---|
| Allocation index and time | Which assignment is being audited and when it occurred. |
| Pre-allocation data state | The data cutoff, relevant accrued outcomes, and any model summaries used at allocation \(t\). |
| Model and algorithm version | The response model, prior, utility definition, tie rule, constraints, and software or configuration version. |
| Posterior probability vector | The \(q_{t,j}\) values, with enough precision and method details to identify how they were obtained. |
| Operational probability vector | The \(\pi_{t,j}\) values actually passed to the randomization step, including any floors or renormalization. |
| Randomization record | The random draw or posterior draw used, the random-number generator details needed for replay where available, and the mapping from draw to arm. |
| Assigned treatment | The arm actually assigned, recorded with its arm label or code and linked to the participant. |
Store probabilities at adequate precision; rounded values may not sum to one or may obscure which interval contained a draw near a boundary. A log should distinguish an exact probability from a Monte Carlo estimate and retain the estimation method, simulation count, and relevant seed or random-stream information if the calculation is to be reproduced. A seed alone may not guarantee an identical replay if software, parallel execution, or random-number generation has changed.
There can be two different random operations. One implementation estimates posterior best-arm probabilities and then draws an arm from the operational probability vector. Another draws a posterior parameter or utility vector and assigns the arm that is best in that draw. These procedures may produce related probabilities, but their audit trails differ. Record which operation the implementation used; if there are two stages, preserve the outputs and random draws from both.
Worked Example: Auditing a Probability Vector After Floors
Suppose a hypothetical three-arm trial has posterior best-arm probabilities \(q_t=(0.70,0.20,0.10)\). The prespecified allocation rule uses a floor \(\varepsilon=0.10\) by assigning each arm the floor and distributing the remaining probability in proportion to \(q_t\):
Here \(K=3\), so \(1-K\varepsilon=1-3(0.10)=0.70\). The operational probabilities are
They sum to \(0.59+0.24+0.17=1.00\), and each is at least \(0.10\). Suppose the allocation step draws \(U_t=0.63\) from a uniform distribution on \([0,1)\), independently of the probability calculation. The cumulative intervals are \([0,0.59)\) for arm 1, \([0.59,0.83)\) for arm 2, and \([0.83,1)\) for arm 3. Since \(0.59\leq 0.63<0.83\), arm 2 is assigned.
The audit should record \(q_t\), the floor rule and its version, \(\pi_t\), the draw \(U_t\), the interval mapping, and the assigned arm. Recording only the posterior vector would not reveal the floor-adjusted probability; recording only the assigned arm would not show whether the randomization was consistent with the operational vector.
Why the Recorded Vector and Draw Match
Suppose the operational probability vector \(\pi_t\) is fixed once the pre-allocation state and any probability-estimation step are complete. Assume its entries are nonnegative and sum to one. Define cumulative cut points \(C_{t,0}=0\) and \(C_{t,j}=\sum_{\ell=1}^{j}\pi_{t,\ell}\). If the allocation draw \(U_t\) is uniform on \([0,1)\) and independent of the probability calculation, the implementation assigns arm \(j\) when \(C_{t,j-1}\leq U_t<C_{t,j}\). Conditional on both the pre-allocation data and the realized vector, the probability of that event is the interval length:
This is the key audit check: given the realized operational vector, the chosen arm must correspond to the recorded draw's interval. The interval endpoints have probability zero under a continuous uniform draw; an implementation should nevertheless use a specified, consistent boundary convention.
The conditioning is important. If \(\pi_t\) is estimated by posterior simulation, it can itself vary across simulation runs even when \(\mathcal{D}_{t-1}\) is held fixed. In that setting, \(\pi_t\) is random conditional on \(\mathcal{D}_{t-1}\). If the allocation draw is conditionally independent of the randomness used to estimate \(\pi_t\), then averaging over that estimation randomness gives
Thus a realized estimated probability is the assignment probability conditional on that realized vector, not generally the assignment probability conditional only on accrued data. If the allocation draw and the probability-estimation randomness are dependent, even the expectation formula requires care: the joint randomization mechanism must be described and audited rather than assuming independence.
Worked Example: A Monte Carlo Probability Estimate
For a hypothetical allocation, suppose 400 posterior simulations yield 240, 120, and 40 draws in which arms 1, 2, and 3 are best. The estimated best-arm probabilities are
Suppose the same three-arm floor rule with \(\varepsilon=0.10\) is used. The estimated operational vector is
The sum is \(1.00\). If the independently generated allocation draw is \(U_t=0.50\), it lies in arm 1's interval \([0,0.52)\), so arm 1 is assigned. The record should label \(\hat{q}_t\) and \(\hat{\pi}_t\) as estimates, preserve the 400 simulation count and the simulation procedure, and separately store \(U_t=0.50\).
A different Monte Carlo run on the same accrued data could produce a different estimated vector. For example, counts of 220, 128, and 52 would give \(\hat{q}_t=(0.55,0.32,0.13)\), and the floor rule would give \(\hat{\pi}_t=(0.485,0.324,0.191)\). The latter is not the same realized probability vector as in the first run. This variation is why an audit must retain the vector actually used, rather than replacing it with a later, more precise estimate.
Worked Example: Conditional and Marginal Assignment Probabilities
Consider a hypothetical two-arm allocation with fixed accrued data but a randomized probability-estimation step. Suppose that step produces \(\pi_t=(0.80,0.20)\) with probability \(1/2\), and \(\pi_t=(0.40,0.60)\) with probability \(1/2\). Given the realized vector, an independent allocation draw assigns arm 1 with probability \(0.80\) or \(0.40\), respectively.
Conditional on the first realized vector, \(P(\text{arm 1}\mid\mathcal{D}_{t-1},\pi_t=(0.80,0.20))=0.80\). Conditional on the second, that probability is \(0.40\). But before the probability-estimation step is observed, the probability of arm 1 given only the accrued data is
So the realized vector is essential to auditing that particular allocation, while the marginal probability \(0.60\) describes the mechanism averaged over its probability-estimation randomness. Substituting a realized \(0.80\) for that marginal probability would be incorrect.
Diagnostics, Alternatives, and Limitations
A basic allocation audit checks that the recorded probabilities are nonnegative and sum to one within a stated numerical tolerance, that any floors are respected, and that the draw maps to the recorded arm under the documented boundary convention. It also checks that the model state and data cutoff precede the assignment, and that the assigned treatment in the allocation log agrees with the participant record. A failed check is a reason to investigate the implementation or record; it does not by itself identify which one is wrong.
For exact posterior probabilities, one can record the computed vector and the method that produced it. For simulation-based probabilities, retain the simulation count, estimated vector, software and configuration details, and random stream information where practical. Saving all posterior draws may aid detailed reanalysis but can be costly; summary counts and a securely retained reproducibility record may be a practical alternative. Neither choice removes the need to record the final operational vector and assigned arm.
There is also a distinction between replay and verification. If the random-number state and software environment are preserved, a replay may reproduce the same draw and assignment. If they are not, the saved draw and vector can still verify that the original assignment was consistent with the logged probabilities, but may not regenerate it. Conversely, a saved seed without the software version, draw order, and probability vector may be insufficient for exact replay.
This audit record describes the allocation mechanism as implemented; it does not, by itself, establish that the response model, utility, or trial design is clinically appropriate. Nor does a marginal assignment probability averaged over simulation randomness replace the realized probability vector when checking an individual draw. The right probability depends on what is being conditioned on and which random stage is under examination.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- What is the difference between \(q_{t,j}\) and \(\pi_{t,j}\) when a minimum allocation floor is applied?
- If an exact operational vector is \((0.50,0.30,0.20)\) and \(U_t=0.84\), which arm is assigned using left-closed, right-open intervals?
- A simulation-based operational vector is random given accrued data. Under what condition is the assignment probability given only those data equal to \(E[\pi_{t,j}\mid\mathcal{D}_{t-1}]\)?
- Name two pieces of information, besides the selected arm, that help verify or reproduce an allocation.
- Why can a saved seed be insufficient for exact replay?
Solutions. (1) \(q_{t,j}\) is the posterior probability that arm \(j\) is best; \(\pi_{t,j}\) is the probability actually used by randomization after the floor rule. (2) The cumulative intervals are \([0,0.50)\), \([0.50,0.80)\), and \([0.80,1)\); \(0.84\) selects arm 3. (3) The allocation draw must be conditionally independent of the probability-estimation randomness given the accrued data, with the draw using the realized vector. (4) Examples include the operational probability vector and random draw, or the pre-allocation data state and algorithm version. (5) Different software, random-number generators, execution order, or configurations can change the draws even when the seed is unchanged.