Feedback Latency and Allocation Efficiency

Why Feedback Timing Affects Allocation

A response-adaptive rule can use only information that is available when an allocation is made. If outcomes take time to become usable, assignments made in the meantime reflect an older estimate of treatment performance. Updating the allocation rule more often does not make those outcomes arrive sooner. Conversely, waiting for a larger batch can provide a more precise estimate, but delays the next allocation change. The practical question is therefore not simply whether to update often or infrequently: it is how much adaptation is lost while waiting, and what precision is gained by batching.

The calculations below isolate that timing trade-off. They use a simple target-tracking model rather than prescribing a particular response-adaptive randomization rule. The target allocation must be specified for the application: it might be the output of a chosen response-adaptive rule, not necessarily the allocation that maximizes any one clinical or statistical objective.

Latency, Batch Size, and Tracking Loss

The squared loss treats deviations above and below the target symmetrically and penalizes larger deviations more heavily. It is useful for comparing timing policies against the same target; it does not establish that the target itself is appropriate. Other loss functions may be more suitable when the costs of over- and under-allocation differ.

For a first calculation, suppose the target changes abruptly at an assignment boundary and remains fixed. Before the first usable response from the new regime, assume the allocation remains at its old value. With fixed latency \(d\), and updates every \(b\) assignments, the change must also wait for the next batch update. If the change occurs at a uniformly random position within the batch cycle, the expected additional wait to that update is \((b-1)/2\) assignments. Thus the expected time until the first opportunity to react is approximately

This approximation counts assignments and treats the response delay as fixed. The batch term is zero when \(b=1\); for larger batches, it is the mean residual wait to a scheduled update. It describes the first opportunity to react, not how quickly the allocation reaches its new target after that update.

A Justification for the Batch Waiting Term

Consider a cycle of \(b\) assignments between updates. If a target change occurs just after an update, nearly a full cycle remains; if it occurs just before the next update, almost none remains. Under a uniform position within the cycle, the residual number of assignments has possible values \(0,1,\ldots,b-1\), each with probability \(1/b\). Its mean is

Adding the fixed outcome latency gives the expression above. The calculation assumes that target changes are not synchronized with batch boundaries. If changes occur only at boundaries, the residual wait is instead zero; if changes tend to occur just after an update, the average residual wait is larger than the uniform-position value.

From Waiting Time to Allocation Tracking Loss

A transparent way to quantify the initial cost of waiting is to consider a sudden target shift from \(q_0^*\) to \(q_1^*\). While the allocation remains at its old value \(q_0^*\), the loss per assignment is constant:

If the allocation begins at the old target and does not change until the first usable update, the expected cumulative loss during this initial waiting period is approximately \(E(W)(q_0^*-q_1^*)^2\). This expression isolates the waiting component. It does not include any further mismatch caused by a gradual update rule, noisy estimates, or a target that continues to move.

In practice, an update usually changes the allocation by an amount determined by an estimate rather than setting it immediately to the target. For illustration, let \(q_j\) be the allocation probability after update \(j\), and suppose a stable target \(q^*\) is approached according to

Here \(\gamma\) is a fixed adaptation fraction. Subtracting \(q^*\) from both sides gives \(q_{j+1}-q^*=(1-\gamma)(q_j-q^*)\). Repeated application yields

Thus, for \(0<\gamma<1\), the discrepancy shrinks geometrically per update. The number of assignments required for a given number of updates depends on \(b\); a larger batch means fewer opportunities to apply this contraction over a fixed enrollment period. This derivation assumes a fixed target and a noiseless update fraction. Real rules use estimated targets, so their trajectories also reflect estimation variability.

Why Larger Batches Can Still Help

A larger batch can provide more information for an update. For example, under independent Bernoulli outcomes with fixed response probabilities \(p_A\) and \(p_B\), the difference between two sample proportions is less variable when each treatment has more observations. If each group has \(m\) observations, its standard error is

This formula assumes independent observations, fixed response probabilities, and \(m\) observations per treatment. In an adaptive trial, group sizes may differ, and the corresponding variance expression must use each group’s actual sample size. This simple comparison describes a precision benefit of information volume; it does not establish that a particular batch size is optimal.

Comparing Update Strategies

These approaches answer different operational needs. Frequent updates minimize extra waiting caused by the schedule, but they cannot eliminate the latency between an outcome and its availability. Larger batches can make an update more stable, but the timing calculation alone cannot show whether that stability is worth the extra delay. Comparing strategies requires the same target definition, a stated delay model, and attention to the amount and quality of information available at each update.

Assumptions, Diagnostics, and Limitations

The simple timing calculation is most informative when latency and update schedules are well characterized. It relies on fixed latency, or on a correctly specified average latency, and on a clear rule for when an outcome is usable. The uniform-position assumption is needed for the \((b-1)/2\) residual-wait term. If delays vary across participants or treatments, the average delay alone may not describe the information available at an update: a large backlog can make the usable data systematically older than expected. Earlier material on informative delays and treatment-specific delay distributions addresses why delay patterns themselves need scrutiny.

Useful operational checks include comparing assignment times with outcome-availability times, tracking the number of unresolved outcomes at each update, and recording the age of the data used to calculate each allocation probability. A mismatch between planned and observed update times, a growing backlog, or unequal delay patterns across treatments signals that a fixed-latency calculation may be misleading. Also verify that a batch is defined consistently: a batch of assignments is not automatically a batch of completed outcomes.

A common mistake is to conclude that updating after every response guarantees fast adaptation. It does not: if responses are unavailable for many assignments, an update rule that is triggered by usable responses still has to wait. The correction is to separate the outcome latency \(d\) from the extra batch wait \((b-1)/2\), and to report both the assumed delay and update schedule. Another mistake is to infer that the largest batch is best because it gives a smaller standard error. Precision is only one consideration; the allocation can be further from its target while waiting for that precision. Finally, tracking loss is target-dependent: a small value means close tracking of the specified target, not necessarily superior patient outcomes or more efficient treatment-effect estimation.