Translating the Rule into Sequential Pseudocode

From a Rule to a Sequential Algorithm

A DBCD rule is applied repeatedly: use the information available before the next participant is assigned, compute probabilities for the arms, make a random assignment, and then record the resulting data. Translating that description into pseudocode requires careful timing. In particular, the response from one participant can affect later assignments only after it has been observed and entered into the data.

The algorithm below assumes a fixed set of \(K\) treatment arms and a binary response, coded 1 for a success and 0 otherwise. For clarity, responses are observed before the next participant is assigned. Participants’ responses are assumed to be generated under the specified trial conditions, and the random assignment is independent of future response noise conditional on the information available at assignment. The target map and response estimator must be specified in advance; the example target map will be normalized estimated success rates. Other prespecified target maps can be substituted without changing the sequence of computational steps.

Definition: Immediately before participant \(t\) is assigned, let \(N_k(t-1)\) be the number assigned to arm \(k\), and let \(S_k(t-1)\) be the number of successes recorded on that arm. The response estimate is \(\hat{\theta}_k(t-1)=S_k(t-1)/N_k(t-1)\) when \(N_k(t-1)>0\). A prespecified target map converts the vector of response estimates into target shares \(\hat{\rho}_k(t-1)\), where each share is positive and \(\sum_{k=1}^K\hat{\rho}_k(t-1)=1\). The current allocation share is \(q_k(t-1)=N_k(t-1)/(t-1)\), when \(t-1>0\).

For the standard power form used here, choose a prespecified tuning parameter \(\gamma\geq 0\). The unnormalized weight for arm \(k\) is its estimated target share multiplied by the target-to-current-share ratio raised to \(\gamma\). Normalizing these weights gives the assignment probabilities:

w_k(t)=\hat{\rho}_k(t-1)\left(\frac{\hat{\rho}_k(t-1)}{q_k(t-1)}\right)^\gamma, \qquad \pi_k(t)=\frac{w_k(t)}{\sum_{j=1}^K w_j(t)}.

This version requires positive current shares and positive target shares for the arms being considered. Thus the adaptive phase needs a valid initialization, such as assigning an initial set of participants across all arms before using the formula. Sparse-data estimation, zero-count handling, and maintaining positive assignment chances require prespecified choices; they should not be improvised inside a loop. The formula also presumes that the chosen response estimator and target map produce usable shares at each update.

Sequential Pseudocode

The sequence below distinguishes the assignment decision from the subsequent response update. It uses an initial allocation phase to ensure every arm has at least one observed response before calculating current shares. The initial phase is part of the design specification; its allocation method and size should be recorded.

Specify and initialize.
Fix the number of arms, response definition, estimator, target map, value of \(\gamma\), initialization rule, and random-number procedure. Create an assignment and response record for each participant. Run the prespecified initialization so that each arm has the data required by the adaptive rule.

Summarize the data available before participant \(t\).
For every arm, calculate its response estimate from its own recorded outcomes and calculate its current allocation share from its assignment count. Do not include participant \(t\), who has not yet been assigned.

Compute the target shares and assignment probabilities.
Apply the prespecified target map to the response estimates. Calculate each DBCD weight, sum the weights, and divide each weight by that sum. Check that all probabilities are finite, nonnegative, and sum to one up to numerical rounding.

Draw and record an arm.
Generate one random draw from the categorical distribution with probabilities \(\pi_1(t),\ldots,\pi_K(t)\). Record participant \(t\)'s assigned arm. Do not replace the draw with the arm having the largest probability.

Observe and record the response.
After the response is available, append it to the assigned arm’s response data and update that arm’s assignment and success counts. Leave the other arms’ response records unchanged. Proceed to participant \(t+1\).

The algorithm can be expressed as a repeated conditional randomization: given the data available just before assignment \(t\), the probability of assigning participant \(t\) to arm \(k\) is \(\pi_k(t)\). It does not say that the observed fraction assigned to arm \(k\) at every intermediate time equals \(\hat{\rho}_k(t-1)\). The target shares guide the next assignment probabilities; they are not deterministic quotas.

Worked Examples

Worked Example: Updating Three Arms Before the Next Draw

Consider hypothetical binary-response data on three arms. After 11 participants have been assigned, the counts are \(N=(4,3,4)\), and the success counts are \(S=(3,1,2)\). The latest participant was assigned to arm 3 and had response 0. Use the raw success proportions, normalize those proportions as the target shares, and set \(\gamma=1\).

The response estimates are calculated separately by arm:

\hat{\theta}_1=\frac{3}{4}=0.75,\qquad \hat{\theta}_2=\frac{1}{3}=0.3333,\qquad \hat{\theta}_3=\frac{2}{4}=0.5.

The target-map inputs sum to \(0.75+0.3333+0.5=1.5833\). Equivalently, the exact proportions are \(9/12,4/12,6/12\), whose sum is \(19/12\). Thus the target shares are \((9/19,4/19,6/19)\), approximately \((0.4737,0.2105,0.3158)\). The current shares, based on all 11 assignments, are \((4/11,3/11,4/11)\).

With \(\gamma=1\), calculate and normalize the weights. For example, the first weight is \((9/19)((9/19)/(4/11))\). Exact weights are proportional to \(2673,704,1188\), so their sum is \(4565\). The assignment probabilities are therefore:

(\pi_1,\pi_2,\pi_3) = \left(\frac{2673}{4565},\frac{704}{4565},\frac{1188}{4565}\right) = \left(\frac{243}{415},\frac{64}{415},\frac{108}{415}\right) \approx (0.5855,0.1542,0.2602).

These probabilities sum to 1. The response of 0 was already included in arm 3’s total of four participants and two successes; it does not change arm 1’s count or success total. This is precisely why an outcome update must be applied only to the arm that received the participant.

Worked Example: Drawing an Assignment and Updating the State

Suppose a hypothetical two-arm trial has \(N=(10,10)\) and \(S=(6,3)\). Use raw success proportions, the normalized-proportion target map, and \(\gamma=1\). A random-number generator returns \(U=0.73\), where \(U\) is uniform on \([0,1)\).

The estimates are \(6/10=0.6\) and \(3/10=0.3\), so the target shares are \(0.6/(0.6+0.3)=2/3\) and \(1/3\). Current shares are \(1/2\) and \(1/2\). The weights are

w_1=\frac{2}{3}\left(\frac{2/3}{1/2}\right)=\frac{8}{9}, \qquad w_2=\frac{1}{3}\left(\frac{1/3}{1/2}\right)=\frac{2}{9}.

Their sum is \(10/9\), giving probabilities \((0.8,0.2)\). Use cumulative intervals \([0,0.8)\) for arm 1 and \([0.8,1)\) for arm 2. Because \(0.73<0.8\), assign the participant to arm 1. If the observed response is 0, the new counts are \(N=(11,10)\) and \(S=(6,3)\). Arm 1’s estimate is now \(6/11\); arm 2’s remains \(3/10\). The update does not add the new zero to arm 2 or change its assignment count.

For an additional check, the next target shares are \(20/31\) and \(11/31\), since \(6/11:3/10=60:33=20:11\). The next current shares are \(11/21\) and \(10/21\). With \(\gamma=1\), the ratio of the next weights is \((20/31)^2/(11/21)\) divided by \((11/31)^2/(10/21)\), which equals \(4000/1331\). Hence the next probability for arm 1 is \(4000/(4000+1331)=4000/5331\approx0.7503\); the probability for arm 2 is approximately \(0.2497\).

Worked Example: Checking the Initialization Boundary

In a hypothetical three-arm trial, the design first assigns one participant to each arm. Their responses, in arm order, are 1, 0, and 1. At the start of the adaptive phase, \(N=(1,1,1)\), \(S=(1,0,1)\), and each current share is \(1/3\). Suppose the design specifies a smoothed response estimate \(\hat{\theta}_k=(S_k+1)/(N_k+2)\), followed by normalization to target shares, and uses \(\gamma=1\).

The estimates are \(2/3,1/3,2/3\), so their sum is \(5/3\) and the target shares are \((2/5,1/5,2/5)\). The DBCD weights are

w_1=\frac{2}{5}\left(\frac{2/5}{1/3}\right)=\frac{12}{25},\quad w_2=\frac{1}{5}\left(\frac{1/5}{1/3}\right)=\frac{3}{25},\quad w_3=\frac{2}{5}\left(\frac{2/5}{1/3}\right)=\frac{12}{25}.

The weights sum to \(27/25\), so the next probabilities are \((4/9,1/9,4/9)\). This example illustrates that initialization, estimator, and target map are design inputs, not details to leave implicit. The smoothing rule used here is only an illustrative prespecified choice; another rule would generally produce different target shares and a different allocation path.

Why Normalization and Timing Matter

The normalization step has a simple justification. If the weights \(w_k(t)\) are finite and nonnegative, and their sum is positive, then \(\pi_k(t)=w_k(t)/\sum_jw_j(t)\) is nonnegative and

\sum_{k=1}^K \pi_k(t) = \sum_{k=1}^K\frac{w_k(t)}{\sum_{j=1}^K w_j(t)} = \frac{\sum_{k=1}^K w_k(t)}{\sum_{j=1}^K w_j(t)} =1.

Thus the vector can be used as a categorical assignment distribution. If \(I_k(t)\) is 1 when participant \(t\) is assigned to arm \(k\), and 0 otherwise, then conditional on the information used to compute the probabilities, \(P(I_k(t)=1)=\pi_k(t)\). This is a probability statement about the next draw, not a guarantee about the realized allocation count.

The data-update logic can also be checked by induction. Before the first adaptive assignment, each stored count must equal the number of recorded participants in that arm, and each success count must equal the number of recorded successes there. At each iteration, exactly one arm receives one assignment; after its response is observed, that arm’s assignment count increases by one and its success count increases by the observed binary response. All other arm counts remain fixed. Therefore these meanings of the counts are preserved after every update.

Diagnostics, Alternatives, and Limitations

Before using a generated assignment, check a few invariants at every iteration: the sum of current assignment counts equals the number of participants already assigned; each success count is between zero and its arm’s assignment count; target shares and probabilities sum to one; and the chosen arm is consistent with the random draw and cumulative probability intervals. Preserve enough information to reproduce each step, including the data available before assignment, the estimates, target shares, probabilities, random draw, assigned arm, and subsequent response.

A sequential pseudocode implementation is one way to express the design. A deterministic target-assignment procedure instead tries to place participants directly into target proportions; it is not the same as drawing from DBCD probabilities. A fixed-allocation randomization procedure uses probabilities chosen in advance rather than updating them from responses. Both can be useful alternatives, but neither reproduces this response-adaptive rule.

The algorithm does not resolve whether the response estimator or target map is scientifically appropriate, whether responses are observed promptly, or how to handle missing outcomes. If responses are delayed, the implementation needs an explicit policy for what information is available at each assignment; silently treating a pending response as an observed failure changes the rule. In addition, the numerical probabilities describe the implemented design only if the recorded inputs, calculations, and random draw follow the specified procedure.

A common mistake is to update every arm’s response estimate after a participant responds, or to recompute the probabilities using that response before assigning the same participant. The correction is to compute the assignment probabilities from information available before assignment, draw and record the arm, and then update only that arm after its response is observed. Another error is to assign deterministically to the arm with the largest probability; DBCD requires a random draw with the full probability vector.

Key takeaway: A correct sequential DBCD implementation has a strict order: summarize past data, form target shares, calculate and normalize weights, draw an assignment, and then record the response. Each step must use the appropriate information set.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Before the next assignment, arm 1 has 8 participants and 5 successes; the newly observed participant was assigned to arm 2. Should arm 1’s estimate change?
For weights \((2,3,5)\), what are the normalized assignment probabilities?
A two-arm rule gives probabilities \((0.7,0.3)\), and the random draw is \(U=0.82\). Which arm is assigned using cumulative intervals in arm order?
Why must the response from participant \(t\) not be used to calculate the probabilities for assigning participant \(t\)?
Name two checks that can detect an implementation error after an iteration.

Solutions. 1. No. Arm 1 remains at \(5/8=0.625\); only the arm assigned to the new participant is updated. 2. The sum is 10, so the probabilities are \((0.2,0.3,0.5)\). 3. Assign arm 2 because \(0.82\) lies in \([0.7,1)\). 4. That response is not available before the assignment is made; using it would reverse the required order and make the assignment depend on a future observation. 5. For example, check that assignment counts sum to the number already assigned and that the probabilities sum to one. One can also check that success counts remain between zero and assignment counts and that the recorded arm matches the random draw.