Play-the-Winner with Three Treatments

Why the Three-Treatment Rule Needs a Precise Update

With two treatments, a play-the-winner rule can respond to failure by reinforcing the one alternative treatment. With three treatments, failure on the assigned treatment leaves two alternatives. A rule must specify how both alternatives are handled; otherwise the urn update, and therefore later allocation probabilities, are ambiguous.

This tutorial uses one direct three-treatment extension of the earlier material on two-treatment play-the-winner: return the drawn ball, reinforce the assigned treatment after success, and reinforce each of the two unassigned treatments after failure. Other multi-treatment rules are possible, so the failure update is part of the rule’s definition rather than something to infer from the name alone.

This is the one-ball-per-colour play-the-winner rule for three treatments. It is not the only possible three-treatment extension. The rule requires a binary response recorded as success or failure, an assigned treatment identified for each response, and the stated return-and-reinforcement procedure. An initial composition with at least one ball of each colour ensures that every treatment remains available: the rule adds balls but does not remove them.

State Updates and Allocation Probabilities

If the current state is \((a,b,c)\), its total number of balls is \(T=a+b+c\). Because each ball is equally likely to be drawn, the next treatment-allocation probabilities are the colour proportions in the urn. The update depends on both the treatment drawn and the binary response.

These probabilities add to one because \(a+b+c=T\). A success adds one ball, so the total increases from \(T\) to \(T+1\). A failure adds two balls, so the total increases from \(T\) to \(T+2\). In either case, the drawn ball is already accounted for in the original composition because it is returned; it is not an additional reinforcement.

Transition Probabilities and Response Assumptions

The urn update is deterministic once the drawn treatment and binary response are known. Randomness enters through the draw and, in a response model, through the outcome. To calculate probabilities for a sequence involving responses, one must specify how response probabilities are represented. A common working model assumes each response is Bernoulli with treatment-specific success probability \(\pi_A,\pi_B,\pi_C\), with responses conditionally independent given the assigned treatments. The allocation mechanism itself does not require these probabilities to be equal or known.

For example, from state \((a,b,c)\), the probability of drawing A and then observing a success is \(\frac{a}{T}\pi_A\) under that model. The state then becomes \((a+1,b,c)\). The probability of drawing A and observing a failure is \(\frac{a}{T}(1-\pi_A)\), and the state becomes \((a,b+1,c+1)\). These are probabilities for the combined draw-and-response event; the update table alone does not specify the clinical response probabilities.

Short Justification of the Transition Probabilities

The transition probabilities follow directly from the random draw and the response model. For a current state \((a,b,c)\), the events “draw A,” “draw B,” and “draw C” are mutually exclusive and exhaustive. Their probabilities are \(a/T\), \(b/T\), and \(c/T\). Conditional on drawing A, success has probability \(\pi_A\) and failure has probability \(1-\pi_A\). Therefore the probabilities of the two A-related transitions are

The two probabilities sum to \(a/T\), the probability of drawing A. Applying the same argument to B and C gives all six possible draw-and-response transitions. Their total probability is \(\frac{a}{T}+\frac{b}{T}+\frac{c}{T}=1\), so the transition probabilities account for every possible next event. This calculation relies on the binary outcome classification and on using the appropriate response probability for the treatment drawn.

How the Rule Differs From Alternatives

The two-treatment play-the-winner rule reinforces the assigned treatment after success and the other treatment after failure. In the three-treatment rule here, a failure reinforces both alternatives, because there are two of them. This extension preserves the basic “success favours the assigned treatment; failure favours alternatives” idea, but it does not make the same-sized update after success and failure: a success adds one ball, whereas a failure adds two.

Under equal randomization, each treatment would instead be assigned with probability \(1/3\) at every step, regardless of earlier responses. The urn rule is response-adaptive: prior assignments and outcomes change the composition, which can change subsequent probabilities. This change is a feature of the allocation mechanism, not evidence by itself that one treatment is more effective. A response observed in a small sample can alter later allocations even when treatment response probabilities are identical.

The rule also differs from any procedure that, after failure, adds a ball to only one alternative or removes a ball of the failed treatment. Such procedures produce different states and probabilities. A protocol or implementation must say exactly which update is used, including whether the drawn ball is returned and how many balls are added.

Diagnostics, Limitations, and Common Mistakes

A useful audit records the composition before each assignment, the colour drawn, the binary outcome, and the resulting composition. At each step, check that the draw probabilities are nonnegative and sum to one, that the update matches the assigned treatment and response, and that the total rises by one after success or by two after failure. With all three initial counts positive, none should become zero under this add-only rule.

• Updating before returning the drawn ball. The drawn ball is returned in this rule. Treating it as removed changes both the next composition and the allocation probabilities.
• Adding only one alternative after a three-treatment failure. The rule defined here adds one ball to each of the two alternatives. Adding to only one is a different procedure.
• Reinforcing the failed treatment. A failure on B adds A and C balls, not another B ball. The B ball drawn is simply returned.
• Using the wrong denominator. Allocation probabilities use the current total after the most recent update, not the starting total or the previous state’s total.
• Treating adaptation as proof of benefit. More balls of one colour mean a larger probability of assignment under this rule; they do not establish that treatment has a higher success probability.

The rule is simple but has limitations. Because failures add two balls and successes add one, the total urn size depends on the observed response history. Randomization probabilities can become quite unequal after a sequence of outcomes, and early chance outcomes can affect later allocations. The rule also presumes that the recorded response is available in time for the next update. It does not, by itself, address delayed responses, competing outcomes, changing treatment effects, or how treatment comparisons should be analyzed.