Exact Probabilities of Short Play-the-Winner Sequences

Why a Short Path Has an Exact Probability

In a play-the-winner design, the probability of the next treatment assignment depends on the urn’s current composition. That composition, in turn, depends on earlier responses. To find the probability of a specified sequence, it is therefore not enough to multiply the assignment probabilities from the beginning of the trial, or to treat the assignments as independent. The calculation must follow the path in order: draw a treatment, account for its response, update the urn, and then calculate the next draw probability.

Here the target is the probability of a complete path: a specified treatment assignment and a specified success or failure response at each step. The calculation is exact under the stated urn and response model. It is a probability of one event under that model, not a test of a treatment effect or a summary of the design’s overall allocation properties.

Model and Path Probability

Consider two treatments, A and B. Just before patient \(t\) is assigned, let \(u_t\) and \(v_t\) be the numbers of A and B balls in the urn. The total number of balls is \(u_t+v_t\). A ball is drawn at random, the patient receives the treatment named by its color, and the drawn ball is returned. Thus, given the current state, the treatment probabilities are \(u_t/(u_t+v_t)\) for A and \(v_t/(u_t+v_t)\) for B.

Assume the two-arm play-the-winner update with a fixed reinforcement size \(m\), a positive integer. After a success, add \(m\) balls of the color drawn; after a failure, add \(m\) balls of the other color. The drawn ball has already been returned, so it remains part of the urn in either case. Write \(Y_t=1\) for success and \(Y_t=0\) for failure. Suppose success probabilities are fixed at \(\theta_A\) and \(\theta_B\) for patients assigned to A and B, respectively. Responses are generated sequentially with these probabilities, independently of the urn draw conditional on the assigned treatment and the past. There are no missing responses or other changes to the urn in the short path being calculated.

Definition: A complete path of length \(k\) specifies \(T_t\), the treatment assigned at step \(t\), and \(Y_t\), the response, for every \(t=1,\ldots,k\). Its probability is the product, in time order, of each conditional urn-draw probability and the corresponding conditional response probability, with the urn state updated after every specified response.

For a specified history, define \(q_t=P(T_t=A\mid\text{history before step }t)\). It equals \(u_t/(u_t+v_t)\); the probability of assigning B is \(1-q_t\). Let \(\theta_{T_t}\) denote the success probability for the assigned treatment. The response factor is \(\theta_{T_t}\) if \(Y_t=1\) and \(1-\theta_{T_t}\) if \(Y_t=0\). Therefore the path probability is

P(T_1,Y_1,\ldots,T_k,Y_k) = \prod_{t=1}^{k} P(T_t\mid\text{history before }t) P(Y_t\mid T_t,\text{history before the response}).

This is the chain rule for the joint probability, with each conditional probability evaluated at the state created by the earlier steps. The model makes the treatment probability depend on the current urn state and the response probability depend on the assigned treatment; the response then determines the next state. For any specified step, all factors are conditional probabilities between 0 and 1.

Worked Path Calculations

Worked Example: Two Assignments to A Followed by B

Suppose the initial urn has \(u_1=2\) A balls and \(v_1=2\) B balls, \(m=1\), \(\theta_A=0.80\), and \(\theta_B=0.50\). Find the probability of the complete path: A success, A failure, B success.

At step 1, the probability of drawing A is \(2/4=0.50\). The probability of success on A is \(0.80\). After that success, add one A ball, so the state before step 2 is \((3,2)\).

At step 2, the probability of drawing A is \(3/5=0.60\), and the probability of failure on A is \(1-0.80=0.20\). A failure adds one B ball, giving state \((3,3)\) before step 3.

At step 3, the probability of drawing B is \(3/6=0.50\), and the probability of success on B is \(0.50\). Thus

P(\text{A success, A failure, B success}) =(0.50)(0.80)(0.60)(0.20)(0.50)(0.50) =0.0120.

Under this model, the probability of this particular three-patient assignment-and-response path is \(0.0120\), or 1.20%. It is not the probability of every path with two assignments to A and one to B; those paths may have different responses and urn states.

Worked Example: Failure Adds Balls to the Other Treatment

Start with \((u_1,v_1)=(1,3)\), use \(m=2\), and let \(\theta_A=0.70\) and \(\theta_B=0.60\). Find the probability of B failure, A success, A success.

The first draw is B with probability \(3/4\). B fails with probability \(1-0.60=0.40\). Because failure adds balls of the other color, two A balls are added, and the state becomes \((3,3)\).

The next draw is A with probability \(3/6=0.50\), followed by success with probability \(0.70\). This adds two A balls, producing \((5,3)\). The final draw is A with probability \(5/8=0.625\), followed by success with probability \(0.70\). Multiplication gives

P(\text{B failure, A success, A success}) =(0.75)(0.40)(0.50)(0.70)(0.625)(0.70) =0.04594.

The two-ball reinforcement changes later draw probabilities but does not change the response probabilities specified by the model. In particular, B’s failure probability is \(0.40\), not A’s failure probability.

When Responses Are Not Specified

An assignment sequence alone is not a complete path. If the responses are unspecified, there are multiple possible response histories, and they can produce different urn states. The probability of the assignment sequence is the sum of the probabilities of its mutually exclusive response paths. Each term must be calculated with the appropriate intermediate urn states; using one arbitrarily chosen response history will generally give the wrong answer.

Worked Example: Probability of an Assignment Sequence Alone

The initial state is \((1,1)\), \(m=1\), \(\theta_A=0.75\), and \(\theta_B=0.50\). Find the probability of assignments A, B, A when responses are not specified.

The first assignment is A with probability \(1/2\). If the first response is a success, which has probability \(3/4\), the state becomes \((2,1)\), and the next B draw has probability \(1/3\). After that B assignment, success and failure are equally likely: success leads to \((2,2)\), where the probability of drawing A is \(1/2\); failure leads to \((3,1)\), where it is \(3/4\). Averaging over those responses, the conditional probability of the final A draw is

\left(\frac{1}{2}\right)\left(\frac{1}{2}\right) +\left(\frac{1}{2}\right)\left(\frac{3}{4}\right) =\frac{5}{8}.

The contribution from a first-step success is therefore \((1/2)(3/4)(1/3)(5/8)=5/64\).

If the first response is a failure, with probability \(1/4\), the state becomes \((1,2)\), so the next B draw has probability \(2/3\). A success on B leads to \((1,3)\), where the final A probability is \(1/4\); a failure leads to \((2,2)\), where it is \(1/2\). Their average is \((1/2)(1/4)+(1/2)(1/2)=3/8\). This branch contributes \((1/2)(1/4)(2/3)(3/8)=2/64\). Summing the disjoint response branches,

P(\text{assignments A, B, A})=\frac{5}{64}+\frac{2}{64} =\frac{7}{64}=0.1094.

The response probabilities matter even though the event specifies assignments only: they determine which urn states can precede the later assignments.

Justification and Checks

The product formula follows by applying the chain rule to the events in time order. At a given step, first condition on everything observed before the draw. The probability of the specified treatment is determined by the current urn state. Then condition on that treatment and the history to obtain the specified response probability. The update rule maps this treatment-response pair to the state for the next step. Repeating these conditional factorizations gives the product for the complete path. No independence between successive assignments is assumed or needed.

A useful check is to verify that, from any state, the probabilities of all possible next treatment-response pairs sum to one. For example, the four possibilities have probabilities

\frac{u_t}{u_t+v_t}\theta_A,\quad \frac{u_t}{u_t+v_t}(1-\theta_A),\quad \frac{v_t}{u_t+v_t}\theta_B,\quad \frac{v_t}{u_t+v_t}(1-\theta_B),

and their sum is \(u_t/(u_t+v_t)+v_t/(u_t+v_t)=1\). This also gives a diagnostic for an implementation: if the four one-step probabilities do not sum to one, a draw probability, response probability, or state convention has been applied incorrectly.

Common Errors, Alternatives, and Limits

Keeping the original draw probabilities. It is wrong to use the initial urn proportions at every step. Recalculate the proportions after every response-driven update.
Updating before the response. The response determines which color receives reinforcement. First calculate the current assignment and response probabilities; then update for the next patient.
Forgetting the returned ball. The drawn ball is returned under this model. Reinforcement adds balls to that urn; it does not replace the drawn ball with the added balls.
Using the wrong response probability. A failure on A has probability \(1-\theta_A\), while a failure on B has probability \(1-\theta_B\).
Multiplying only the assignment factors for a complete path. That omits the probability of observing the specified responses. Assignment factors alone are appropriate only for a conditional allocation calculation with responses treated as given; they are not the joint probability of a specified assignment-and-response path.
Calling a path probability a p-value. A path probability is the probability of a specified event under a model. It is not, by itself, a measure of evidence against a null hypothesis or a probability that the model is true.

For a complete path, direct multiplication is the clearest approach. For an assignment-only event, summing over response histories is required; as the number of patients grows, that sum can become large. A fixed-randomization calculation is a different alternative: it uses assignment probabilities that do not depend on responses, so it does not reproduce the response-driven urn path probabilities. Exact calculations are particularly useful for short sequences and for checking a probability calculation, but a very small probability for one path does not by itself describe the overall behavior of an adaptive design. Results also depend on the model assumptions, including the reinforcement rule and the response probabilities. If those probabilities are estimated rather than treated as known, a plug-in path probability does not account for their estimation uncertainty.

Key takeaway: For a specified complete path, use the urn composition immediately before each draw, multiply that draw probability by the probability of the specified response, and update the urn before moving to the next step. If responses are unspecified, sum over the possible response paths.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

An urn starts at \((2,1)\), with \(m=1\), \(\theta_A=0.80\), and \(\theta_B=0.60\). What is the probability of B success followed by A failure?
With \(m=1\), what is the state after an A assignment followed by failure from an initial state of \((2,2)\)?
An urn starts at \((1,1)\), with \(m=1\) and \(\theta_A=0.50\). What is the probability of assignments A, A when responses are unspecified?
Why is the probability of a complete assignment-and-response path not automatically a p-value?

Solutions. 1. B is drawn with probability \(1/3\) and succeeds with probability \(0.60\), giving state \((2,2)\). A is then drawn with probability \(1/2\) and fails with probability \(0.20\). The path probability is \((1/3)(0.60)(1/2)(0.20)=0.0200\). 2. Failure on A adds one B ball, so the state is \((2,3)\). 3. The first A has probability \(1/2\). Its success and failure are each probability \(1/2\); they lead to states \((2,1)\) and \((1,2)\), respectively, with next-A probabilities \(2/3\) and \(1/3\). Thus \(P(\text{A,A})=(1/2)[(1/2)(2/3)+(1/2)(1/3)]=1/4\). 4. A p-value is defined relative to a null hypothesis and a specified test statistic or extremeness criterion. A single path probability gives neither by itself.