Recursing Expected Counts Over a Short Trial

Why Recurse Over Urn States?

In a play-the-winner urn, the next treatment depends on the current urn composition, and the next composition depends on both the assigned treatment and the patient’s response. Consequently, an early assignment probability is generally not a fixed fraction of the initial balls. To calculate it, we track the probabilities of the possible urn states as patients are enrolled.

The state probabilities provide two useful summaries. They give the expected number of balls for each treatment after a specified number of patients, and they give the probability that a particular treatment is assigned to the next patient. These quantities are related, but they are not the same: an expected ball count describes the urn, while an assignment count describes patients.

Model and State-Probability Recursion

Consider two treatments, A and B. Initially the urn contains $a$ A-balls and $b$ B-balls, where $a$ and $b$ are positive integers. For each patient, one ball is drawn uniformly, the treatment indicated by the ball is assigned, and the ball is returned. The patient has a binary response: conditional on receiving A, success has probability $p_A$; conditional on receiving B, success has probability $p_B$. Responses are generated according to these probabilities given the assigned treatment. After each response, exactly $r$ new balls are added: a success adds balls for the treatment just assigned, and a failure adds balls for the other treatment. Here $r$ is a fixed positive integer.

These assumptions specify the transition rule. In particular, the returned ball remains in the urn, and the reinforcement size is the number of new balls added, not the number of balls including the returned ball. The recursion below treats the response probabilities as fixed throughout the short trial.

Definition: Let $q_n(i,j)$ be the probability that, after $n$ patients, the urn contains $i$ A-balls and $j$ B-balls. Initially, $q_0(a,b)=1$, and all other initial-state probabilities are zero. Define $g_A(i,j)$ as the probability of adding $r$ A-balls at the next update, given current state $(i,j)$, and $g_B(i,j)$ as the probability of adding $r$ B-balls. Then $$ g_A(i,j)=\frac{i}{i+j}p_A+\frac{j}{i+j}(1-p_B), \qquad g_B(i,j)=\frac{i}{i+j}(1-p_A)+\frac{j}{i+j}p_B. $$ The state probabilities satisfy $$ q_{n+1}(i,j)=q_n(i-r,j)g_A(i-r,j)+q_n(i,j-r)g_B(i,j-r), $$ where a term is zero if its predecessor state is impossible. The first term represents an update that adds A-balls; the second represents one that adds B-balls.

The two contributions to $g_A(i,j)$ are: drawing A and observing success, or drawing B and observing failure. The two contributions to $g_B(i,j)$ are: drawing A and observing failure, or drawing B and observing success. Since the two updates exhaust the possible responses,

$$ g_A(i,j)+g_B(i,j)=1. $$

After $n$ patients, each patient has added exactly $r$ balls to one of the two treatments. Thus the total urn size is $a+b+nr$, and there are at most $n+1$ possible states: one for each possible number of updates that added A-balls. This makes direct recursion practical for short trials.

From State Probabilities to Assignment and Ball Counts

At state $(i,j)$, the next patient receives A with probability $i/(i+j)$. Averaging this probability over the states after $k-1$ patients gives the probability of assigning A to patient $k$:

P(\text{patient }k\text{ receives A}) =\sum_{i,j}q_{k-1}(i,j)\frac{i}{i+j}.

The expected number of A assignments among the first $n$ patients is the sum of the individual assignment probabilities. By contrast, the expected number of A-balls after $n$ patients is obtained by averaging the A-ball count across the states:

E(N_A(n))=\sum_{k=1}^{n}P(\text{patient }k\text{ receives A}), \qquad E(C_A(n))=\sum_{i,j}i\,q_n(i,j).

Here $N_A(n)$ denotes the number of patients assigned A and $C_A(n)$ the number of A-balls after $n$ patients. These are distinct random quantities. One patient assigned A can lead to reinforcement of B if the response is a failure.

Justification of the Recursion

To be in state $(i,j)$ after $n+1$ patients, the trial must have been in one of two states after $n$ patients. It can have been at $(i-r,j)$, followed by an update that adds A-balls; or at $(i,j-r)$, followed by an update that adds B-balls. These alternatives are mutually exclusive, and every path to $(i,j)$ has exactly one of these predecessor states. By conditioning on the predecessor state and multiplying the probability of that state by the appropriate transition probability, we obtain the recursion. Adding the two contributions therefore counts all paths into $(i,j)$, with none omitted or counted twice.

Worked Example: One-State Transition Probabilities

Suppose the current urn has 3 A-balls and 2 B-balls, $p_A=0.70$, $p_B=0.40$, and $r=1$. Verify the update probabilities from this state.

The probability of drawing A is $3/5$, and the probability of drawing B is $2/5$. An A-ball is added after an A success or a B failure:

g_A(3,2)=\frac{3}{5}(0.70)+\frac{2}{5}(1-0.40) =0.42+0.24=0.66.

A B-ball is added after an A failure or a B success:

g_B(3,2)=\frac{3}{5}(1-0.70)+\frac{2}{5}(0.40) =0.18+0.16=0.34.

The next state is $(4,2)$ with probability $0.66$, or $(3,3)$ with probability $0.34$. The probabilities sum to 1, as required. The assignment probability for the current patient was $3/5=0.60$; it is not the same as either update probability.

Worked Example: Recursing Through Three Patients

Take an initial urn with $a=b=2$, $r=1$, $p_A=0.70$, and $p_B=0.40$. Calculate the assignment probabilities for the first three patients and the expected ball counts after two patients.

For patient 1, the initial assignment probability for A is $2/4=0.50$. If A is assigned, the probability of adding an A-ball is $0.70$; if B is assigned, the probability of adding an A-ball is $1-0.40=0.60$. Therefore,

q_1(3,2)=0.50(0.70)+0.50(0.60)=0.65, \qquad q_1(2,3)=1-0.65=0.35.

For patient 2, average the probability of drawing A over these two states:

P(\text{patient 2 receives A}) =0.65\frac{3}{5}+0.35\frac{2}{5} =0.39+0.14=0.53.

To get the state probabilities after patient 2, first calculate the update probabilities. From $(3,2)$, $g_A=0.66$, so the contributions to $(4,2)$ and $(3,3)$ are $0.65(0.66)=0.429$ and $0.65(0.34)=0.221$. From $(2,3)$,

g_A(2,3)=\frac{2}{5}(0.70)+\frac{3}{5}(0.60)=0.64, \qquad g_B(2,3)=0.36.

That state contributes $0.35(0.64)=0.224$ to $(3,3)$ and $0.35(0.36)=0.126$ to $(2,4)$. Thus $q_2(4,2)=0.429$, $q_2(3,3)=0.221+0.224=0.445$, and $q_2(2,4)=0.126$. These probabilities sum to 1.

The probability that patient 3 receives A is consequently

0.429\frac{4}{6}+0.445\frac{3}{6}+0.126\frac{2}{6} =0.286+0.2225+0.042=0.5505.

The expected A-ball count after two patients is

$$ E(C_A(2))=0.429(4)+0.445(3)+0.126(2) =1.716+1.335+0.252=3.303. $$

The total is 6 balls in every state after two patients, so $E(C_B(2))=6-3.303=2.697$. The expected number of A assignments among those two patients is instead $0.50+0.53=1.03$. This difference is not an error: ball reinforcement depends on responses as well as assignments.

Worked Example: Ball Counts Do Not Give Assignment Counts

Consider an urn with 1 A-ball and 3 B-balls, reinforcement size $r=2$, and response probabilities $p_A=0.60$ and $p_B=0.25$. Find the expected A-ball count after patient 1 and the probability that patient 2 receives A.

Patient 1 receives A with probability $1/4$ and B with probability $3/4$. An A update occurs after an A success or a B failure, so

P(\text{A update}) =\frac{1}{4}(0.60)+\frac{3}{4}(1-0.25) =0.15+0.5625=0.7125.

The state after patient 1 is $(3,3)$ with probability $0.7125$, and $(1,5)$ with probability $0.2875$. Therefore,

$$ E(C_A(1))=0.7125(3)+0.2875(1)=2.425. $$

For patient 2, average the A-draw probability over those states:

P(\text{patient 2 receives A}) =0.7125\frac{3}{6}+0.2875\frac{1}{6} =0.35625+0.0479167 =0.4042.

The expected A-ball count is 2.425, but the next-patient A assignment probability is about 0.4042. Substituting the expected counts into a ratio would give $2.425/6=0.4042$ in this particular calculation only because both possible states have the same total of 6 balls. In general, the relevant assignment probability is the average of the state-specific ratios, not a ratio formed from averages; when total urn size varies across states, those expressions need not agree.

Practical Checks and Limitations

A recursion is only as reliable as its state definitions and transition probabilities. Useful checks for a short calculation include verifying that every state has the correct total number of balls, that all state probabilities are nonnegative, and that the probabilities at each patient index sum to 1. Also check that the update rule matches the response coding: under the rule used here, success reinforces the assigned treatment and failure reinforces the other treatment.

A common mistake is to calculate an assignment probability as the expected number of A-balls divided by the expected total number of balls. The exact calculation averages $i/(i+j)$ over states. If the total is fixed at a particular patient index, the ratio of expected counts does equal the average ratio; if total size varies across states, it generally does not. In either case, the state-probability calculation makes the relevant averaging explicit.

Direct recursion is a convenient alternative to listing every possible sequence of assignments and responses. It combines paths that reach the same urn state, while preserving their total probability. Simulation is another alternative, useful when the trial is too long for direct enumeration, but it estimates probabilities with Monte Carlo error rather than calculating them exactly. The recursion itself does not estimate $p_A$ or $p_B$, address uncertainty in those probabilities, or establish that the design is clinically or ethically appropriate. Its exact results are conditional on the stated urn rule and response probabilities.

Key takeaway: Propagate state probabilities one patient at a time. Use them to average state-specific allocation probabilities for a specified patient, and to average ball counts for expected urn composition; do not confuse these two summaries.

Check Your Understanding

Answer each question, then compare with the solutions below.

At state $(2,3)$, with $p_A=0.50$ and $p_B=0.20$, what is the probability of adding an A-ball?
An urn starts at $(1,1)$, has $r=1$, and has $p_A=0.80$, $p_B=0.30$. What are the possible states after one patient and their probabilities?
Why is $E(C_A(n))$ not generally the expected number of patients assigned A?
After two patients, an urn has states $(3,1)$ with probability $0.40$ and $(1,3)$ with probability $0.60$. What is the probability that patient 3 receives A?

Solutions. 1. $g_A(2,3)=\frac{2}{5}(0.50)+\frac{3}{5}(1-0.20)=0.20+0.48=0.68$. 2. The initial A-draw probability is $1/2$. An A update has probability $\frac12(0.80)+\frac12(0.70)=0.75$, giving state $(2,1)$; the remaining probability $0.25$ gives state $(1,2)$. 3. A-ball counts change through response-dependent reinforcement, whereas assigned-patient counts record only which treatment was given. 4. Average the state-specific probabilities: $0.40(3/4)+0.60(1/4)=0.30+0.15=0.45$.