From Response Categories to Urn Updates
In a multi-treatment response-adaptive urn, the response to an assigned treatment can have more than two possible categories. A binary success/failure rule needs only two response-specific updates; with three or more categories, each category must be matched to its own replacement vector. The vector specifies how the counts of all treatment balls change after that response. Once the update rule is explicit, the urn state determines the next allocation probabilities.
The response category and the treatment assigned are distinct pieces of the process. The current urn state determines which treatment is assigned; the assigned treatment influences the distribution of the response; and the observed response category selects the replacement vector. The examples below use that sequence explicitly.
Each entry of \(d_c\) is a signed change in a treatment's ball count. A positive entry adds balls, a zero entry leaves that count unchanged, and a negative entry removes balls. This convention treats the replacement vector as a change, not as the final number of balls. The updated state must have nonnegative integer entries and a positive total if another allocation is to be made.
Let \(q_{ic}=P(Y=c\mid A=i)\), where \(A\) is the assigned treatment and \(Y\) is the response category. For each treatment \(i\), the category probabilities satisfy \(q_{ic}\geq 0\) and \(\sum_{c=1}^C q_{ic}=1\). This allows treatment-specific response distributions while keeping the replacement vector for a given category common across treatments. The vector rule is fixed for the update being described; state-dependent rules require specifying how the vector changes with the state.
One-Step Transition Probabilities
The urn state alone does not determine the response: first a treatment is drawn, then its response category is realized. Consequently, the probability of a category is a mixture over possible assigned treatments. Conditional on state \(x\), the chance of a transition by \(d_c\) is
This follows by partitioning the event \(Y=c\) according to the mutually exclusive assigned treatments. For each \(i\), the joint probability of assigning \(i\) and observing \(c\) is \((x_i/S)q_{ic}\); summing these joint probabilities gives the displayed result. If two categories have identical replacement vectors, they lead to the same next state and their probabilities must be added to obtain the probability of that state transition.
The new total urn size is determined by the sum of the entries in the selected vector:
Thus the next allocation probabilities, if the update is feasible, are \((x_j+d_{cj})/S'\). A category can change the composition without changing the total size, increase or decrease the total, or change some treatment counts in opposite directions. The vector entries, not the category label or its numerical code, determine those effects.
Worked Multi-Treatment Updates
Worked Example: Three Treatments and Three Multinomial Responses
Suppose the current state is \(x=(4,3,2)\), so \(S=4+3+2=9\). There are three possible response categories, \(C_1,C_2,C_3\), with replacement vectors \(d_1=(1,0,0)\), \(d_2=(0,1,0)\), and \(d_3=(0,0,1)\). For this hypothetical example, the response probabilities are:
| Assigned treatment | \(C_1\) | \(C_2\) | \(C_3\) |
|---|---|---|---|
| 1 | 0.60 | 0.30 | 0.10 |
| 2 | 0.20 | 0.50 | 0.30 |
| 3 | 0.10 | 0.30 | 0.60 |
The current treatment allocation probabilities are \(4/9,3/9,2/9\). The one-step probability of response \(C_1\), allowing for all three possible assignments, is
Similarly,
These probabilities sum to \(1.0001\) when rounded, and to exactly \(1\) before rounding. The three possible next states are \(x+d_1=(5,3,2)\), \(x+d_2=(4,4,2)\), and \(x+d_3=(4,3,3)\). Each update adds one ball, so each new total is \(10\). For example, if treatment 2 is assigned and category \(C_3\) is observed, the same category vector applies: the state is \((4,3,2)+(0,0,1)=(4,3,3)\), with next allocation probabilities \(4/10,3/10,3/10\).
The model assumptions used here are explicit: the response probabilities in each row sum to one, the allocation probabilities follow the current ball proportions, and each response category triggers its stated vector. All resulting counts are nonnegative and the total remains positive.
Worked Example: Ordinal Responses With Different Updates
Suppose three treatments are in state \(x=(2,3,1)\), with \(S=6\). A hypothetical ordinal response has categories poor, intermediate, and good, in that order. Assigning a numerical order to the labels does not make the gaps between categories quantitatively meaningful; the update rule is specified category by category. Let the replacement vectors be
Suppose treatment 2 is assigned and the observed response is good. The state changes by the good-response vector:
Therefore the allocation probabilities for the next assignment are \(3/8=0.3750\) for treatment 1, \(3/8=0.3750\) for treatment 2, and \(2/8=0.2500\) for treatment 3. The good category adds two balls in total, while the poor and intermediate categories each add one. This calculation uses the category's vector; it does not treat “good” as a numeric score or multiply a common vector by an ordinal score.
For a complete probability calculation, suppose the response probabilities for treatment 1 are \((0.20,0.50,0.30)\), for treatment 2 are \((0.10,0.40,0.50)\), and for treatment 3 are \((0.30,0.40,0.30)\), listed as poor, intermediate, good. Then the probability of a good response from the current state is
The response rows each sum to one, and every vector adds balls, so all possible updates from this state are feasible. In particular, the probability \(0.4000\) is the probability of the good-response update, not the probability that treatment 2 is assigned and produces a good response. That joint probability would instead be \((3/6)(0.50)=0.2500\).
Worked Example: Two Categories, One State Transition
Consider three treatments with state \(x=(3,2,1)\), total \(S=6\), and four possible response categories. Suppose their replacement vectors are \(d_1=(1,0,0)\), \(d_2=(1,0,0)\), \(d_3=(0,1,0)\), and \(d_4=(0,0,1)\). Categories 1 and 2 are distinct responses but produce the same state update. Let their response probabilities by assigned treatment be:
| Assigned treatment | Category 1 | Category 2 | Category 3 | Category 4 |
|---|---|---|---|---|
| 1 | 0.40 | 0.30 | 0.20 | 0.10 |
| 2 | 0.10 | 0.20 | 0.40 | 0.30 |
| 3 | 0.20 | 0.20 | 0.30 | 0.30 |
The allocation probabilities are \(3/6,2/6,1/6\). The marginal probabilities of categories 1 and 2 are
Both categories lead to state \((4,2,1)\), so the probability of that state transition is \(0.2667+0.2500=0.5167\) (using unrounded values, \(3.10/6\)). The other two transitions are to \((3,3,1)\) with probability \(1.70/6\approx0.2833\), and to \((3,2,2)\) with probability \(1.20/6=0.2000\). The transition probabilities sum to \(1\). Keeping the categories separate is necessary if later calculations use category-specific outcomes, even though they can be combined when calculating this next-state distribution.
Checking the Rule Before Using It
A replacement-vector specification is a valid urn rule only if its updates can actually be carried out in the states the design may reach. For every possible state \(x\) and response category \(c\) that can occur there, check
If the rule is intended to continue for multiple assignments, checking only the initial state is not sufficient: an update may create a later state in which a removal is impossible or the urn is empty. A design with negative entries therefore needs a clear feasibility condition or a rule that prevents invalid updates. Also check that each response-probability row sums to one, that the treatment allocation probabilities sum to one, and that the reported next-state probabilities sum to one.
The one-step mean change can also be obtained directly from the category probabilities. Conditional on state \(x\), the state change is \(d_c\) with probability \(P(Y=c\mid x)\). Therefore its conditional expected value is
The second equality substitutes the treatment-mixture formula for each category probability and rearranges the finite sums. This is a vector expectation, component by component. It summarizes the average one-step change, but it does not describe the full distribution of possible states: distinct category vectors and their probabilities are needed for that.
Common Mistakes, Alternatives, and Limits
- Using a response category as an update amount. A category called “3” does not mean “add three balls.” The replacement vector must be specified; category labels alone define no urn operation.
- Updating only the assigned treatment count. A replacement vector can change any or all treatment counts. Apply every component of \(d_c\), unless the design explicitly defines a different rule.
- Confusing conditional and marginal probabilities. \(q_{ic}\) is conditional on assigning treatment \(i\). The probability of category \(c\) at state \(x\) averages across treatments using \(x_i/S\).
- Forgetting coincident transitions. If two categories have the same vector, add their probabilities when reporting the probability of the resulting state, while retaining the categories separately when their meanings matter.
- Ignoring feasibility after an update. A negative replacement entry may be valid in one state but invalid in another. Check the rule over all states it can reach.
A binary-response rule is the special case \(C=2\); it has two vectors and the same assignment-then-response calculation. A multinomial or ordinal rule uses one vector for each response category, allowing more detailed updates than a binary summary. A replacement matrix is a compact way to store these vectors—for example, with one row per response category and one column per treatment—but the row convention must be stated so the update is unambiguous. Fixed allocation is different: treatment probabilities do not change through an urn state update. These approaches answer different design questions, and an urn rule should not be interpreted as fixed allocation merely because its probabilities are known at the start.
The method also has limits. The vector rule does not by itself specify the response model \(q_{ic}\), guarantee desirable allocation properties, or establish a clinical benefit. A correct one-step calculation is conditional on the stated response probabilities and current state. For a full trial analysis, the dependence induced by successive updates matters; a one-step mean or a single transition table is not a substitute for analyzing the trial's complete allocation and response process.
Check Your Understanding
Answer each question using the stated urn convention. Solutions follow the questions.
- With state \(x=(5,2,3)\), what is the probability of assigning treatment 3?
- For a response category with vector \(d=(0,1,1)\), find the updated state and the next allocation probabilities from \(x=(2,3,1)\).
- At state \(x=(2,1)\), a response has vector \((-1,0)\). Is this update feasible? What additional issue must be checked for a multi-step rule?
- Two response categories have probabilities \(0.18\) and \(0.27\) at the current state and share the same replacement vector. What is the probability of the state transition they produce?
- In one sentence, distinguish a treatment-conditional response probability from a marginal response-category probability at a given urn state.
Solutions. 1. The total is \(5+2+3=10\), so treatment 3 is assigned with probability \(3/10=0.3000\). 2. The updated state is \((2,3,1)+(0,1,1)=(2,4,2)\), with total \(8\), so the next allocation probabilities are \(2/8,4/8,2/8\), or \(0.2500,0.5000,0.2500\). 3. The updated state is \((1,1)\), so this one-step update is feasible and the urn remains nonempty. For a multi-step rule, check that every later reachable state also permits the removal and retains a positive total. 4. The transition probability is \(0.18+0.27=0.45\), because the categories lead to the same state. 5. A treatment-conditional response probability is \(P(Y=c\mid A=i)\) for a specified treatment, whereas the marginal probability at state \(x\) averages these probabilities over treatments using their current urn allocation probabilities.