Tutorials › Response-Adaptive Randomization › Outcome-Specific Replacement Matrices

Home  /  Tutorials  /  Urn Models

Urn Models · Tutorial 49 of 501

Outcome-Specific Replacement Matrices

You will learn to select the correct replacement matrix for an observed response and treatment, then calculate and check the resulting urn state.

Intermediate 11 min read

What You'll Learn

  • Define an outcome-specific replacement matrix and its treatment-column convention.
  • Calculate an urn update for a specified treatment and response category.
  • Distinguish the matrix-encoded additions from returning the drawn ball.
  • Check updated urn states and the allocation probabilities they imply.
  • Identify indexing errors and limitations of outcome-specific updates.

Why the response category matters

An urn design changes its treatment-allocation probabilities as balls are added to or removed from the urn. In a simple play-the-winner rule, the update depends on whether the response is a success or a failure. A single replacement matrix cannot express that distinction unless the same update is intended for both outcomes. Outcome-specific replacement matrices make the distinction explicit: one matrix applies after one response category, and a different matrix applies after another.

The essential operation is to use both pieces of the observed treatment-response pair. The assigned treatment determines which column to select; the response category determines which matrix to use. Reversing either choice changes the urn update.

Definition: Let \(x=(x_1,x_2)^T\) be the urn-state vector for two treatments, where \(x_j\) is the number of balls of treatment \(j\). For each response category \(k\), an outcome-specific replacement matrix \(R^{(k)}\) is a \(2\)-by-\(2\) matrix whose column \(j\) gives the numbers of balls to add for treatments 1 and 2 after treatment \(j\) is assigned and response category \(k\) is observed. If treatment \(j\) is assigned, the drawn ball is returned, and response \(k\) occurs, the updated state is \(x'=x+R^{(k)}e_j\), where \(e_j\) is the vector with a 1 in position \(j\) and 0 elsewhere.

In this convention, rows identify the treatment type of the balls being added, and columns identify the treatment assigned. The matrices describe additions after the drawn ball has been returned. Thus, returning the drawn ball is not another addition encoded in \(R^{(k)}\). Unless a design explicitly allows removals, entries are nonnegative integers.

For example, if the trial has success and failure categories, it has two matrices, \(R^{(S)}\) and \(R^{(F)}\). To update after a particular participant, first identify the observed response category, select that matrix, and then use the column for the assigned treatment. The composition determines the next allocation probabilities: if the new state is \(x'\), treatment \(j\) is drawn with probability \(x'_j/(x'_1+x'_2)\), provided the total number of balls is positive.

Applying the selected matrix

Consider these hypothetical success and failure matrices:

$$ R^{(S)}= \begin{pmatrix} 1 & 2\\ 0 & 1 \end{pmatrix}, \qquad R^{(F)}= \begin{pmatrix} 0 & 1\\ 2 & 0 \end{pmatrix}. $$

If treatment 2 is assigned, use the second column of whichever matrix matches the response. If success is observed, the addition vector is \((2,1)^T\). If failure is observed, it is \((1,0)^T\). In both cases the treatment assigned is the same; the response category changes the update matrix.

Worked Example: Updating After Treatment 2

Suppose a hypothetical urn has state \(x=(4,3)^T\), and treatment 2 is assigned. The participant has a success. Use the matrices above to find the new state and next allocation probabilities.

Because the response is a success, select \(R^{(S)}\). Treatment 2 selects the second column, which can also be calculated by multiplying by \(e_2=(0,1)^T\):

$$ R^{(S)}e_2 = \begin{pmatrix} 1 & 2\\ 0 & 1 \end{pmatrix} \begin{pmatrix} 0\\ 1 \end{pmatrix} = \begin{pmatrix} 1(0)+2(1)\\ 0(0)+1(1) \end{pmatrix} = \begin{pmatrix} 2\\ 1 \end{pmatrix}. $$

The replacement adds 2 treatment-1 balls and 1 treatment-2 ball. The previously drawn treatment-2 ball is returned as stipulated, so the state becomes \(x'=(4,3)^T+(2,1)^T=(6,4)^T\). There are 10 balls in total, so the next draw assigns treatment 1 with probability \(6/10=0.600\) and treatment 2 with probability \(4/10=0.400\).

This calculation gives a useful check on matrix multiplication: the first entry of the result is the first row dotted with the selected column, and the second entry is the second row dotted with that column. A row-by-column calculation must preserve that order. In the example, the first entry is \(1(0)+2(1)=2\), while the second is \(0(0)+1(1)=1\).

Worked Example: A Different Response, the Same Assigned Treatment

Start instead from the same hypothetical state \(x=(4,3)^T\). Treatment 2 is assigned again, but this time failure is observed. Find the updated state and its allocation probabilities.

Failure selects \(R^{(F)}\), and treatment 2 selects its second column:

$$ R^{(F)}e_2 = \begin{pmatrix} 0 & 1\\ 2 & 0 \end{pmatrix} \begin{pmatrix} 0\\ 1 \end{pmatrix} = \begin{pmatrix} 0(0)+1(1)\\ 2(0)+0(1) \end{pmatrix} = \begin{pmatrix} 1\\ 0 \end{pmatrix}. $$

After returning the drawn ball, add this vector: \(x'=(4,3)^T+(1,0)^T=(5,3)^T\). The total is 8, so the next allocation probabilities are \(5/8=0.625\) for treatment 1 and \(3/8=0.375\) for treatment 2. The contrast with the previous example is due to the response-specific matrix, not a change in the assigned treatment.

Why the update formula holds

The matrix operation is a compact way to select one column. Let \(e_j\) have a 1 in position \(j\) and zeros elsewhere. In the product \(R^{(k)}e_j\), every column except column \(j\) is multiplied by zero, while column \(j\) is multiplied by one. Therefore the product equals precisely the addition vector prescribed for assigned treatment \(j\) and observed response category \(k\).

$$ R^{(k)}e_j = \begin{pmatrix} R^{(k)}_{1j}\\ R^{(k)}_{2j} \end{pmatrix}, \qquad x'=x+ \begin{pmatrix} R^{(k)}_{1j}\\ R^{(k)}_{2j} \end{pmatrix}. $$

This also explains the roles of the two indices: \(k\) selects the matrix, while \(j\) selects a column within that matrix. The state update is simply the old counts plus the additions prescribed by the observed pair.

Worked Example: Checking Both Possible Updates from One State

For a further hypothetical check, let \(x=(2,3)^T\) and suppose treatment 2 is assigned. Using the same matrices, calculate the two possible post-response states. This is a comparison of possible updates, not a claim that both responses occur for the same participant.

If success occurs, the selected column is \(R^{(S)}e_2=(2,1)^T\), so the state becomes \((2,3)^T+(2,1)^T=(4,4)^T\). The next treatment probabilities would each be \(4/8=0.500\).

If failure occurs, \(R^{(F)}e_2=(1,0)^T\), so the state becomes \((2,3)^T+(1,0)^T=(3,3)^T\). The next treatment probabilities would each be \(3/6=0.500\). The allocation probabilities happen to match in this example, but the urn states and totals do not. Equal probabilities at one update do not mean that the matrices or the resulting states are equivalent.

Assumptions, checks, and alternatives

To apply this update rule, the urn state must record nonnegative counts for the treatment types, exactly one treatment must be identified as assigned, and the observed response must map unambiguously to one of the response categories with a specified matrix. The matrix dimensions must match the number of treatment types. The drawn ball is returned according to the stated urn rule, and the matrix entries are interpreted as additional balls. These conditions are enough to calculate an update; they do not require a particular probability model for patient responses.

If probabilities of future paths are also being calculated, a response model is needed in addition. For instance, one may specify that after assignment to treatment \(j\), response category \(k\) has probability \(q_{jk}\), with \(\sum_k q_{jk}=1\). Such probabilities describe how likely the response categories are; the replacement matrices describe how the urn changes once a category is observed. The update formula itself remains valid if response probabilities vary over time, as long as the observed category and its applicable matrix are defined.

An outcome-independent replacement rule is the special case in which the same replacement matrix is used for every response category. It is simpler, but it cannot prescribe different updates for success and failure. Outcome-specific matrices can encode that distinction, as in play-the-winner designs, but their use alone does not guarantee desirable allocation probabilities, balance, or ethical advantage. A matrix is a rule for changing counts, not evidence that one treatment is more effective.

Useful checks after each calculation are to confirm that the selected column corresponds to the assigned treatment, the matrix corresponds to the observed response, and the state update adds the correct entries. Then verify that all resulting counts are nonnegative and that the new total equals the old total plus the sum of the selected column. Finally, recompute allocation probabilities using the updated counts and total, not the pre-update state.

  • Column/row reversal: The assigned treatment selects a column, not a row. A row gives the amount added of one treatment across the possible assignments.
  • Wrong response matrix: Choosing a treatment column correctly is not enough; using the success matrix after a failure still gives the wrong update.
  • Returning the ball twice: The state \(x\) already contains the drawn ball after it is returned. Add only the selected replacement column.
  • Confusing response probabilities with additions: A response probability such as \(q_{jk}\) is not an entry in the replacement matrix. One governs chance of response; the other governs counts added after the response.
  • Ignoring an invalid state: If an update would create a negative count or an undefined total, the specified rule or its implementation needs review rather than silent continuation.
Key takeaway: For an observed treatment-response pair, choose the response-specific matrix first and then select the column for the assigned treatment. Add that column to the returned-ball urn state, and calculate allocation probabilities from the resulting counts.

Check Your Understanding

Answer each question using the matrix convention in this tutorial. Complete solutions follow the questions.

  1. If treatment 1 is assigned and success is observed, which matrix and column are used?
  2. Using the displayed matrices, what addition vector follows treatment 1 and failure?
  3. From state \((3,2)^T\), treatment 1 is assigned and success occurs. What is the new state and the probability of drawing treatment 2 next?
  4. Why is the drawn ball not added again as part of the replacement-matrix column?

Solutions. (1) Use \(R^{(S)}\), column 1, because the response is success and treatment 1 was assigned. (2) Use column 1 of \(R^{(F)}\): \(R^{(F)}e_1=(0,2)^T\). (3) Column 1 of \(R^{(S)}\) is \((1,0)^T\), so the new state is \((3,2)^T+(1,0)^T=(4,2)^T\). Its total is 6, and the probability of treatment 2 is \(2/6=0.3333\). (4) The rule states that the drawn ball is returned before the replacement additions are made; the state being updated already includes that returned ball, so the matrix column records only additional balls.

Previous

Next

Back to Tutorials
Clinical Biostats