Why Predictive Updating Is Different
When outcomes are delayed, the trial has information about who was assigned to each treatment but not yet about every assigned participant’s response. A predictive distribution describes what the unresolved responses might be, given the observed data and the model. It can support an allocation decision now, but it does not turn an unknown response into an observed one.
The key distinction is between predicting pending responses and using responses as if they had been observed. Predictive updating averages a decision rule across possible completions of the pending data, weighting each completion by its predictive probability. The result is an allocation probability for the next participant, not a claim about which pending responses actually occurred.
The rule \(r_a\) must be specified as part of the design: for example, it might allocate in proportion to posterior mean response rates, subject to a minimum probability for each treatment. Predictive averaging specifies how uncertainty about the pending responses enters that rule; it does not choose the rule for the trial.
Binary Responses Under a Beta-Binomial Model
Consider treatment arm \(a\), with binary response probability \(p_a\). A response is coded 1 and a nonresponse 0. Suppose the prior is \(p_a\sim\operatorname{Beta}(\alpha_a,\beta_a)\), with \(\alpha_a,\beta_a>0\). Assume responses are conditionally independent Bernoulli variables given \(p_a\), and that the prior distributions for different arms are independent. These assumptions are a simple model for arm-specific response probabilities; they do not account for covariates or treatment-effect structure.
If \(x_a\) responses and \(f_a\) nonresponses have matured in arm \(a\), the observed-data posterior is \(\operatorname{Beta}(A_a,B_a)\), where \(A_a=\alpha_a+x_a\) and \(B_a=\beta_a+f_a\). Write \(m_a\) for the number of pending outcomes in that arm and \(K_a\) for the number of responses among them. Conditional on the current posterior, \(K_a\) has a beta-binomial predictive distribution.
Here \(B(u,v)\) is the beta function. This formula comes from averaging the binomial probability conditional on \(p_a\) over its current posterior. After a particular pending count \(K_a=k\) is observed, the posterior becomes \(\operatorname{Beta}(A_a+k,B_a+m_a-k)\). Before that observation, the posterior remains a mixture over these possible updated posteriors, with the beta-binomial probabilities as weights.
The integral yields the beta-function expression above. Pending outcomes in different arms are predictively independent under the stated independent-arm model. Within an arm, however, pending responses are generally dependent after integrating over the uncertain \(p_a\): a response among the pending participants provides evidence about \(p_a\), which changes the predictive probability of another response.
Worked Example: Predicting Two Pending Responses
Suppose an arm has posterior \(p\mid D\sim\operatorname{Beta}(2,2)\) and two pending binary outcomes. Find the predictive probabilities for zero, one, or two responses.
Here \(A=B=2\) and \(m=2\). For zero responses, $$ P(K=0\mid D)=\frac{B(2,4)}{B(2,2)} =\frac{1/20}{1/6}=0.3000. $$ For one response, $$ P(K=1\mid D)=2\frac{B(3,3)}{B(2,2)} =2\frac{1/30}{1/6}=0.4000. $$ By symmetry, \(P(K=2\mid D)=B(4,2)/B(2,2)=0.3000\). The probabilities sum to \(0.3000+0.4000+0.3000=1.000\).
A plug-in calculation using the posterior mean \(E(p\mid D)=2/(2+2)=0.5000\) would assign probability \(0.25\) to two responses. The predictive calculation instead gives $$ P(K=2\mid D)=E(p^2\mid D)=\frac{2\cdot3}{4\cdot5}=0.3000. $$ The difference arises because the posterior is uncertain about \(p\); the beta-binomial distribution averages over that uncertainty rather than fixing \(p\) at its mean.
Turning Predictions Into Allocation Probabilities
For each possible pending response pattern, first update the posterior as though that pattern had been observed, then apply the chosen allocation rule. Finally, average the resulting allocation probabilities using the joint posterior predictive distribution. This order matters for nonlinear rules: applying a rule to averaged response rates need not equal averaging the rule’s outputs.
Use the observed successes and failures to form each arm’s current posterior. Keep pending outcomes marked as unresolved.
Calculate the predictive probability of each possible pending response pattern from the current posterior.
For each hypothetical completion, update the posterior and calculate the allocation probabilities that the specified rule would assign.
Weight each hypothetical allocation probability by its predictive probability. Use the resulting probabilities for the next assignment, without claiming that any hypothetical completion was observed.
Worked Example: Averaging an Allocation Rule
Two treatments, A and B, have current posteriors \(p_A\mid D\sim\operatorname{Beta}(4,2)\) and \(p_B\mid D\sim\operatorname{Beta}(3,3)\). Each arm has one pending response. For this example, define the allocation rule after any completion as the posterior mean response rate for A divided by the sum of the two posterior mean response rates. Calculate the predictive allocation probability for A.
The current posterior means are \(4/6=0.6667\) for A and \(3/6=0.5000\) for B. Thus the predictive probabilities of a pending response are \(2/3\) in A and \(1/2\) in B. Under the independent-arm model, the four completion probabilities and updated-rule probabilities are:
| Pending responses (A, B) | Predictive probability | Updated means (A, B) | Rule probability for A |
|---|---|---|---|
| (1, 1) | \((2/3)(1/2)=1/3\) | \((5/7,4/7)\) | \((5/7)/(9/7)=5/9\) |
| (1, 0) | \((2/3)(1/2)=1/3\) | \((5/7,3/7)\) | \((5/7)/(8/7)=5/8\) |
| (0, 1) | \((1/3)(1/2)=1/6\) | \((4/7,4/7)\) | \(1/2\) |
| (0, 0) | \((1/3)(1/2)=1/6\) | \((4/7,3/7)\) | \((4/7)/(7/7)=4/7\) |
Therefore the allocation probability is $$ q_A(D)=\frac{1}{3}\frac{5}{9}+\frac{1}{3}\frac{5}{8} +\frac{1}{6}\frac{1}{2}+\frac{1}{6}\frac{4}{7} =0.5721. $$ The probability for B is \(q_B(D)=1-0.5721=0.4279\). The next participant is randomized using these probabilities. The calculation does not assert that the pending responses were \(1,1\), or any other particular pair.
If instead the rule were applied directly to the current posterior means, it would give \(0.6667/(0.6667+0.5000)=0.5714\) for A. This is close but not identical. The difference reflects the nonlinearity of the ratio: the average of the rule across possible completions is not generally the rule applied to average posterior means.
What Predictive Averaging Can and Cannot Change
A useful check is that predictive averaging does not create new outcome information. For any arm, the expected posterior mean after pending responses resolve equals the current posterior mean. In the two-pending-outcome example, the possible updated means are \(1/3,1/2,2/3\), with probabilities \(0.3,0.4,0.3\), so $$ 0.3\left(\frac{1}{3}\right)+0.4\left(\frac{1}{2}\right)+0.3\left(\frac{2}{3}\right)=0.5000. $$ This is the current posterior mean. More generally, the identity follows from the law of iterated expectation: averaging a posterior expectation over data that have not yet been observed returns the current posterior expectation.
The same principle applies to posterior probabilities of events. If \(E\) is an event about the treatment parameters, such as \(p_A>p_B\), then $$ \sum_z P(Z=z\mid D)P(E\mid D,Z=z)=P(E\mid D). $$ This identity is a calibration check on a calculation that averages posterior probabilities over all possible pending outcomes. It does not mean every allocation rule is unchanged: nonlinear transformations, thresholds, caps, or other decision rules can make the averaged allocation probability differ from the rule applied to current summaries.
Worked Example: Checking a Predictive Posterior Mean
Suppose the current posterior for an arm is \(\operatorname{Beta}(2,2)\), and two outcomes are pending. The predictive probabilities from the earlier calculation are \(0.3000,0.4000,0.3000\) for \(K=0,1,2\). The posterior means after those respective counts are \(2/6=0.3333\), \(3/6=0.5000\), and \(4/6=0.6667\).
The predictive average is $$ 0.3000(0.3333)+0.4000(0.5000)+0.3000(0.6667)=0.5000. $$ It agrees with the current posterior mean \(2/(2+2)=0.5000\), up to rounding. Thus the pending responses widen the range of possible updated beliefs, but their unresolved status alone does not shift the mean belief.
Assumptions, Diagnostics, and Alternatives
The calculations above require a suitable response model and a suitable account of why outcomes are pending. In particular, the simple beta-binomial calculation assumes a common response probability within each arm, conditionally independent responses given that probability, and a specified beta prior. It also assumes that the fact an outcome remains unresolved supplies no additional information about its value after conditioning on the modeled data. If, for example, response status affects the speed of resolution, the set of pending outcomes may be informative and the delay process must be modeled rather than ignored.
Before using predictive allocation probabilities, check that the pending counts and treatment assignments are correct, the observed-data posterior parameters include only matured responses, and predictive probabilities sum to one. Compare the predictive response rates with plausible subject-matter ranges and examine whether the result is unduly sensitive to the prior. When a rule has caps, minimum allocation probabilities, or other constraints, apply those parts consistently to each hypothetical completion before averaging; otherwise the calculated probabilities may not correspond to the intended design.
Predictive updating differs from two common alternatives. Allocation using only matured outcomes avoids forecasting pending responses, but may discard useful model-based information about what unresolved outcomes could imply. A plug-in approach replaces uncertain response probabilities with a point estimate and can understate predictive uncertainty. Predictive averaging retains that uncertainty, but depends on the model and prior. It is not automatically superior: if the response model or delay assumption is poor, the resulting probabilities can be misleading.
A further distinction is from treating a single imputed completion as the truth. Predictive averaging considers all possible completions with their probabilities; it does not select one forecast and then proceed as if it had been observed. Nor does it provide an excuse to update a participant’s response record before the outcome is available. The next allocation is randomized from the calculated probabilities, while the pending outcomes remain pending in the trial record.
Check Your Understanding
Answer each question, then compare with the solutions below.
- With \(p\mid D\sim\operatorname{Beta}(3,2)\), one response is pending. What is the predictive probability that it is a response?
- For \(p\mid D\sim\operatorname{Beta}(2,2)\) and two pending outcomes, why is the predictive probability of two responses not obtained by simply squaring the posterior mean?
- In one sentence, distinguish averaging an allocation rule over pending outcomes from recording a predicted outcome as observed.
- Under what delay condition is the simple beta-binomial predictive calculation inadequate?
Solutions. 1. The predictive response probability is the posterior mean, \(3/(3+2)=0.6000\). 2. The correct probability is \(E(p^2\mid D)=2\cdot3/(4\cdot5)=0.3000\), whereas the squared posterior mean is \(0.25\); integrating over uncertainty in \(p\) induces dependence between the pending responses. 3. Averaging weights each possible completion’s allocation probability by its predictive probability, whereas recording a forecast as observed falsely adds outcome evidence. 4. It is inadequate when resolution status is informative about response after conditioning on the modeled data, such as when response status changes the time to resolution; that delay mechanism must be represented in the model.