Sensitivity to Prior Location

Why Prior Location Matters

In response-adaptive randomization, observed outcomes influence the probabilities of assigning future participants to each treatment. In Bayesian or bandit-style designs, prior distributions affect the early estimates that drive those probabilities. The prior's location can therefore matter even when its uncertainty is unchanged: an optimistic prior places more mass on higher response rates, a skeptical prior places more mass on lower rates, and a treatment-differential prior initially favors one arm over another.

The aim here is to isolate that location effect. The prior concentration will be held fixed across comparisons, so a change in the results is not being attributed to a change in prior uncertainty. This is a controlled sensitivity analysis, not a claim that one location is universally appropriate. The consequences depend on the true response rates, the adaptation rule, and how much data accrue before allocation becomes strongly adaptive.

The beta-binomial model assumes that, within each arm, responses are Bernoulli with a constant probability \(\theta_j\), conditional on \(\theta_j\), and that participant responses are conditionally independent. For the examples, outcomes are available before the next assignment, treatment effects are stable during enrollment, and assignment probabilities follow the specified rule. These assumptions make the updating calculation transparent; departures from them can make the resulting operating characteristics inapplicable.

For a concrete comparison, consider Thompson sampling: after each observed response, draw one value from each arm's posterior distribution and assign the next participant to the arm with the larger draw. With independent continuous posteriors, the probability of assigning arm A is \(P(\theta_A>\theta_B\mid\text{data})\). The examples compare priors for this rule, not every possible adaptive algorithm. Initial allocation, stopping, and any allocation safeguards must also be specified in an actual design.

Constructing Comparable Prior Scenarios

Take two treatment arms, A and B, and set the concentration to \(\kappa=20\) for each arm in every scenario. A symmetric optimistic prior uses a higher prior mean on both arms; a symmetric skeptical prior uses a lower mean on both arms. A treatment-differential prior instead gives A a higher prior mean than B. A neutral symmetric prior is useful as a reference.

Each prior in the table has \(a_j+b_j=20\). The optimistic and skeptical scenarios shift both arms together, while the differential scenario changes the prior contrast between arms. These are distinct perturbations: a global shift concerns the assumed level of response, whereas a differential shift concerns the assumed relative advantage of a treatment.

Posterior Updating and the Effect of Location

If \(x_j\) responses are observed among \(n_j\) participants on arm \(j\), beta-binomial conjugacy gives the posterior \(\mathrm{Beta}(a_j+x_j,b_j+n_j-x_j)\). Its mean can be expressed using the prior mean and the observed response proportion:

This is a weighted average of the prior mean and the observed response proportion, with weights proportional to prior concentration and sample size. For the same \(n_j\) and \(x_j\), changing the prior mean from \(m_1\) to \(m_0\), while keeping \(\kappa_j\) fixed, changes the posterior mean by

The difference shrinks as \(n_j\) increases, but it need not be negligible early in a trial. Importantly, Thompson sampling responds to the full posterior distributions, not just their means. A prior-location change can affect both the posterior mean and the probability that one arm has the larger response probability.

Comparing Operating Characteristics

A prior sensitivity analysis should compare repeated-trial behavior under prespecified data-generating conditions. Each simulated trial must apply the same enrollment horizon, response model, randomization rule, safeguards, and analysis across prior scenarios. Only the prior locations should change. Useful outcomes include the fraction assigned to each arm and, under a fixed analysis, the probability of rejecting \(H_0\) under specified null and alternative response rates. These answer different questions: allocation fractions describe the design's assignments; rejection rates describe the behavior of the design-analysis pair.

For simulation, each trial produces one allocation fraction for A, such as \(F_A=N_A/N\), where \(N_A\) is the number assigned to A and \(N\) is total enrollment. Across \(R\) independently simulated trials, estimate the mean allocation fraction by \(\bar{F}_A\). Its Monte Carlo standard error is \(s_F/\sqrt{R}\), where \(s_F\) is the sample standard deviation of the \(R\) trial-level fractions. It is not generally valid to treat the \(N\) assignments within an adaptive trial as independent Bernoulli observations: their probabilities depend on prior outcomes and assignments.

When comparing scenarios using shared random numbers, calculate each trial's paired difference \(D_r=F_{A,r}^{(1)}-F_{A,r}^{(0)}\). The estimated difference is \(\bar{D}\), with Monte Carlo standard error \(s_D/\sqrt{R}\). Pairing can reduce simulation noise, but the standard error must use the variability of the trial-level differences, not the variability of individual assignments.

Justification and Practical Diagnostics

The posterior-mean sensitivity formula follows directly by subtracting the two posterior means for prior locations \(m_1\) and \(m_0\), with common concentration \(\kappa\), sample size \(n\), and response count \(x\):

This establishes a simple diagnostic expectation: a fixed prior-location perturbation has its greatest direct influence when few outcomes have accrued. It does not imply that allocation differences vanish at a predictable rate, because the adaptive path also changes which arm receives participants and therefore which outcomes are observed.

Before simulation, check that the prior means are clinically interpretable and that each scenario has the intended concentration. Then inspect allocation trajectories over enrollment, not only final averages. Large early divergence, persistent arm imbalance under a symmetric prior, or allocations that remain highly sensitive after substantial data accrue can indicate prior domination, implementation errors, or an overly aggressive adaptation rule. Check that the treatment labels and prior assignments are correct; swapping A and B in a differential prior should swap the allocation behavior when the rest of the design is symmetric.

Also distinguish Monte Carlo uncertainty from trial-to-trial variability. A large \(s_F\) describes variation among possible trial outcomes; \(s_F/\sqrt{R}\) describes uncertainty in the simulated mean. Neither measures uncertainty about whether the assumed response model or prior scenarios are scientifically appropriate.

What the Comparison Does and Does Not Establish

A symmetric optimistic or skeptical prior can shift posterior response levels without initially favoring either treatment. If the allocation rule depends only on which posterior draw is larger, equal shifts do not guarantee unchanged allocation: posterior spread and the nonlinear comparison probability also matter. A treatment-differential prior can create an initial preference for one arm and may change allocation even before outcomes provide evidence. Under a true treatment difference, this could align with the better arm or direct participants toward the worse arm; under equal true response rates, an unbalanced prior can still generate unequal allocation.

The main alternatives clarify what is being tested. A neutral symmetric prior provides a reference but is not automatically correct. Fixed randomization removes dependence of future assignments on observed responses, but answers a different design question. A prior sensitivity analysis can also examine several plausible locations rather than selecting one favored value; each comparison should still keep concentration and all non-prior design features fixed. None of these comparisons alone establishes which prior is best.

Limitations remain. Results are conditional on the response model, follow-up assumptions, enrollment horizon, adaptation rule, safeguards, and analysis. A prior location that is plausible for one population may be misleading in another. Conclusions based on one alternative response-rate configuration may not hold under the null or under weaker or stronger treatment effects. Assessing those other dimensions requires separate, prespecified scenarios.