Why the Conditioning Matters
An adaptive trial does not follow a single allocation sequence. Its assignments respond to accumulating information, so repeated trials under the same design can produce different allocation histories and different outcomes. A performance summary can describe what happens given a particular history, or it can average over the histories the design can generate. These are distinct questions, and neither summary substitutes for the other.
For example, a conditional success probability can describe the chance of meeting a prespecified trial criterion among trials with a particular allocation history. An unconditional success probability describes the chance of meeting that criterion across all trials run under the policy. The first can help explain how performance varies across paths; the second describes the design's average performance in the stated scenario.
The calculation must respect the adaptive data-generating process. An observed path can be associated with earlier outcomes because those outcomes influenced subsequent assignments. Conditioning on the path therefore does not mean that the outcomes are distributed as if the assignments had been fixed in advance.
Definitions and Assumptions
Fix a trial scenario: this specifies the outcome-generating model, the trial procedures, the adaptive allocation policy, and the analysis or decision rule being evaluated. Consider repeated trials run under that same scenario and policy. Let \(H\) denote the allocation history for one trial, such as the sequence of treatment assignments. Let \(T\) be a performance quantity computed from the trial data, or let \(D\) be an event indicating that a prespecified criterion is met.
The conditioning event must be defined precisely. In this tutorial, \(H\) records assignments, not the full sequence of outcomes. The conditional distribution of outcomes given \(H=h\) is the distribution implied by the joint adaptive process. In particular, it need not equal the outcome distribution obtained by simply fixing the assignments to \(h\) and ignoring how that path arose.
The relevant assumptions are those of the scenario being evaluated. The outcome model and allocation policy must be specified, including how assignments depend on previously observed data and any randomization used by the policy. A common setup assumes consistency of observed outcomes with assigned treatments and no interference between participants; in other words, one participant's assignment does not change another participant's outcome. These assumptions make the repeated-trial probability distribution well-defined, but they do not make adaptive assignments independent of previous responses. Every conditional summary also requires a path with positive probability.
From Path-Specific to Average Performance
Suppose the possible allocation histories are \(h_1,\ldots,h_K\). The histories are mutually exclusive and exhaustive, and each has probability \(P(H=h_k)\) under the policy and scenario. For any event \(D\), partitioning on the history gives the law of total probability:
Thus the unconditional event probability is a probability-weighted average, not an unweighted average of the path-specific probabilities. More generally, for a numerical quantity \(T\), the law of total expectation gives \(E(T)=\sum_k E(T\mid H=h_k)P(H=h_k)\).
For variability, the corresponding decomposition is:
The first term is average within-path variability; the second is variability between the conditional means across paths. To justify the identity, write \(T-E(T)=[T-E(T\mid H)]+[E(T\mid H)-E(T)]\), square, and take expectations. The cross-product has expectation zero because \(E[T-E(T\mid H)\mid H]=0\). The squared first term has expectation \(E\{\operatorname{Var}(T\mid H)\}\), and the squared second term has expectation \(\operatorname{Var}\{E(T\mid H)\}\). This proves the decomposition.
Worked Example: Averaging a Path-Specific Success Probability
Suppose a hypothetical adaptive trial has three possible allocation-history categories. Under its specified scenario, their probabilities are 0.20, 0.50, and 0.30. The conditional probabilities of meeting a prespecified trial success criterion are 0.04, 0.06, and 0.12, respectively. Find the unconditional probability of meeting the criterion.
The history probabilities sum to one: \(0.20+0.50+0.30=1.00\). Apply the law of total probability, weighting each conditional probability by the probability of its history:
The unconditional success probability is 0.074, or 7.4%, under this hypothetical scenario and policy. It is not the simple average \((0.04+0.06+0.12)/3=0.0733\), because the three histories are not equally likely. The conditional values remain informative: they show that performance differs across path categories. They do not, by themselves, state the design's average success probability.
Conditional Bias and Mean Squared Error
For a numerical estimator \(T\) of a fixed target \(\theta\), conditional summaries can reveal how estimation behaves across allocation histories. Define the conditional error mean \(b_h=E(T-\theta\mid H=h)\) and conditional variance \(v_h=\operatorname{Var}(T\mid H=h)\). Then \(E(T-\theta)=\sum_h b_hP(H=h)\). The unconditional mean squared error follows by applying the variance decomposition to the error \(T-\theta\):
This calculation separates within-path variation from squared conditional error. It also prevents a common interpretive shortcut: a conditional error mean is not automatically the estimator's unconditional bias. The latter averages conditional error means using the probabilities of the histories.
Worked Example: Combining Conditional Error Summaries
In a hypothetical study, let \(\theta=0\), and suppose an estimator \(T\) has three possible allocation-history categories. Their probabilities are 0.25, 0.50, and 0.25. The conditional error means \(E(T-\theta\mid H=h)\) are 0.20, \(-0.10\), and 0.40; the corresponding conditional variances are 0.09, 0.04, and 0.16. Calculate the unconditional bias and mean squared error.
First average the conditional error means using the history probabilities:
Thus the unconditional bias is 0.10 in the units of \(T\). For the mean squared error, add each conditional variance to the squared conditional error mean, then average:
The unconditional mean squared error is 0.1375 in squared units of \(T\). Reporting only the conditional mean squared error for one history would not give this average. Nor would averaging the conditional variances alone, because that omits the squared conditional error means.
Why an Allocation Path Can Be Informative
In response-adaptive randomization, assignment probabilities at a later time can depend on earlier observed responses. Consequently, the realized path can reveal something about those responses. This is a feature of the design's joint probability structure, not a reason to treat assignment as a response measurement.
Worked Example: Conditioning Selects Different Earlier Responses
Consider a hypothetical two-participant process. The first participant is assigned to treatment A, and their binary response \(R\) equals 1 with probability 0.50. The second participant is assigned to treatment B with probability 0.80 if \(R=1\), and with probability 0.20 if \(R=0\); otherwise the second participant receives A. Let \(H=B\) denote the path where the second participant receives B. Find \(P(H=B)\), and find the probability that the first response was 1 conditional on each possible second assignment.
By conditioning on the first response, the probability of the B path is:
Bayes' rule now gives:
The probability of the A path is \(P(H=A)=1-0.50=0.50\). Therefore:
Although the marginal probability of \(R=1\) is 0.50, it is 0.80 among trials on the B path and 0.20 among trials on the A path. The path-specific outcome distributions are different because the assignment policy uses \(R\). This is why a conditional operating characteristic must be evaluated under the joint adaptive process rather than by treating the realized assignments as unrelated to earlier outcomes.
Comparisons, Diagnostics, and Limitations
Conditional and unconditional summaries answer different questions. Comparing them with other design quantities helps prevent misinterpretation.
| Quantity | What it conditions on or averages over | Question it answers |
|---|---|---|
| Conditional operating characteristic | Trials with a specified allocation history | How does performance behave on this path? |
| Unconditional operating characteristic | All histories, weighted by their policy probabilities | What is average performance under this design? |
| Assignment probability | Past observed information at a particular assignment time | How likely is a treatment assignment at that point? |
| Nonadaptive design summary | A different policy, often with assignments set independently of interim responses | How does performance compare under a different design? |
An assignment probability such as \(P(A_t=a\mid\text{past data})\) is a rule used to generate the next assignment. It is not itself the conditional probability of a trial-level success event given the completed allocation history. Similarly, a nonadaptive comparison changes the policy and thus generally changes the distribution over histories; its results should not be substituted for either conditional or unconditional performance under the adaptive policy.
For simulation studies, a direct estimate of unconditional performance comes from simulating complete trials under the specified policy and scenario, then averaging the performance indicator or quantity over those trials. Conditional estimates can be obtained by grouping simulated trials by history, but the number of trials in each group matters. Rare histories yield imprecise conditional estimates, even when the overall simulation contains many trials. Report the number of simulated trials contributing to each conditional summary, and use histories or categories defined clearly enough to interpret.
A key diagnostic is whether the simulated allocation frequencies agree with the policy's implied path probabilities in a setting where those probabilities can be calculated. Also check that every conditioning history has positive probability, that the history definition is fixed before interpreting the summaries, and that the simulation preserves the policy's sequential response feedback. A calculation that fixes a path and generates outcomes as if the path had been assigned independently can give the wrong conditional distribution.
Conditional summaries have limitations. A single observed trial supplies only one realized history, so its path-specific performance does not reveal what would have happened on other paths. Some paths may be too rare for stable conditional estimates, and coarsening many distinct paths into categories can hide meaningful differences. Unconditional summaries avoid the need to interpret every path, but can conceal poor performance on particular paths. Reporting both can be useful when the scientific question calls for average performance and when path-specific variation is consequential.
Common mistake: “The conditional success probabilities are 4%, 6%, and 12%, so unconditional success is their arithmetic mean, 7.33%.” This is wrong unless the three histories are equally likely. The correction is to weight each conditional probability by its history probability; in the example, the result is 7.4%. A related error is to condition on the observed assignment sequence while assuming earlier outcomes retain their marginal distribution. Under response adaptation, that assumption can fail, as the two-participant calculation demonstrates.
Check Your Understanding
Answer each question using the stated hypothetical values. Full solutions follow the questions.
- Two path categories have probabilities 0.70 and 0.30, with conditional probabilities of a criterion being met equal to 0.05 and 0.15. What is the unconditional probability?
- An estimator has conditional error means 0.30 and \(-0.10\) on two paths, with path probabilities 0.25 and 0.75. What is its unconditional bias?
- For the estimator in Question 2, conditional variances on the two paths are 0.04 and 0.09. Find its unconditional mean squared error.
- Explain why, in a response-adaptive trial, conditioning on the completed allocation path can change the distribution of earlier responses.
Solutions. (1) Weight by the path probabilities: \((0.70)(0.05)+(0.30)(0.15)=0.035+0.045=0.080\), or 8.0%. (2) The unconditional bias is \((0.25)(0.30)+(0.75)(-0.10)=0.075-0.075=0\). (3) Add conditional variance and squared conditional error mean within each path, then average: \((0.25)(0.04+0.30^2)+(0.75)(0.09+(-0.10)^2)=(0.25)(0.13)+(0.75)(0.10)=0.0325+0.075=0.1075\), in squared units of the estimator. (4) Later assignments may depend on earlier responses, so the allocation path carries information about those responses. The conditional response distribution must be derived from the joint adaptive process rather than assumed equal to its marginal distribution.