Expected Arm Counts Over a Fixed Trial Horizon

Why Expected Counts Need History Averaging

In an adaptive trial, the chance of assigning the next participant to a treatment can change as assignments and responses accumulate. Consequently, the expected number assigned to an arm over the whole trial is not generally the number obtained by multiplying the horizon by an allocation probability observed at one particular point. The probability at each assignment must be averaged over the histories that could have occurred by then.

This calculation answers a precise planning question: under the specified allocation policy and response process, how many participants are expected to be assigned to each arm by a fixed stopping time $T$? It does not say how many participants a particular trial will actually assign to that arm. The realized count is random.

Definition: Consider $K$ treatment arms and a fixed horizon of $T$ assignments. Let $A_t$ denote the arm assigned at time $t$, and let $H_{t-1}$ denote the information available to the allocation policy immediately before that assignment, including the previous assignments and observed responses it uses. Define $$ p_{t,k}(H_{t-1})=P(A_t=k\mid H_{t-1}) $$ as the history-conditional probability of assigning participant $t$ to arm $k$. The arm count is $N_k(T)=\sum_{t=1}^{T} I(A_t=k)$, where $I(\cdot)$ is 1 when its condition is true and 0 otherwise.

The allocation probabilities satisfy $0\leq p_{t,k}(H_{t-1})\leq 1$, and $\sum_{k=1}^{K}p_{t,k}(H_{t-1})=1$ for each history at which an assignment is made. For numerical calculation, the policy alone is not enough: the response model, or another specification of how histories arise, is also needed to average over histories. Expectations below are taken under the joint probability law of responses and assignments induced by those specifications.

The Expected-Count Identity

The count for arm $k$ is the sum of the assignment indicators. Each indicator has conditional expectation $p_{t,k}(H_{t-1})$, since conditional on the history it is a Bernoulli indicator with success probability equal to the allocation probability. Taking expectations and using linearity gives the finite-horizon identity.

\begin{aligned} E[N_k(T)] &=E\left[\sum_{t=1}^{T}I(A_t=k)\right]\\ &=\sum_{t=1}^{T}E[I(A_t=k)]\\ &=\sum_{t=1}^{T}E\left[E\{I(A_t=k)\mid H_{t-1}\}\right]\\ &=\sum_{t=1}^{T}E[p_{t,k}(H_{t-1})]. \end{aligned}

The third line uses the law of iterated expectation; the last uses $E\{I(A_t=k)\mid H_{t-1}\}=P(A_t=k\mid H_{t-1})$. This is a short but important justification: the identity does not assume that assignments at different times are independent. They usually are not, because earlier assignments and responses can affect later probabilities.

Theorem: For any fixed finite horizon $T$, if each assignment at time $t$ follows the stated conditional allocation probabilities, then the expected count for arm $k$ is the sum over times of the probabilities averaged over the histories possible at those times: $$ E[N_k(T)]=\sum_{t=1}^{T}\sum_{h}P(H_{t-1}=h)\,p_{t,k}(h). $$ Here the inner sum ranges over possible histories with positive probability. If histories have a continuous component, the sum is replaced by an expectation or integral over the history distribution.

An adaptive probability along one realized path is not itself an unconditional expected probability. To average at time $t$, weight each history-specific probability by the probability of reaching that history. In finite examples, enumeration makes this weighting explicit. For larger state spaces, the same calculation can be implemented by recursion or by simulating complete trial paths and averaging the resulting counts.

Worked Calculations

Worked Example: Two Assignments With One Response-Driven Update

Suppose a hypothetical two-arm trial has $T=2$. The first participant is assigned to arm A with probability 0.40 and arm B with probability 0.60. A binary response is then observed. The assumed response probabilities are $P(\text{favorable}\mid A)=0.70$ and $P(\text{favorable}\mid B)=0.20$. For the second assignment, the policy sets $p_{2,A}=0.80$ after a favorable response and $p_{2,A}=0.30$ after an unfavorable response. Find the expected count for each arm.

The probability of a favorable response across the first participant's possible assignments is $$ P(\text{favorable})=(0.40)(0.70)+(0.60)(0.20)=0.28+0.12=0.40. $$ Thus the probability of an unfavorable response is $1-0.40=0.60$. Averaging the second-assignment probability over these two histories gives $$ E[p_{2,A}(H_1)]=(0.40)(0.80)+(0.60)(0.30)=0.32+0.18=0.50. $$ The first participant's probability of assignment to A is 0.40, so $$ E[N_A(2)]=0.40+0.50=0.90\text{ participants}. $$ Since every assignment goes to exactly one of the two arms, $E[N_B(2)]=2-0.90=1.10$ participants. Equivalently, arm B has expected assignment probabilities $0.60$ at time 1 and $0.50$ at time 2. The expected counts sum to the fixed horizon, $0.90+1.10=2$.

Worked Example: A Policy That Uses the First Response for Both Later Assignments

In a hypothetical three-participant trial, the first participant is assigned to arm A with certainty. The first response is favorable with probability 0.60. Conditional on a favorable response, each of assignments 2 and 3 has probability 0.80 of going to A; conditional on an unfavorable response, each has probability 0.30 of going to A. Assume the policy ignores the second participant's response for these two allocation decisions. Calculate both expected arm counts.

The first assignment contributes exactly 1 to $N_A(3)$. The probability of assignment to A at time 2, averaged over the first response, is $$ E[p_{2,A}(H_1)]=(0.60)(0.80)+(0.40)(0.30)=0.48+0.12=0.60. $$ The same averaging applies at time 3 because its allocation probability depends on the same first response and not on the second response: $$ E[p_{3,A}(H_2)]=(0.60)(0.80)+(0.40)(0.30)=0.60. $$ Therefore, $$ E[N_A(3)]=1+0.60+0.60=2.20\text{ participants}. $$ The expected count for B is $E[N_B(3)]=3-2.20=0.80$ participants. These expectations do not require the time-2 and time-3 assignments to be independent. In fact, their probabilities are linked through the shared first-response history. Linearity of expectation is sufficient for the count calculation.

Worked Example: Averaging Thompson-Sampling Probabilities Across Histories

Consider a hypothetical two-arm Thompson-sampling trial with $T=2$. At the first assignment, the recorded probability of assigning to A is 0.50. Suppose the response model gives $P(\text{favorable}\mid A)=0.60$ and $P(\text{favorable}\mid B)=0.30$. After the first assignment and response, suppose the recorded Thompson-sampling probability of A at time 2 is 0.75 after A/favorable, 0.55 after A/unfavorable, 0.65 after B/favorable, and 0.35 after B/unfavorable. These are hypothetical history-specific policy probabilities. Calculate the expected counts.

The four histories before assignment 2 have probabilities $$ \begin{aligned} P(A,\text{favorable})&=(0.50)(0.60)=0.30,\\ P(A,\text{unfavorable})&=(0.50)(0.40)=0.20,\\ P(B,\text{favorable})&=(0.50)(0.30)=0.15,\\ P(B,\text{unfavorable})&=(0.50)(0.70)=0.35. \end{aligned} $$ They sum to 1.00, as required. The average probability of assigning to A at time 2 is $$ \begin{aligned} E[p_{2,A}(H_1)] &=(0.30)(0.75)+(0.20)(0.55)+(0.15)(0.65)+(0.35)(0.35)\\ &=0.225+0.110+0.0975+0.1225=0.555. \end{aligned} $$ The first assignment contributes probability 0.50 to A, so $E[N_A(2)]=0.50+0.555=1.055$ participants. The expected count for B is $2-1.055=0.945$ participants. The recorded probabilities at time 2 must be averaged over the probability of each history; averaging the four probabilities equally would give the wrong answer because the histories are not equally likely.

Practical Computation and Checks

For a small trial, a history tree is usually the clearest exact calculation. At each time, list the histories that can occur, compute their probabilities under the response model and assignment policy, and multiply each history probability by its next allocation probability. Add those weighted probabilities over histories and then over times. The following sequence makes the bookkeeping explicit.

Specify the horizon and policy.
Record $T$, the arms, and $p_{t,k}(h)$ for every relevant time and history.

Specify how histories arise.
Give the response probabilities needed to calculate the chance of each next history under the trial's assumed model.

Average at each time.
For arm $k$, calculate $\sum_h P(H_{t-1}=h)p_{t,k}(h)$ for each $t$.

Sum over the horizon and check.
Add the time-specific averages. Across all arms, the expected counts must sum to $T$.

The final check follows directly from the fact that exactly one arm is assigned at each time: $$ \sum_{k=1}^{K}E[N_k(T)] =\sum_{t=1}^{T}E\left[\sum_{k=1}^{K}p_{t,k}(H_{t-1})\right] =\sum_{t=1}^{T}1=T. $$ If the total differs from $T$, likely causes include an omitted history, probabilities that do not sum to one, or an arithmetic error.

Interpretation, Diagnostics, and Limitations

Expected counts are useful summaries of the allocation policy under its assumed response model. They can compare the policy's average allocation across arms over the specified horizon, and they make explicit how response uncertainty affects allocation. However, an expected count is not a promised or typical count for every realized trial. For example, $E[N_A(T)]=2.20$ does not mean that any trial assigns 2.20 participants to A; realized counts are integers.

A main alternative is to report counts from one simulated or observed trial path. Such counts describe that path, not the expectation over all possible histories. Repeated simulation can estimate expected counts by generating many complete paths and averaging $N_k(T)$, but finite simulation introduces Monte Carlo error; it is not the exact history average unless the state space is exhaustively enumerated. A fixed, nonadaptive allocation is a simpler special case: if $p_{t,k}$ is constant over time and history, then $E[N_k(T)]=T p_k$. In an adaptive design, replacing all history-specific probabilities with one current probability or a probability evaluated at an “average” history generally has no justification.

Expected counts also do not describe the spread or dependence of the realized arm counts. To quantify that uncertainty, one needs the distribution of $N_k(T)$, or at least its variance; that calculation depends on how assignment indicators co-vary across times. The sum-of-expectations identity alone does not make those indicators independent. Results also depend on the assumed response model: if the response probabilities used to weight histories are misspecified, the resulting expected counts need not describe the operating trial.

Diagnostic: Check that the history probabilities at each time sum to 1 and that the allocation probabilities across arms sum to 1 within each history.
Diagnostic: Confirm that each history probability includes the earlier assignments and responses that lead to that history, rather than only the response probability at the final step.
Failure mode: Treating the observed path's allocation probabilities as though they represented every possible path ignores uncertainty in the histories.
Failure mode: An expected count outside $[0,T]$, or arm expectations that do not sum to $T$, signals a calculation or specification problem.

Key takeaway: For a fixed horizon, the expected count for an arm is the sum of its allocation probabilities, each averaged over the histories that could precede that assignment. The expectation averages over paths; it does not predict the exact count on any one path.

Check Your Understanding

Answer each question, then check the solutions below.

Why can’t a history-specific probability at time $t$ generally be used directly as the unconditional expected allocation probability at time $t$?
A three-assignment trial has probabilities of assignment to A equal to 0.20 at time 1, 0.70 after a favorable first response at time 2, and 0.40 after an unfavorable first response at time 2. The favorable response probability is 0.50. Find $E[N_A(2)]$.
If expected counts for three arms are 1.4, 2.1, and 1.5 over a fixed horizon, what is the horizon? Does this imply any realized trial has those exact counts?
State one reason that knowing expected arm counts does not determine the variability of the realized counts.

Solutions. (1) It applies only conditional on a particular history. The unconditional probability must average the history-specific probability using the probability of reaching each history. (2) The time-2 average is $(0.50)(0.70)+(0.50)(0.40)=0.55$. Thus $E[N_A(2)]=0.20+0.55=0.75$ participants. (3) The horizon is $1.4+2.1+1.5=5$ assignments, since expected counts across arms sum to $T$. No: realized counts are integer-valued, while expected counts need not be integers. (4) The expected-count identity uses only linearity of expectation; variability also depends on the joint behavior, including dependence, of assignments across times.