ODE Limits for Allocation Proportions

Why an ODE describes allocation proportions

An adaptive allocation rule changes the probability of assigning the next participant as data accumulate. The resulting allocation proportions are random, but each new assignment changes a proportion by only about the reciprocal of the current sample size. On the right time scale, these small updates can be approximated by a deterministic differential equation. That approximation is useful because it turns a complicated random path into a trajectory whose direction and possible stationary points can be examined.

The ODE is not automatically a proof that the observed proportions converge to a particular target. Its role is more precise: under suitable regularity and noise conditions, the stochastic path follows ODE trajectories over bounded intervals of an algorithmic time scale. Conclusions about a specific limit require additional arguments.

Here \(q(x)\) is the vector of next-assignment probabilities, so its entries are nonnegative and sum to one. It is not the same object as the current allocation proportions \(x\). The difference \(q(x)-x\) is the mean direction in which those proportions change. The simplex is a natural state space: a valid trajectory should retain nonnegative components summing to one.

From the discrete update to fluid time

Let \(I_{n+1}\) be the \(K\)-vector that has a one in the arm receiving participant \(n+1\) and zeros elsewhere. The proportions satisfy the exact identity

If the conditional assignment probabilities are \(q_n\), then \(E(I_{n+1}\mid\mathcal{F}_n)=q_n\), where \(\mathcal{F}_n\) records the information available before the next assignment. Write \(q_n=q(x_n)+r_{n+1}\), where \(r_{n+1}\) represents the discrepancy between the actual probabilities and the limiting probability map. Define the martingale difference \(M_{n+1}=I_{n+1}-q_n\), which has conditional mean zero. Substitution gives

The increments \(a_{n+1}\) shrink, but their cumulative sum diverges. The appropriate continuous time is therefore \(t_n=\sum_{j=1}^{n}1/j\), which grows approximately like \(\log n\), rather than ordinary sample size \(n\). An interval of fixed length on this time scale spans a multiplicative change in sample size: increasing \(t\) by about one corresponds roughly to multiplying \(n\) by \(e\).

The compactness condition is automatic for allocation proportions because they lie in the simplex. The Lipschitz condition ensures that nearby starting points generate nearby ODE solutions over finite intervals. Bounded assignment indicators give bounded noise; more generally, the stated conditional second-moment bound suffices for the weighted martingale noise to become negligible over late bounded fluid-time windows. Vanishing perturbations are important when assignment probabilities depend on estimated response quantities: the limiting map \(q(x)\) must accurately describe the conditional probabilities in the long run.

Why the tracking approximation is justified

The justification has two parts. First, over a bounded fluid-time interval, the accumulated random noise becomes small. Second, the deterministic part of the recursion is an Euler approximation to the ODE. To see the second point, freeze the state near some \(x\) for a short time increment \(a_{n+1}\). The ODE changes by approximately \(a_{n+1}h(x)\), matching the recursion’s drift term. Lipschitz continuity controls the error from the state changing during that increment.

For the noise, the weighted partial sums are sums of \(a_{n+1}M_{n+1}\). The martingale-difference property removes cross terms in their second moments, and bounded conditional second moments give a tail variance bounded by a constant times \(\sum a_{n+1}^2\). Since that series converges, the late noise accumulated over a bounded fluid-time window vanishes. If \(r_n\to0\), its weighted sum over such a window also vanishes: the total step size in the window is bounded, while the largest perturbation in a sufficiently late window is small. These facts, together with the Lipschitz bound, yield the finite-window tracking statement.

One immediate implication is that if the allocation proportions themselves converge to a point \(x^*\), then under these conditions \(h(x^*)=0\). If some coordinate \(h_i(x^*)\) were nonzero, the recursion would keep changing that coordinate in the same direction over a late fluid-time interval, by an amount approaching the interval length times \(h_i(x^*)\). That is incompatible with convergence to \(x^*\). Thus a convergent allocation proportion must be an equilibrium, meaning \(q(x^*)=x^*\). This necessary condition does not show that a particular equilibrium is reached.

Reading the ODE without overclaiming

The main alternatives answer different questions. Direct simulation of the adaptive design gives finite-sample paths and can reveal variability, but does not by itself establish a limiting behavior. Solving the equilibrium equation \(q(x)=x\) identifies stationary points, but does not describe how the system moves between them. The ODE provides that directional, large-sample-scale description. It still does not prove that the stochastic process converges to a particular equilibrium; tracking only says that late segments resemble ODE segments.

Several checks help determine whether the approximation is being used appropriately:

• Check the drift. The assignment-probability vector is \(q(x)\); the proportion drift is \(q(x)-x\). Omitting the \(-x\) term models assignment probabilities, not changes in allocation proportions.
• Check the state space. The map \(q\) should be a valid probability vector, and the ODE should preserve the simplex. If a proposed approximation leaves the simplex, revisit the drift or boundary assumptions.
• Check perturbations. If response estimates or target estimates do not settle into the proposed map, \(r_n\) may not vanish. A deterministic ODE with the wrong drift may then give a misleading path.
• Check time units. The ODE time is approximately logarithmic in sample size. Treating \(t=1\) as one participant is a scale error.

A common mistake is to find \(q(x^*)=x^*\) and conclude that the adaptive design must converge to \(x^*\). The equation only says that \(x^*\) is stationary for the ODE. Trajectory tracking does not rule out other equilibria or more complicated limiting behavior, nor does the equilibrium equation establish that the process enters the relevant region. Showing that a particular target attracts the stochastic process requires a separate stability argument.

The fluid approximation also has limits. It is a large-sample description, not a finite-sample confidence statement, and it does not quantify the random fluctuations around a trajectory. It can be unreliable when the probability map is discontinuous, the noise is not controlled, or the estimation perturbation persists. Near boundaries, one must verify that the rule remains valid and that the limiting ODE respects the simplex; a formal algebraic solution outside the simplex is not a possible allocation proportion.