Consistency of Targeted Allocation Proportions

What Consistency of an Allocation Target Means

An adaptive design may update its allocation probabilities in response to accumulated outcomes, while the realized allocation proportions also reflect random assignment. A target is consistent if the realized proportions approach the prescribed allocation vector as the number of participants grows. This is a statement about the long-run allocation path; it does not by itself describe the size or distribution of deviations at a finite sample size.

The result below applies to an interior target: every component of the target is strictly positive. It uses the stochastic-approximation representation of an allocation recursion and a Lyapunov function to show that stable drift overcomes accumulated random assignment noise. The conclusion is almost-sure convergence under explicit step-size and noise conditions.

Definition: For \(K\) treatment arms, let \(q_n=(q_{n,1},\ldots,q_{n,K})\) be the vector of allocation proportions after \(n\) assignments. The probability simplex is \(\Delta_K=\{q\in\mathbb{R}^K:q_j\geq 0,\ \sum_{j=1}^K q_j=1\}\). A prescribed target \(\rho\) is interior if \(\rho\in\Delta_K\) and \(\rho_j>0\) for every arm \(j\). The allocation proportions are consistent for \(\rho\) if \(q_n\to\rho\); almost-sure consistency means \(P(\lim_{n\to\infty}q_n=\rho)=1\).

Recursion and Sufficient Conditions

Write the allocation update in the form

q_{n+1}=q_n+\gamma_{n+1}\{h(q_n)+M_{n+1}+r_{n+1}\}.

Here \(h(q_n)\) is the mean drift of the allocation rule, \(M_{n+1}\) is a mean-zero random innovation, and \(r_{n+1}\) is a remainder, such as a small error from estimating a target. The step size \(\gamma_{n+1}\) controls how much one update changes the proportions. Let \(\mathcal F_n\) represent the information available after assignment \(n\).

The following are convenient sufficient conditions. They are stronger than necessary in some applications, but make the roles of drift, noise, and step size transparent.

The proportions remain in a compact subset of the domain of \(h\), and the recursion is well-defined there. The target \(\rho\) is in that domain.
The function \(h\) is Lipschitz continuous on the domain, and \(h(\rho)=0\).
There is a continuously differentiable function \(V\), with bounded second derivatives on the domain, such that \(V(q)\geq 0\), \(V(q)=0\) only at \(\rho\), and \(W(q)=-\nabla V(q)^\mathsf{T}h(q)\geq 0\), with equality only at \(\rho\).
\(M_{n+1}\) is a bounded martingale difference: \(E(M_{n+1}\mid\mathcal F_n)=0\). The remainder \(r_{n+1}\) is bounded and \(\sum_{n=0}^{\infty}\gamma_{n+1}\lVert r_{n+1}\rVert<\infty\) almost surely.
The step sizes are nonnegative and \(\mathcal F_n\)-measurable (deterministic step sizes are a common special case), with \(\sum_n\gamma_{n+1}=\infty\) and \(\sum_n\gamma_{n+1}^2<\infty\) almost surely.

Predictability of the step size matters. Since \(\gamma_{n+1}\) is known given \(\mathcal F_n\), the martingale-difference condition implies \(E(\gamma_{n+1}M_{n+1}\mid\mathcal F_n)=\gamma_{n+1}E(M_{n+1}\mid\mathcal F_n)=0\). If the step size depends on the next assignment or outcome, this equality need not hold; the stated sum conditions alone would not control a resulting nonzero accumulated noise term.

Theorem: Under the conditions above, \(q_n\to\rho\) almost surely. In particular, the target is the unique point at which the Lyapunov drift \(W(q)=-\nabla V(q)^\mathsf{T}h(q)\) vanishes.

Why the Lyapunov Argument Works

The proof tracks \(V(q_n)\). A second-order Taylor expansion, using bounded second derivatives and bounded update terms, gives a constant \(C<\infty\) such that

V(q_{n+1})\leq V(q_n)-\gamma_{n+1}W(q_n) +\gamma_{n+1}\nabla V(q_n)^\mathsf{T}M_{n+1} +\gamma_{n+1}\nabla V(q_n)^\mathsf{T}r_{n+1} +C\gamma_{n+1}^2.

The first-order drift term is nonpositive by the Lyapunov condition. The remainder terms have finite total absolute size: for the remainder this is an assumption, and for the Taylor errors it follows from \(\sum_n\gamma_{n+1}^2<\infty\). The martingale terms also have a convergent sum almost surely. Their increments have conditional mean zero because \(\nabla V(q_n)\) is determined by \(\mathcal F_n\) and the step size is \(\mathcal F_n\)-measurable; their conditional variances are bounded by a constant times \(\gamma_{n+1}^2\). The martingale convergence theorem therefore applies.

Consequently, \(V(q_n)\) converges and the nonnegative drift losses have finite weighted sum:

\sum_{n=0}^{\infty}\gamma_{n+1}W(q_n)<\infty \quad\text{almost surely}.

To see why this forces convergence to the target, fix any neighborhood of \(\rho\). On the compact set outside that neighborhood, continuity of \(W\) and its unique zero imply that \(W\) has a strictly positive minimum. Also, the update size is bounded by a constant times \(\gamma_{n+1}\), and \(\gamma_{n+1}\to0\) follows from square summability. Thus a path that makes infinitely many excursions from a smaller neighborhood to outside the fixed one must spend a positive amount of cumulative step size outside the smaller neighborhood on each such excursion. Each excursion would contribute a fixed positive amount to \(\sum_n\gamma_{n+1}W(q_n)\), contradicting its finiteness. The path cannot stay outside the neighborhood forever either, because \(\sum_n\gamma_{n+1}=\infty\) would then make that drift sum infinite. Since this holds for every neighborhood, \(q_n\to\rho\) almost surely.

Worked Allocation Examples

Worked Example: A Two-Arm Target With Noisy Assignments

Consider two arms and let \(q_n\) be the proportion assigned to arm 1 after \(n\) participants. Suppose the next participant is assigned to arm 1 with probability \(\pi(q_n)=0.36+0.40q_n\). This probability lies between \(0.36\) and \(0.76\) for \(0\leq q_n\leq1\), so it is valid throughout the possible range of \(q_n\). The target is \(\rho=0.60\), since \(\pi(0.60)=0.60\).

Let \(I_{n+1}=1\) if the next participant is assigned to arm 1 and \(0\) otherwise. The exact proportion update is \(q_{n+1}=q_n+(I_{n+1}-q_n)/(n+1)\). Its drift is \(h(q)=\pi(q)-q=0.36-0.60q=-0.60(q-0.60)\), with martingale difference \(M_{n+1}=I_{n+1}-\pi(q_n)\).

Take \(V(q)=(q-0.60)^2\). Then \[ W(q)=-V'(q)h(q) =-2(q-0.60)\{-0.60(q-0.60)\} =1.20(q-0.60)^2. \] Thus \(W(q)>0\) for every \(q\ne0.60\), and \(W(0.60)=0\). The step size \(1/(n+1)\) has a divergent sum and a convergent sum of squares; the bounded assignment innovation has conditional mean zero. The theorem therefore gives \(q_n\to0.60\) almost surely.

For an arithmetic check, suppose hypothetically that \(n=20\), \(q_n=0.50\), and the next assignment is to arm 1. Then \(\pi(q_n)=0.56\), \(M_{n+1}=1-0.56=0.44\), and \(h(q_n)=0.06\). Hence \(q_{n+1}=0.50+(0.06+0.44)/21=0.5238\), which also equals \(11/21\), the updated observed proportion.

Worked Example: A Three-Arm Interior Target

Suppose the target is \(\rho=(0.20,0.30,0.50)\). At current proportion vector \(q_n\), assign the next participant according to \(p(q_n)=0.60q_n+0.40\rho\). This is a valid probability vector because it is a convex combination of two vectors in the simplex; each component is nonnegative and the components sum to one. The target is interior, and \(p(\rho)=\rho\).

If \(A_{n+1}\) is the vector with a 1 for the assigned arm and 0 for the other arms, the empirical allocation proportions satisfy \(q_{n+1}=q_n+(A_{n+1}-q_n)/(n+1)\). The mean drift is \(h(q)=p(q)-q=-0.40(q-\rho)\); the noise is \(M_{n+1}=A_{n+1}-p(q_n)\), which is bounded and has conditional mean zero.

Use \(V(q)=\lVert q-\rho\rVert^2\). Its gradient is \(2(q-\rho)\), so \[ W(q)=-2(q-\rho)^\mathsf{T}\{-0.40(q-\rho)\} =0.80\lVert q-\rho\rVert^2. \] This is positive away from the target and zero at the target. With step size \(1/(n+1)\), the theorem establishes almost-sure convergence of all three observed allocation proportions to their target components.

For example, at \(q_n=(0.30,0.20,0.50)\), the assignment probabilities are \(p(q_n)=(0.26,0.24,0.50)\). The drift is \(p(q_n)-q_n=(-0.04,0.04,0)\), which points toward \(\rho\). This direction is a statement about the conditional mean update; a particular random assignment can move the observed proportions in another direction.

Worked Example: Checking the Step-Size Conditions

Suppose an allocation recursion uses the deterministic step size \(\gamma_{n+1}=1/(n+1)^{0.7}\). The exponent is less than 1, so the step-size series diverges; it is greater than \(1/2\), so the squared step-size series converges: \[ \sum_{n=0}^{\infty}\gamma_{n+1}=\sum_{m=1}^{\infty}m^{-0.7}=\infty, \qquad \sum_{n=0}^{\infty}\gamma_{n+1}^2=\sum_{m=1}^{\infty}m^{-1.4}<\infty. \] Because these step sizes are deterministic, they are also \(\mathcal F_n\)-measurable. If the drift has a strict Lyapunov function, the martingale differences are bounded, and there is no nonsummable remainder, the stated consistency theorem applies.

By contrast, \(\gamma_{n+1}=1/(n+1)^{1.2}\) has a finite sum. Even if its squared sum is finite, the updates may have only a finite total opportunity to correct an initial allocation error. This schedule fails a key hypothesis, so the theorem gives no consistency guarantee.

Diagnostics, Alternatives, and Limits

A practical verification begins with the actual recursion, not just a plot of observed proportions. Identify the conditional mean drift and confirm that the noise is mean zero given the recorded history. Check that the target is a zero of the drift, that the proportions remain in the claimed domain, and that a proposed Lyapunov function decreases in drift away from the target. For empirical allocation proportions, assignment indicators are bounded, but a response-adaptive rule may introduce additional estimation error; that error must be represented in \(r_{n+1}\) and shown to be summable after multiplication by the step size, or controlled by a different theorem.

The Lyapunov approach is especially useful when a single function decreases along the mean drift throughout the relevant domain. An ODE-limit argument instead studies trajectories of the differential equation \(\dot q=h(q)\); it can be useful when stability is established through the behavior of those trajectories. A direct scalar recursion or martingale argument may be shorter when the update has a simple explicit form. These approaches are not interchangeable without their assumptions: a unique equilibrium alone does not establish that the recursion approaches it, and local stability alone does not rule out convergence elsewhere from some starting values.

Two failure checks deserve particular attention. First, an apparent zero of \(W\) away from \(\rho\) defeats the unique-attractor conclusion. Second, an adaptive step size chosen using the next assignment can correlate with the innovation, so the product need not have conditional mean zero. In that case, either establish an appropriate conditional mean or bias bound for the product itself, or use a result designed for that dependence; merely checking \(\sum\gamma_n^2<\infty\) is insufficient.

Finally, consistency does not imply that every finite trial closely matches the target, nor does it state a rate of convergence. It also does not ensure that the target is ethically or statistically appropriate. The result concerns long-run allocation proportions under the specified recursion and assumptions. Quantifying the distribution of deviations around an interior target is a distinct question.

Key takeaway: A unique Lyapunov-stable equilibrium is not enough on its own. Almost-sure consistency follows when the allocation recursion has a stable mean drift, the predictable step sizes accumulate without accumulating squared noise, and the weighted remainder is negligible.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

Why must a random step size be \(\mathcal F_n\)-measurable for the stated martingale argument?
For \(V(q)=(q-\rho)^2\) and \(h(q)=-a(q-\rho)\), where \(a>0\), calculate \(W(q)\).
Does the step size \(\gamma_{n+1}=1/(n+1)^{0.4}\) satisfy both step-size sum conditions? Explain.
Suppose \(W(q)\) is zero at both the target and another point in the domain. Does the theorem establish convergence to the target? Why or why not?

Solutions. (1) It ensures \(\gamma_{n+1}\) is known given \(\mathcal F_n\), so \(E(\gamma_{n+1}M_{n+1}\mid\mathcal F_n)=\gamma_{n+1}E(M_{n+1}\mid\mathcal F_n)=0\). (2) Since \(V'(q)=2(q-\rho)\), \(W(q)=-2(q-\rho)\{-a(q-\rho)\}=2a(q-\rho)^2\), positive away from \(\rho\). (3) The first series diverges because its exponent \(0.4<1\), but the squared series has exponent \(0.8<1\) and also diverges. Thus it fails square summability. (4) No. The strict Lyapunov condition requires the target to be the unique zero of \(W\); another zero means the stated argument cannot exclude convergence to that point.