Why a Ratio Can Be Biased
An arm-specific response mean is often reported as the total response observed on that arm divided by the number assigned to it. Under adaptive randomization, that denominator is random: earlier responses can influence later assignments. The numerator and denominator can therefore move together in ways that matter even when each new response has the correct conditional mean. A ratio can have finite-sample bias as a result.
The key point is not simply that a random denominator creates bias. A numerator that changes proportionally with the denominator can cancel much of the usual ratio-curvature effect. What remains, in the common adaptive sample-mean setting, is driven at second order by covariance between the arm's accumulated response noise and its assignment count.
The General Second-Order Expansion
Write \(a_n=E[A_n]\), \(b_n=E[B_n]\), \(\Delta_n=A_n-a_n\), and \(\delta_n=B_n-b_n\). Expanding the reciprocal of the denominator around \(b_n\) gives
Multiplying by \(a_n+\Delta_n\), keeping terms through degree two, and taking expectations yields
The linear terms disappear because \(E[\Delta_n]=E[\delta_n]=0\). The two retained terms come from the product \(-\Delta_n\delta_n/b_n^2\) and the curvature term \(a_n\delta_n^2/b_n^3\). The approximation is not an exact identity: its accuracy depends on higher-order terms being small.
Specialization to an Adaptive Arm Mean
Consider one treatment arm. At trial time \(i\), let \(I_i=1\) if participant \(i\) is assigned to this arm and \(I_i=0\) otherwise. The assignment \(I_i\) is predictable: it is determined using information available before that participant's response is observed. Let \(Y_i\) be the response when assigned to the arm, with conditional mean \(\mu\), and define the accumulated count and response total by
For the derivation, assume the response residuals have conditional mean zero when the participant is assigned to the arm: \(E[Y_i-\mu\mid\mathcal{F}_{i-1}, I_i=1]=0\), where \(\mathcal{F}_{i-1}\) denotes the information available before the \(i\)th response. Also assume \(N_n>0\). Define the accumulated residual \(M_n=S_n-\mu N_n\). Then \(E[M_n]=0\), and the arm mean is
Applying the general expansion to \(A_n=S_n=\mu N_n+M_n\) and \(B_n=N_n\) explains why denominator variability alone is not the leading bias. In fact,
Substitution into the general formula cancels the two terms involving \(\mu\operatorname{Var}(N_n)\). If \(m_n=E[N_n]\), the resulting leading bias is
Here \(M_n\) is measured in response units and \(N_n\) in participants, so the covariance divided by \(m_n^2\) has units of response per participant, matching the arm mean. Under regular large-sample conditions, if \(m_n\) grows proportionally to \(n\), both \(\operatorname{Var}(N_n)\) and \(\operatorname{Cov}(M_n,N_n)\) are of order \(n\), and the displayed bias term is of order \(1/n\). This is a finite-sample correction, not a replacement for a first-order limit result.
Worked Examples
Worked Example: A Positive Feedback Covariance
Suppose a hypothetical adaptive trial has \(E[N_n]=100\) participants on an arm and \(\operatorname{Cov}(M_n,N_n)=50\) response-participants. The target arm mean is \(\mu\) responses per participant. Assume the denominator is concentrated enough around 100 that the second-order approximation is appropriate.
The estimated leading bias is
Thus the ratio estimator is expected, to this order, to fall about \(0.00500\) responses per participant below \(\mu\). A positive covariance means that larger residual totals tend to occur with larger arm counts. The negative sign in the expansion determines the direction of this leading bias. This is an approximation based on the stated moments, not a claim that every realized estimate is below \(\mu\).
Worked Example: Checking the Expansion Against an Exact Expectation
In a hypothetical two-outcome construction, suppose \(N_n\) is 95 or 105 with equal probability. Let \(M_n=-10\) when \(N_n=95\), and \(M_n=10\) when \(N_n=105\), in response units. Then \(E[N_n]=100\) and \(E[M_n]=0\). The covariance is
The second-order estimate of the bias of \(\widehat{\mu}_n=\mu+M_n/N_n\) is therefore
The exact expectation of the residual ratio is
The approximation is close to the exact value in this construction. The match illustrates the expansion; it does not establish that every adaptive trial has such a small remainder.
Worked Example: Zero Covariance Can Remove the Leading Bias
Suppose, hypothetically, that \(N_n\) is equally likely to be 90 or 110, while \(M_n\) independently takes values \(-10\) and \(10\), each with probability one half. Then \(E[M_n]=0\) and independence gives \(\operatorname{Cov}(M_n,N_n)=0\). The second-order adaptive-arm formula gives a leading bias of
where \(E[N_n]=(90+110)/2=100\). In this example, the exact expectation is also zero:
A calculation that keeps only denominator variance would incorrectly suggest a nonzero bias. For the arm mean, the numerator contains the term \(\mu N_n\), so its covariance with the denominator cancels the reciprocal-curvature contribution. The residual covariance, not denominator variability by itself, is the relevant second-order term.
Worked Example: How Adaptive Feedback Creates Covariance
Consider a hypothetical two-participant trial. The first participant is assigned to arm A. Their response residual \(\varepsilon_1=Y_1-\mu\) is \(+1\) or \(-1\), each with probability one half. The second participant is assigned to arm A only if \(\varepsilon_1=+1\); otherwise they receive another arm. If assigned to A, the second participant's residual \(\varepsilon_2\) has conditional mean zero and is independent of \(\varepsilon_1\). For arm A, \(N=1+I(\varepsilon_1=+1)\) and \(M=\varepsilon_1+I(\varepsilon_1=+1)\varepsilon_2\).
The expected count is \(E[N]=1.5\), and \(E[M]=0\). Since the second residual has conditional mean zero, it contributes zero to \(E[MN]\). The first residual gives
The second-order approximation is
For comparison, the exact expectation is \(E[M/N]=\tfrac12(1/2)+\tfrac12(-1)=-0.2500\) responses per participant: conditional on a positive first residual, the second residual averages to zero, leaving \(M/N\) with conditional mean \(1/2\); after a negative residual, it equals \(-1\). The approximation has the same direction but differs in magnitude because the denominator is small and variable. The example makes the mechanism explicit: a positive early residual triggers an additional assignment to arm A, creating positive covariance between residual total and arm count.
What to Check in an Application
The second-order formula is useful only when its assumptions and scale are plausible. In practice, examine the allocation rule and the data-generating assumptions together. A rule that reacts to interim estimated responses can link earlier residuals to later arm counts; a rule that does not use response information may produce a different covariance structure. The direction and size of the bias cannot be inferred from the word “adaptive” alone.
- Check that the arm count is not near zero. If \(N_n\) can be zero or very small with appreciable probability, the ratio may be undefined or dominated by rare, extreme denominators.
- Check the conditional response model. The decomposition \(S_n=\mu N_n+M_n\) is useful for a common conditional mean \(\mu\). If response means change over time or depend on covariates not accounted for, the residual sum may not have mean zero under the stated model.
- Check the covariance, not just the allocation variance. \(N_n\) can be variable while \(\operatorname{Cov}(M_n,N_n)\) is zero, or it can have a nonzero covariance because earlier responses drive later assignments.
- Check approximation scale. Compare the predicted bias with the estimator's finite-sample variability and, when possible, with simulation under the specified adaptive rule. Large differences can indicate a poor Taylor approximation or a model mismatch.
One important failure is to condition on the entire realized allocation path and then claim the arm mean is unbiased because each response has mean \(\mu\). The full path can encode earlier responses: in the feedback example, observing that a second participant was assigned to arm A reveals that the first residual was positive. Conditional on that path, the first residual is not mean zero. Predictability of each assignment before its response supports the martingale residual argument unconditionally; it does not make the completed allocation path independent of the observed outcomes.
How the Ratio Approach Compares
| Approach | What it uses | Relevance to second-order bias |
|---|---|---|
| Adaptive ratio \(S_n/N_n\) | The observed arm total divided by its realized count | Directly interpretable as the arm's observed mean; its leading bias can reflect residual-count covariance. |
| Fixed-denominator mean | A response total divided by a nonrandom sample size | Removes random-denominator curvature, though it is not generally available when the arm count is adaptive. |
| Weighted total or ratio | Responses reweighted using assignment probabilities | Can target a different estimand or adjust for unequal assignment probabilities, but requires correctly specified probabilities and has its own finite-sample behavior. |
These approaches answer related but not identical questions. The observed arm mean summarizes responses among participants actually assigned to that arm. A weighted estimator may instead target a population quantity defined under a different assignment scheme. Neither choice eliminates the need to state the estimand, the assignment mechanism, and the assumptions used for inference.
Common Mistakes and Limitations
- Wrong reasoning: “The count is random, so the ratio must have a bias proportional to its variance.” Correction: In an arm mean, the numerator includes \(\mu N_n\). Its covariance with \(N_n\) cancels the denominator-variance term; the leading term is \(-\operatorname{Cov}(M_n,N_n)/m_n^2\).
- Wrong reasoning: “Each response has conditional mean \(\mu\), so the ratio is exactly unbiased.” Correction: A mean-zero residual total \(M_n\) does not imply \(E[M_n/N_n]=0\) when \(M_n\) and \(N_n\) are dependent.
- Wrong reasoning: “The second-order expression is an exact correction.” Correction: It drops higher-order terms. Small denominators, strong feedback, or highly variable responses can make those terms consequential.
- Wrong reasoning: “A positive covariance proves that the estimator is biased downward in every sample.” Correction: It predicts the direction of the leading expected bias under the expansion; individual estimates can lie above or below the target.
The approximation is limited to the ratio and target represented by its numerator and denominator. It does not by itself provide a variance estimate, a confidence interval, or a general correction that is valid for every allocation rule. Those tasks require their own assumptions and calculations.
Check Your Understanding
Answer each question, then compare with the solutions below.
- For a general ratio \(A_n/B_n\), which two second-order terms contribute to its approximate bias relative to \(E[A_n]/E[B_n]\)?
- An adaptive arm has \(E[N_n]=80\) and \(\operatorname{Cov}(M_n,N_n)=-32\) response-participants. What is the approximate bias, with units?
- Why can conditioning on the complete allocation path invalidate a claim that the arm's residual total has conditional mean zero?
- What happens to the leading bias expression for an adaptive arm mean if \(\operatorname{Cov}(M_n,N_n)=0\)? Does that prove the exact finite-sample bias is zero?
Solutions. (1) The terms are \(-\operatorname{Cov}(A_n,B_n)/E[B_n]^2\) and \(E[A_n]\operatorname{Var}(B_n)/E[B_n]^3\). (2) The approximation is \(-(-32)/80^2=0.00500\) responses per participant, so the leading bias is upward. (3) Later assignments can depend on earlier responses, so the completed path can reveal information about earlier residuals; mean-zero increments before observing their responses do not guarantee mean zero after conditioning on that path. (4) The leading second-order term is zero, but higher-order terms may remain, so exact unbiasedness does not follow from zero covariance alone.