Why Model Form Matters for Allocation
In covariate-adjusted response-adaptive allocation, an outcome model helps determine which treatment appears more promising for a patient with a particular baseline profile. If the model misses an important feature of the response relationship, the resulting treatment ranking can be wrong for some profiles. Since allocation probabilities are then based on those rankings, model error can affect who receives each treatment—not just how an analyst describes the data afterward.
This tutorial focuses on three sources of error: leaving out nonlinear covariate effects, leaving out treatment-by-covariate interactions, and choosing an unsuitable link function for the outcome. The examples use hypothetical data and an intentionally simple allocation rule to make the consequences visible. They do not imply that every adaptive design uses a hard ranking threshold; many designs translate fitted response differences into probabilities more gradually.
For a binary success outcome, \(m_t(x)\) is the probability of success under treatment \(t\), so \(\Delta(x)\) is a difference in probabilities. For a continuous outcome, the contrast has the outcome’s measurement units. In either case, the ranking is profile-specific: A ranks above B at profile \(x\) when \(\Delta(x)>0\).
The interpretation assumes that the outcome is defined consistently across treatments, that baseline covariates are measured before assignment, and that both treatments have a nonzero chance of being assigned across the profiles used to learn their responses. Because allocation adapts over time, the model is fitted from data gathered under an assignment process that may depend on earlier outcomes. Adequate exploration and correct accounting for that process matter for learning; a flexible formula alone cannot recover comparisons unsupported by the data.
How a Ranking Error Changes Allocation
To isolate the role of the ranking, consider a two-arm illustrative rule. If \(\hat\Delta(x)>0\), assign treatment A with probability 0.80; if \(\hat\Delta(x)<0\), assign A with probability 0.20; and if the estimated contrast is exactly zero, assign A with probability 0.50. The remaining probability is assigned to B. This rule preserves exploration, since neither treatment’s probability falls below 0.20. A misspecified model can still send more patients toward the less effective treatment for a profile.
The rule is only an illustration: a design may use a smooth function of \(\hat\Delta(x)\), impose a different exploration floor, or account for additional design goals. The central dependence remains: if a model systematically gets \(\hat\Delta(x)\)’s sign wrong, an allocation rule that favors the estimated winner can systematically favor the wrong treatment at that profile.
Omitting Nonlinear Covariate Effects
A linear covariate term assumes that each one-unit increase in that covariate changes the modeled mean by a constant amount, unless the model includes additional terms. Real response patterns can bend: the change may be small at one range and large at another. If treatment response curves bend differently, an additive straight-line fit can distort their difference and its ranking.
Worked Example: Curvature Hidden by a Straight-Line Fit
Suppose a hypothetical pilot trial has a continuous outcome measured in points, and the observed mean responses at three covariate values are as follows. The covariate \(x\) is measured in years above a reference value; both treatments were represented at each value.
| \(x\) | Mean response on A | Mean response on B | Observed contrast A minus B |
|---|---|---|---|
| 0 | 4 points | 5 points | -1 point |
| 1 | 9 points | 6 points | 3 points |
| 2 | 6 points | 7 points | -1 point |
The pattern for A bends, rising and then falling. A quadratic mean model reproduces these three hypothetical means: \(m_A(x)=4+9x-4x^2\) points. For example, at \(x=2\), \(m_A(2)=4+9(2)-4(2^2)=6\) points. The linear pattern for B is \(m_B(x)=5+x\) points. Thus the fitted quadratic contrast is
It gives \(\Delta(0)=-1\), \(\Delta(1)=3\), and \(\Delta(2)=-1\) points. A straight-line regression fitted separately to the three listed means for each treatment gives \(\hat m_A(x)=5.333+1x\) and \(\hat m_B(x)=5+1x\), in points. For A, the mean of the three responses is \(19/3=6.333\) points and the fitted slope is \(1\) point per covariate unit; for B, the mean is \(6\) points and the slope is also \(1\). The fitted contrast is therefore \(0.333\) points at every \(x\).
Under the illustrative allocation rule, this linear model assigns A with probability 0.80 at all three profiles. But at \(x=0\) and \(x=2\), the listed response means favor B by 1 point, so the model directs more allocation to A where B has the higher mean. The example assumes the listed means represent the profile-specific response pattern of interest; in a real trial, sampling uncertainty would also need consideration.
This is not a claim that a straight line is always wrong. It shows that a straight-line model can average over a bend and obscure profile-specific differences. The risk is greatest when allocation depends strongly on the sign of a fitted contrast or when the model is used at covariate values where data are sparse.
Omitting Treatment-by-Covariate Interactions
An interaction allows the treatment contrast to change with a covariate. Without an interaction, a model may impose the same treatment contrast at every profile, even when one treatment is better for one group and the other treatment is better for another. This is distinct from curvature: the key issue is that the response difference between treatments varies across profiles.
Worked Example: A Treatment Effect That Changes by Profile
Imagine a hypothetical trial with a binary baseline profile indicator \(z\), and a binary success outcome. The observed success proportions are 0.70 on A and 0.60 on B when \(z=0\), and 0.55 on A and 0.75 on B when \(z=1\). Assume both treatments have been observed in both profiles.
The profile-specific contrasts are
Thus A has a 10-percentage-point higher success proportion at \(z=0\), while B has a 20-percentage-point higher success proportion at \(z=1\). An additive model that omits the treatment-by-\(z\) interaction forces a common treatment contrast. With equal numbers of patients in the two profiles, the pooled proportions are \(0.625=(0.70+0.55)/2\) on A and \(0.675=(0.60+0.75)/2\) on B. The common contrast is \(0.625-0.675=-0.050\), or minus 5 percentage points.
That additive summary favors B in both profiles. Under the illustrative rule, it assigns A with probability 0.20 in both groups, although the profile-specific proportions favor A at \(z=0\). The interaction model retains the two distinct contrasts and would favor A at \(z=0\) and B at \(z=1\). Equal profile sizes make the pooled calculation transparent; with unequal sizes, the pooled contrast depends on the profile weights and still need not represent either group’s contrast.
Incorrect Link Functions for Binary Outcomes
For a binary outcome, a link function specifies how a model’s linear predictor relates to the conditional success probability. For example, a logistic model uses \(p=1/(1+\exp(-\eta))\), where \(\eta\) is the linear predictor. A linear probability model instead sets \(p=\eta\). The latter can be convenient, but its predictions are not constrained to the interval from zero to one, and the shape it imposes may be unsuitable.
A useful distinction is that applying the same strictly increasing link to two fixed linear predictors cannot by itself change their ordering: if \(\eta_A>\eta_B\), then \(g^{-1}(\eta_A)>g^{-1}(\eta_B)\) for an increasing inverse link \(g^{-1}\). In practice, however, fitting different link functions generally produces different coefficient estimates and fitted predictors. Those fitted contrasts can have different signs, especially with limited data, nonlinear response patterns, or treatment-specific relationships to covariates.
Worked Example: Different Links, Different Fitted Ranking
Consider a hypothetical randomized pilot with a binary success outcome and a continuous covariate \(x\). Both treatments were assigned across the observed range, which includes \(x=2\). Suppose separate model fits to the pilot data produce the following fitted predictors. These are illustrative fitted estimates, not population truths.
The logistic fit gives \(\hat\eta_A(x)=-1.386+0.7935x\) and \(\hat\eta_B(x)=-0.500+0.5595x\). At \(x=2\), the predictors are \(0.201\) and \(0.619\). Transforming them to probabilities gives
The logistic fit estimates a contrast of \(0.550-0.650=-0.100\), so it favors B by 10 percentage points. A separate linear-probability fit gives \(\hat p_A(x)=0.30+0.18x\) and \(\hat p_B(x)=0.50+0.04x\). At \(x=2\), these yield \(0.30+0.18(2)=0.66\) and \(0.50+0.04(2)=0.58\), a contrast of \(0.66-0.58=0.08\). Both predictions are within the probability range at this profile, but this fit favors A by 8 percentage points.
The two fits therefore produce opposite rankings, and the illustrative rule assigns A with probability 0.20 under the logistic fit versus 0.80 under the linear-probability fit. The reversal is due to the different fitted models, not to a monotone transformation reversing a fixed pair of predictors. A real analysis would examine the pilot data, uncertainty, and fit across relevant profiles before using either ranking to steer allocation.
Justification: Why a Small Model Error Can Matter
Allocation depends on estimated contrasts, while treatment ranking depends on their signs. Consequently, a small numerical prediction error can matter little when the estimated contrast is far from zero but can change the decision when the contrast is near zero. The following inequality makes that point precise. Let the true contrast be \(\Delta(x)\) and the model’s error at profile \(x\) be \(e(x)=\hat\Delta(x)-\Delta(x)\). If the model ranks A above B when the truth ranks B above A, then \(\Delta(x)<0\) and \(\hat\Delta(x)>0\). Since \(\hat\Delta(x)=\Delta(x)+e(x)\), it follows that \(e(x)>-\Delta(x)=|\Delta(x)|\).
The same argument applies with the treatment labels reversed. Thus a sign reversal requires prediction error at least as large as the true contrast’s distance from zero. Near-zero contrasts are especially vulnerable: even modest misspecification can change the estimated winner. A hard-threshold allocation rule can then change abruptly, as the examples illustrate. A smoother rule reduces abrupt changes but does not eliminate the possibility of systematically favoring the wrong treatment.
Warning Signs, Alternatives, and Limits
Model form should be considered alongside the data support and the consequences of allocation. Useful warning signs include systematic residual patterns across the covariate range, different treatment contrasts in clinically meaningful profiles, implausible probabilities, and rankings that change sharply when a reasonable nonlinear term, interaction, or link is substituted. Such checks can reveal sensitivity; they cannot prove that a particular model is correct. In particular, a good overall fit can conceal poor predictions in a subgroup that receives a distinct allocation probability.
| Approach | What it allows | Trade-off for allocation |
|---|---|---|
| Additive linear model | Constant covariate slopes and a common treatment contrast unless specified otherwise | Simple to estimate, but can miss curvature or effect modification |
| Nonlinear terms or interactions | Curved responses or profile-specific treatment contrasts | More responsive to real structure, but needs adequate data across profiles |
| Alternative link functions | Different relationships between predictors and outcome means or probabilities | May better suit the outcome scale, but fitted rankings can be sensitive to model choice |
A practical comparison is to assess whether profile-specific rankings remain similar under a small set of defensible model forms. If they do not, the design team should recognize that allocation is model-sensitive rather than treating one fitted ranking as certain. More complex models are not automatically safer: sparse profiles, limited outcomes, or little treatment overlap can make their estimated contrasts unstable. An exploration floor can prevent probabilities from reaching zero, but it cannot make a misspecified ranking accurate.
One common mistake is to say, “The average treatment contrast is negative, so B is better for every patient.” That reasoning confuses an average over a chosen covariate distribution with a contrast at a particular profile. The correction is to distinguish the overall average from \(\Delta(x)\), inspect whether the model allows that contrast to vary, and avoid interpreting an imposed common effect as evidence of uniform benefit.
Finally, a model-based allocation probability is a design decision informed by current estimates, not a probability that the favored treatment is truly superior. Misspecification primarily threatens the quality of profile-specific predictions and the allocation decisions built from them. It does not, by itself, establish that every later treatment comparison is biased; the consequences for inference depend on the analysis and whether the adaptive assignment mechanism is properly accounted for.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- If a linear model estimates \(\hat\Delta(x)=0.2\) points at every profile but the true contrast is \(-0.5\) points for one profile, what is the contrast error there, and what ranking does the model give?
- In the interaction example, why is the pooled contrast of minus 5 percentage points not a profile-specific conclusion?
- Can applying one strictly increasing inverse link to two fixed treatment predictors reverse their order? Explain.
- Name one warning sign of omitted curvature and one reason a more complex model might still perform poorly.
- Under the illustrative allocation rule, what probability of A is used when \(\hat\Delta(x)=0\), and what does that probability mean?
Solutions. 1. The error is \(0.2-(-0.5)=0.7\) points. The model ranks A higher, although the true contrast favors B. 2. The pooled contrast averages responses over the two profiles; it is not either profile’s contrast, which is \(0.10\) at \(z=0\) and \(-0.20\) at \(z=1\). 3. No. A strictly increasing transformation preserves order; reversals can arise because fitting different links can change the estimated predictors. 4. A systematic residual pattern across \(x\) can signal curvature; a more complex model may be unstable when data are sparse across profiles. 5. The probability is 0.50. It is the design’s chance of assigning the next patient with that profile to A under this illustrative rule, not a probability that A is superior.