Why Misspecification Matters in an Adaptive Trial
A joint dose model is a working description of how efficacy and toxicity vary across doses and how the endpoints relate. It is necessarily an approximation: the true dose-response pattern is unknown when the trial begins. Ordinary estimation uncertainty arises even when the working model is correct. Model misspecification is different: the working model cannot adequately represent the data-generating pattern, even with arbitrarily much data.
In a fixed design, a poor model can distort estimates and conclusions. In an adaptive design, it can also change who receives which dose next. An incorrect efficacy trend may encourage escalation toward an ineffective dose; an incorrect toxicity trend may make a hazardous dose appear acceptable. If the decision uses the probability of efficacy without toxicity, an incorrect dependence assumption can distort benefit–risk even when both endpoint marginals are estimated well. These errors can feed back: the model affects allocation, and allocation determines which doses generate the next observations.
The focus here is not on diagnosing sampler convergence or choosing a particular joint model. It is on understanding the decision consequences when the model’s structural assumptions are wrong, and on safeguards that make decisions less sensitive to that possibility.
What Is Misspecified?
Let \(E=1\) denote an efficacy response and \(T=1\) a toxicity event at a given dose. Write \(p_E=P(E=1)\), \(p_T=P(T=1)\), and \(p_{11}=P(E=1,T=1)\), with each probability understood at that dose. A model can be wrong in its marginal dose trends, in the joint dependence between endpoints, or in both. These are distinct problems. For example, a model might estimate \(p_E\) and \(p_T\) accurately at each dose but still estimate \(p_{11}\) poorly because it assumes independence.
A dose-selection rule might maximize a utility, such as \(P(E=1,T=0)\), subject to a toxicity constraint. The particular utility and constraint must be specified by the trial; there is no universally correct benefit–risk score. Whatever rule is used, misspecification matters when it changes the estimated quantities that drive the rule, or makes their uncertainty misleadingly narrow.
How Trend Errors Alter Escalation
Many dose-finding procedures use a structured trend to stabilize estimates when only a few participants have been treated at each dose. That structure can be valuable, but a constraint is not evidence that the true trend obeys it. If efficacy is constrained to increase with dose, a true plateau or decline can be obscured. If toxicity is assumed to rise gradually, a sudden increase can be smoothed over. A fitted curve may then look orderly while misrepresenting the dose region that matters for the decision.
The adaptive effect is consequential. Underestimated toxicity can lead to escalation or continued enrollment at a dose whose true risk exceeds the protocol limit. Overestimated toxicity can block access to a useful dose. Overestimated efficacy at a high dose can draw allocation away from a more beneficial lower dose. Conversely, a model that underestimates efficacy at a dose can terminate or bypass it prematurely. The direction depends on the mismatch and the decision rule; misspecification does not always produce conservative decisions.
Worked Example: A Monotone Efficacy Model Misses a Peak
Suppose four ordered doses have hypothetical true efficacy probabilities \(0.20,\ 0.50,\ 0.60,\ 0.30\). Efficacy rises through dose 3, then falls at dose 4. Consider a simple monotone working fit to hypothetical observed data: 2 responses among 10 participants at dose 1, 5 among 10 at dose 2, 6 among 10 at dose 3, and 3 among 10 at dose 4. The observed proportions are \(0.20,\ 0.50,\ 0.60,\ 0.30\).
For illustration, impose a nondecreasing trend by least-squares isotonic pooling. Since \(0.60>0.30\), the last two observations cannot both remain as fitted values under a nondecreasing sequence. Pooling those two gives \((6+3)/(10+10)=9/20=0.45\). The resulting sequence \(0.20,\ 0.50,\ 0.45,\ 0.45\) still decreases from dose 2 to dose 3, so those three observations must also be pooled. Their pooled value is \((5+6+3)/(10+10+10)=14/30=0.4667\). The fitted sequence is therefore \(0.20,\ 0.4667,\ 0.4667,\ 0.4667\).
This constrained fit does not reveal the true decline at dose 4; it represents doses 2 through 4 as having the same fitted efficacy. A more flexible dose model would not necessarily produce these exact estimates, but the example shows the structural limitation: a monotonicity constraint cannot express a peak followed by a decline. If dose choice depends on efficacy alone, the constrained fit can make a less suitable high dose appear as attractive as the peak. A toxicity constraint or other information could change the final decision, but does not repair the efficacy trend assumption.
Dependence Errors in Joint Benefit–Risk
When the decision concerns joint outcomes, correct marginal probabilities do not guarantee a correct joint probability. Under independence, the working model sets \(P(E=1,T=1)=p_Ep_T\). But the probability of efficacy without toxicity is \(P(E=1,T=0)=p_E-p_{11}\). Substituting the independence assumption gives the working estimate \(p_E-p_Ep_T\). The difference between this working estimate and the true probability is
Thus positive association, for which \(p_{11}>p_Ep_T\), makes the independence estimate of efficacy without toxicity too high. Negative association makes it too low. This comparison isolates the dependence error while holding the marginals fixed; it does not claim that every joint dose model estimates the marginals correctly.
Worked Example: Independence Overstates Efficacy Without Toxicity
At a hypothetical dose, suppose \(p_E=0.60\), \(p_T=0.30\), and the true joint probability \(p_{11}=0.24\). These values define a feasible joint distribution: efficacy and toxicity together \(0.24\), efficacy without toxicity \(0.36\), toxicity without efficacy \(0.06\), and neither endpoint \(0.34\). The four probabilities are nonnegative and sum to \(1.00\); their marginals are \(0.60\) and \(0.30\).
The true probability of efficacy without toxicity is \(p_E-p_{11}=0.60-0.24=0.36\). Under independence, the working joint probability is \(p_Ep_T=0.60(0.30)=0.18\), so the estimated efficacy-without-toxicity probability is \(0.60-0.18=0.42\).
The independence estimate exceeds the true value by \(0.42-0.36=0.06\), or 6 percentage points. Equivalently, the difference is \(p_{11}-p_Ep_T=0.24-0.18=0.06\). Here efficacy and toxicity are positively associated, so treating them as independent overstates the chance of efficacy without toxicity. If that quantity contributes to dose utility, the error could make this dose appear more favorable than it is.
Worked Example: A Toxicity Cliff and an Overly Smooth Trend
Consider hypothetical true toxicity probabilities \(0.02,\ 0.04,\ 0.08,\ 0.30\) at four ordered doses. Suppose the protocol treats a probability above \(0.20\) as unacceptable. The fourth dose is above that boundary by \(0.30-0.20=0.10\), or 10 percentage points; the first three are below it. Now suppose a working model forces a smooth, gradually rising curve and, in a sparse interim data set, estimates toxicity at dose 4 as \(0.14\).
The model-based estimate \(0.14\) is below the \(0.20\) boundary, so a rule based only on that estimate could regard dose 4 as acceptable. But the hypothetical true probability \(0.30\) exceeds the boundary. The discrepancy is \(0.30-0.14=0.16\), or 16 percentage points. The particular estimate \(0.14\) is stipulated to illustrate the decision consequence; it is not implied by the true probabilities alone, since the fitted value also depends on the observed data and model.
The risk is especially serious if few participants have been treated near the cliff: a smooth model may borrow strength from safer doses and understate uncertainty about the abrupt rise. A toxicity cap or stopping rule that is evaluated using the same misspecified curve may not protect against the error. A safeguard that uses an independently specified safety boundary or prevents escalation beyond a dose with insufficient safety information can reduce dependence on the curve, although it cannot eliminate uncertainty.
Recognizing Consequences and Choosing Safeguards
Model checking can reveal warnings, but no single diagnostic proves a model correct. Examine whether observed outcomes systematically differ from predictions by dose, whether predictions change sharply under modest alternative trend assumptions, and whether the dose recommendation is driven by extrapolation beyond well-observed doses. A model can converge computationally and still be structurally wrong. Likewise, a good-looking overall fit may conceal a local failure at the dose that determines escalation or selection.
Several safeguards address different failure modes; they are complementary rather than interchangeable.
| Approach | What it can protect against | Important limitation |
|---|---|---|
| Compare plausible alternative trend forms | Shows whether escalation or selection depends on a monotonic, smooth, or otherwise restrictive curve | Cannot guarantee that the true pattern is among the alternatives |
| Use model averaging or a robust model class | Can reduce reliance on one narrowly specified trend or dependence structure | Results still depend on the candidate models and their weights |
| Apply explicit dose and toxicity constraints | Limits escalation or selection when safety evidence is inadequate or risk is too high | A constraint based on a wrong risk estimate may still fail |
| Require adequate local data and retain dose-level summaries | Reduces decisions driven entirely by borrowing or extrapolation from other doses | Needs more observations and may slow learning |
| Check recommendations across plausible scenarios | Reveals sensitivity of decisions to trends and endpoint dependence | Scenario checks are only as informative as the scenarios considered |
These approaches differ from simply fitting a more complex model. Flexibility can represent more patterns, but with sparse data it can also increase uncertainty and instability. A simpler model may be useful for estimation while a conservative operational rule limits what the model is allowed to authorize. Robustness is best assessed at the level of the actual decision: do the escalation path, stopping behavior, and selected dose remain acceptable under credible alternatives?
Common Pitfall and Limitations
- Wrong reasoning: “The model converged and its fitted curve is smooth, so the dose recommendation is reliable.” Correction: Convergence concerns computation, not whether the model represents the true dose-response pattern. A stable fit can be stably wrong.
- Wrong reasoning: “The efficacy and toxicity marginals look plausible, so the probability of efficacy without toxicity is also credible.” Correction: That joint probability depends on endpoint dependence; with positive association, an independence calculation overstates it.
- Wrong reasoning: “A safety rule automatically protects the trial from misspecification.” Correction: A rule that uses the same biased model-based risk estimate can inherit its error. The rule’s inputs and the assumptions behind them require scrutiny.
No safeguard removes the need for scientific judgment. Plausible alternatives may omit the true pattern, local data may remain sparse, and conservative rules can slow escalation or exclude a beneficial dose. The objective is not to claim certainty about the unknown model. It is to identify which assumptions can change decisions, make consequential decisions less dependent on a single assumption, and state the remaining limitations.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- If a true efficacy curve rises and then falls, what feature of a monotone increasing working model causes the mismatch?
- Suppose \(p_E=0.50\), \(p_T=0.20\), and \(p_{11}=0.14\). Find the true and independence-based probabilities of efficacy without toxicity and identify the direction of error.
- Why can a toxicity rule based on a fitted dose-response curve fail to protect against an abrupt toxicity increase?
- Name one diagnostic warning and one operational safeguard that address different aspects of misspecification.
Solutions. (1) The model cannot represent a decline after the peak, so it must flatten or otherwise distort the trend. (2) The true probability is \(0.50-0.14=0.36\). Independence gives \(0.50-0.50(0.20)=0.40\). The estimate is high by \(0.04\), because \(p_{11}-p_Ep_T=0.14-0.10=0.04\). (3) If the curve smooths over the increase, it can underestimate risk at the dose where the rule is applied; the rule then inherits the model’s error. (4) A warning could be dose-specific residual patterns or a recommendation that changes under plausible trend forms. An operational safeguard could restrict escalation until adequate local safety evidence is available.