Why Diagnose the Dose-Response Model?
A dose-response model compresses observed outcomes across dose arms into a curve that can support a development decision. The curve may fit the overall pattern while missing important features at particular doses. If the misspecification changes the estimated response near a decision threshold, it can also change which dose appears suitable. Diagnosis therefore asks more than whether a curve looks smooth or has a favorable fit statistic: it asks whether departures between the data and the fitted model are structured, credible, and consequential for the decision.
The focus here is on arm-level evidence. Let \(D_i\) be the dose in arm \(i\), let \(y_i\) be the estimated arm-level response on the endpoint scale used for the decision, and let \(\hat{y}_i=f(D_i;\hat{\theta})\) be the response predicted by a fitted dose-response model. For illustration, assume higher response values are favorable. The same diagnostic logic applies when lower values are favorable, provided the endpoint direction is handled consistently.
Diagnostics are most interpretable when the model, endpoint, dose range, and decision criterion match the intended use. They also require that the arm-level estimates and their uncertainty be appropriate for the estimand. If arm estimates share a control group or otherwise have correlated errors, their covariance must be considered in formal uncertainty calculations. A pattern in a residual plot is a warning to investigate, not proof that the model is wrong: random variation, imprecise arm estimates, or an influential observation can also create apparent structure.
Read Residuals in Dose Order
Plot residuals against dose, preferably with dose displayed in a scale that makes relevant spacing clear, and include uncertainty information for the arm-level estimates. A suitable model should leave residuals without a persistent dose-related pattern after accounting for uncertainty. Random scatter around zero is reassuring but does not establish that the model is correct; a small number of doses can have little power to reveal misspecification.
Patterns worth investigating include a run of positive residuals at higher doses, alternating positive and negative residuals, or residuals that grow in magnitude with dose. A run of positive residuals may indicate that the model underpredicts responses in that region. Alternation can indicate that the chosen curve is too rigid to capture the observed shape. Increasing spread may indicate that the assumed error model is inadequate. These are clues, not diagnoses by themselves: dose order, uncertainty, endpoint scale, and the number of arms all matter.
Worked Example: Detecting Structure in Dose-Ordered Residuals
Suppose a hypothetical study estimates responses at five doses. The table gives arm-level estimates, predictions from a candidate model, and standard errors for the observed estimates.
| Dose (mg) | Observed response \(y_i\) | Predicted response \(\hat{y}_i\) | Residual \(r_i\) | SE of \(y_i\) |
|---|---|---|---|---|
| 0 | 0.0 | 0.0 | 0.0 | 0.4 |
| 25 | 2.1 | 3.0 | -0.9 | 0.5 |
| 50 | 5.2 | 5.2 | 0.0 | 0.5 |
| 100 | 7.9 | 6.8 | 1.1 | 0.6 |
| 200 | 8.1 | 7.5 | 0.6 | 0.7 |
The residual calculation at 25 mg is \(2.1-3.0=-0.9\) response units; at 100 mg it is \(7.9-6.8=1.1\) response units. Scaling these discrepancies only by the observed-estimate standard errors gives \(-0.9/0.5=-1.80\) and \(1.1/0.6=1.83\), respectively. These ratios are descriptive here, not formal standardized residuals, because they omit uncertainty induced by fitting the model.
The signs move from negative at 25 mg to positive at both higher doses. This could mean the candidate model overpredicts at a lower dose and underpredicts at higher doses. The 200 mg discrepancy is smaller relative to its standard error than the 100 mg discrepancy. With only five arms, this is a reason to examine model adequacy and decision sensitivity, not a basis for declaring the curve misspecified or for asserting that a particular alternative is correct.
Use Held-Out Observed-versus-Predicted Contrasts
In-sample residuals are partly constrained by the fitting process: the model parameters were estimated using the same observations being checked. A useful complement is to omit one arm, refit the model to the other arms, and predict the omitted arm. This leave-one-arm-out prediction asks whether the fitted relationship, learned without that arm, predicts its observed response. It is especially informative when a dose-specific discrepancy might be hidden by that arm's influence on the fitted curve.
Let \(\hat{y}_{i,-i}\) be the prediction at dose \(D_i\) from the fit that omits arm \(i\), and let \(SE_{i,-i}\) be the standard error of that prediction. If the omitted arm's estimate and the prediction are independent, the contrast \(c_i=y_i-\hat{y}_{i,-i}\) has estimated variance equal to the sum of their variances. The resulting standardized contrast is a diagnostic summary, not automatically a hypothesis test with a calibrated \(p\)-value.
Worked Example: Checking a Held-Out Dose Arm
At a hypothetical 60 mg dose, the observed arm response is \(y_i=7.0\) units with \(SE_i=0.6\). Refitting the candidate model without this arm predicts \(4.8\) units at 60 mg, with prediction standard error \(SE_{i,-i}=0.8\). Assume the omitted arm's estimate is independent of this refitted prediction.
The observed-minus-predicted contrast is \(c_i=7.0-4.8=2.2\) response units. Its estimated standard error is
The standardized contrast is \(2.2/1.00=2.20\). Thus the held-out observation is 2.20 estimated standard errors above the prediction. This is a notable discrepancy to investigate, but it is not by itself a probability that the model is wrong or a formal \(p\)-value. The conclusion depends on how well the prediction uncertainty is estimated, the number of arms examined, and whether the independence assumption is justified.
Check Whether an Arm Can Change the Decision
An unusual residual is not necessarily important to dose selection, and an arm with a modest residual can still be influential if it anchors a region that matters to the decision. Influence is therefore best assessed by refitting the model after omitting each arm in turn and comparing the results with the full-data fit. The comparison should include relevant model parameters or predictions and, most importantly, the dose implied by the prespecified decision criterion.
This is a sensitivity analysis, not a license to discard an inconvenient arm. Every omitted-arm fit answers a counterfactual question: how would the analysis change if that arm's information were unavailable? The complete analysis remains based on the full evidence unless a data-quality or design issue independently warrants exclusion. Record the reason for any exclusion separately from the effect that the arm has on the result.
Worked Example: Assessing Influence on a Target Dose
Consider a hypothetical Emax model for a favorable response, \(E(D)=E_{\max}D/(ED_{50}+D)\), with \(E_{\max}\) and \(ED_{50}\) positive. Suppose the decision criterion is a model-predicted response of at least 6 units, and the dose is to be compared under a full-data fit and a fit omitting the highest-dose arm. The full-data estimates are \(E_{\max}=12\) units and \(ED_{50}=80\) mg. The omitted-arm refit gives \(E_{\max}=12\) units and \(ED_{50}=55\) mg. Treat these as hypothetical fitted outputs.
For this model, setting \(E(D)=6\) gives \(6=E_{\max}D/(ED_{50}+D)\), hence \(6ED_{50}=D(E_{\max}-6)\), and therefore \(D=6ED_{50}/(E_{\max}-6)\). Under the full-data fit, the target dose is \(6(80)/(12-6)=480/6=80\) mg. Under the omitted-arm fit, it is \(6(55)/(12-6)=330/6=55\) mg.
The omitted-arm result is 25 mg lower; relative to the full-data target, the change is \(25/80=0.3125\), or 31.3%. This indicates material sensitivity to the highest-dose arm for this illustrative criterion. It does not establish which fit is correct. The arm-level residual, the omitted-arm prediction, data quality, and other model checks should be considered together before drawing a dose-selection conclusion.
The same comparison can be made for predictions at decision-relevant doses, not only for a target dose. For instance, a model may yield a similar target while predicting meaningfully different responses across a range of doses. Conversely, a parameter may change substantially while the decision remains stable. Report both the change in model behavior and whether the decision itself changes.
Diagnostics, Alternatives, and Limitations
A practical diagnostic sequence is to examine dose-ordered residuals, calculate held-out arm contrasts when feasible, and perform leave-one-arm-out refits. Then ask whether the observed discrepancies are larger than expected given arm-level uncertainty and whether they affect the response region relevant to the decision. Earlier material on predictive checks and validation of exposure-response models provides broader context for checking model predictions; here the emphasis is on dose-arm patterns and their influence on dose selection.
Check dose labels, endpoint direction, arm-level estimates, standard errors, and any covariance induced by shared comparators.
Look for structured signs or changing spread, and interpret the pattern against the uncertainty and number of arms.
Compare held-out predictions with observations, then repeat the fit with each arm omitted and track decision-relevant outputs.
Describe which doses or criteria are sensitive, and distinguish a robustness concern from evidence that a particular alternative model is true.
Other approaches can complement these checks. A lack-of-fit comparison can assess whether departures exceed what is expected under a specified error model, but its usefulness depends on replication and appropriate assumptions. Flexible candidate models can reveal whether a restrictive shape is driving the result, but added flexibility can also make estimates less stable. Predictive checks assess whether the model reproduces features of the data under its assumed sampling process. No single approach replaces examination of dose-specific discrepancies and decision sensitivity.
Several limitations are important. With few dose arms, residual patterns and held-out predictions may be highly uncertain. Omitting an arm can leave little information to estimate model parameters, so an unstable refit may reflect sparse information rather than a uniquely influential arm. Arm-level summaries can conceal patient-level structure, and uncertain standard errors or dependence between arms can distort diagnostic contrasts. Diagnostics also depend on the endpoint scale: a pattern on one scale need not look the same on another. Finally, apparently reassuring residuals cannot rule out misspecification in dose regions with little or no data.
The aim is not to find a curve that explains every fluctuation. It is to determine whether the fitted relationship is a sufficiently credible summary for the intended decision, and to make clear when a dose recommendation depends strongly on a particular arm or model assumption.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- At one dose, the observed response is 5.4 units and the fitted response is 6.1 units. What is the residual, and what does its sign mean?
- An omitted arm has observed estimate 4.0 units with standard error 0.3; the leave-one-arm-out prediction is 3.2 units with standard error 0.4. Assuming independence, calculate the contrast and its standard error.
- Why is a leave-one-arm-out contrast generally more informative about prediction than an in-sample residual?
- A dose target changes from 70 mg in the full-data fit to 84 mg when one arm is omitted. Calculate the relative change using the full-data target as the reference, and state what this does and does not show.
Solutions. 1. \(r=5.4-6.1=-0.7\) response units; the observed response is 0.7 units below the fitted prediction. 2. The contrast is \(4.0-3.2=0.8\) units, and its standard error is \(\sqrt{0.3^2+0.4^2}=\sqrt{0.25}=0.50\) units. 3. The omitted observation did not help estimate the parameters used to predict it, so the comparison is less directly constrained by fitting that observation; uncertainty and dependence still need consideration. 4. The relative change is \((84-70)/70=0.20\), or 20%. This shows sensitivity of the estimated target to that arm, not that either fit is correct or that the arm should be removed.