Diagnostics for Biomarker Exposure-Response Models

Why Diagnose a Biomarker Exposure-Response Model?

A fitted exposure-response model can produce a plausible average curve while systematically missing important features of the data. It may predict responses that are too early or too late, miss a change in response across exposure levels, or fit well only because one unusual observation pulls the curve toward itself. Diagnostics help reveal these failures before the model is used to interpret biomarker effects or inform a development decision.

This tutorial focuses on three complementary tools: residual plots, observed-versus-predicted displays, and predictive checks. None proves that a model is correct. Each can show patterns inconsistent with the model assumptions and help locate where further investigation is needed. In early-phase studies, the number of participants and exposure levels may be limited, so plots should be interpreted alongside the sampling schedule, assay performance, baseline and time-course controls, and the model’s intended use.

Residuals and the Patterns They Reveal

A residual is not the same as the unobserved error in the data-generating model: model parameters were estimated from the same observations, so residuals can be smaller or more structured than the underlying errors. Still, a well-specified mean model with an appropriate error model should generally leave residuals without a systematic pattern against variables that the model is supposed to explain.

Before interpreting a residual display, state the relevant assumptions. For an additive-error model, one might assume \(Y_i = f(C_i,t_i;\theta)+\epsilon_i\), where \(f\) is the biomarker model, \(C_i\) is exposure, \(t_i\) is time, \(\theta\) is the parameter vector, and errors have mean zero conditional on the modeled predictors. A residual pattern against time suggests that the conditional mean may be missing time structure; a pattern against exposure suggests that the exposure-response relationship may be inadequate. If the model instead assumes response variability changes with the mean, raw residual spread may change even when the variance model is correct. In that case, inspect residuals scaled by their modeled standard deviations as well as raw residuals.

In repeated-measures data, observations from one participant are not automatically independent. If the model includes participant-level effects or a residual correlation structure, the diagnostics should respect that structure. A run of positive residuals within one participant can reflect a participant-specific offset or time course, not necessarily a population-wide exposure-response failure. Plot points by participant or connect each participant’s observations to make that structure visible.

Three Complementary Displays

A residual-versus-time plot places residuals on the vertical axis and sampling time on the horizontal axis. A smooth trend, repeated sign changes, or a systematic early-positive/late-negative pattern can indicate a misspecified time course. For example, a direct-effect model may fail to represent delayed response, while a turnover or effect-compartment model may capture delay but still miss the observed return toward baseline. If the model includes a time trend or a time-varying effect, the diagnostic question is whether that structure adequately describes the observations, not whether residuals are unrelated to time in every possible way.

A residual-versus-exposure plot can reveal a response curve that is too shallow, too steep, or incorrectly shaped. Plotting against observed exposure is often useful for examining the fitted exposure-response relationship, but exposure and time can be strongly related in a dosing study. A pattern against exposure may therefore also reflect a time-course error. Use the sampling schedule and the corresponding time display to distinguish these explanations where the design allows it.

An observed-versus-predicted plot places \(\hat{y}_i\) on the horizontal axis and \(y_i\) on the vertical axis. The reference line \(y_i=\hat{y}_i\) represents exact agreement. Points consistently above the line indicate underprediction; points below it indicate overprediction. A curved departure from the line or increasing scatter at larger predictions can indicate a mean-model or variance-model problem. The plot should use predictions defined consistently: population predictions show how well the typical model follows the data, while predictions incorporating participant-specific fitted effects answer a different question. State which is displayed, since a close fit after using individual fitted effects does not by itself show that the population relationship is adequate.

Influential Observations and Predictive Checks

An observation is influential when it has a substantial effect on fitted parameters or model predictions. A large residual alone does not establish influence: a point can be poorly predicted yet have little effect on the fitted curve, or a point with a moderate residual can strongly affect a parameter if it occurs in a sparsely sampled exposure region. A practical diagnostic is to refit the model after omitting one observation or one participant at a time, then compare key parameter estimates and predictions over the exposure-time range relevant to the study. This is a sensitivity analysis, not an automatic rule for deleting data.

Before treating an unusual point as a model failure, verify data handling and context: assay flags, units, dose and sample times, baseline values, and any protocol deviations. If the point is valid, report how conclusions change with and without it, and consider whether the error model or participant structure is inadequate. Removing a valid observation just because it disagrees with the model risks concealing a real biological feature.

A predictive check asks whether the observed data look plausible among datasets generated from the fitted model. Simulate replicate datasets under the model, including the assumed variability and study design, and compare features chosen in advance: for example, response trajectories over time, distributions of participant-level maximum response, or the spread of responses at low and high exposure. The simulated data must reproduce the relevant design features, including sampling times and exposure distribution; otherwise, the comparison may diagnose a difference in design rather than model adequacy. A predictive check is most informative when its summaries target the possible failure modes of interest.

A Short Justification for Residual Pattern Checks

The basic logic can be seen from the assumed mean model. If \(Y_i=f(x_i;\theta)+\epsilon_i\), where \(x_i\) collects exposure, time, and other predictors, and \(E(\epsilon_i\mid x_i)=0\), then at the true parameters: $$ E\{Y_i-f(x_i;\theta)\mid x_i\}=E(\epsilon_i\mid x_i)=0. $$

Thus, systematic residual direction across a region of \(x_i\) is inconsistent with the stated conditional-mean model, or with the assumptions used to obtain the predictions. In practice, parameters are estimated, so fitted residuals are not independent and need not have exactly zero average in every subgroup. A visible pattern is therefore a prompt to investigate, not a standalone test that identifies the cause. The model may be wrong, the variance or correlation assumptions may be wrong, or a data issue may be present.

Diagnostics in Practice: Interpretation and Limits

A useful diagnostic sequence is to examine observed-versus-predicted agreement, then residuals against time and exposure, then participant-level patterns, and finally targeted predictive checks. This sequence is not a mechanical decision rule. It organizes questions: Is there overall bias? Does it vary with time or exposure? Is it concentrated in a participant or region? Can the fitted model reproduce the feature that motivated the analysis?

Alternatives answer different questions. A numerical goodness-of-fit summary compresses discrepancies but can hide their location; plots preserve structure but rely on visual judgment. A formal test can provide a calibrated test statistic only under its specified assumptions and may have little power in a small early-phase dataset. Predictive checks assess whether selected data features are reproduced but depend on the model and summaries chosen. These tools complement one another; no single display certifies a model.

Common failure modes include treating residuals as independent despite repeated measurements, confusing a time-course problem with an exposure-response problem when exposure and time are confounded, and relying on a close population-average fit while missing participant-level patterns. Small samples also make smooth trends and predictive-check tail areas unstable. Diagnostics can identify tension between data and assumptions, but they cannot establish biomarker causality, prove target engagement, or replace scientific judgment about assay quality and study design.