Using Available Evidence to Bound Model Complexity

Evidence Sets a Resolution Limit

A QSP model can represent biological processes at many levels of detail, but the availability of biological language does not mean the available evidence can support every detail. The practical question is not simply whether a mechanism is plausible. It is whether the evidence can constrain the mechanism at the resolution needed for the model’s intended decision.

Evidence can constrain different things to different degrees. A clinical concentration-time profile may constrain systemic exposure while saying little about drug concentration in a particular tissue. A biomarker that changes after treatment may establish a response pattern without identifying the pathway that caused it. A perturbation that changes a measured mediator and a downstream outcome may provide stronger mechanistic evidence, but only for the contexts, time scales, and populations actually studied.

Definition: Evidence constrains a model feature when its measurements or experimental contrasts restrict the plausible values or structures for that feature enough to matter for model predictions. Evidence resolves a feature when its type and resolution can distinguish that feature from relevant alternatives. Evidence may constrain an observed response without resolving the mechanism that produced it.

Resolution has several dimensions. Measurement resolution includes assay precision, detection limits, and whether the assay measures the proposed quantity or only a proxy. Temporal resolution depends on when observations are taken relative to the process’s likely time scale. Spatial resolution concerns whether measurements distinguish organs, tissues, or cell types that the model would represent separately. Population resolution concerns whether the data identify variation across relevant patients or only a pooled average. Perturbation coverage matters too: observations made under one condition may not distinguish mechanisms that behave differently when the system is challenged.

A useful boundary decision therefore asks two questions together: Can the evidence support this level of mechanistic detail? and Would representing that detail change a decision-relevant prediction? A mechanism can be biologically plausible but too weakly resolved to encode as a specific pathway. Conversely, an unresolved mechanism may still deserve explicit uncertainty if plausible versions materially change the decision.

From Evidence to Model Representation

For a candidate mechanism, compare a simpler model \(M_0\), which omits or aggregates the mechanism, with a more detailed model \(M_1\), which represents it explicitly. Evidence is informative about the added detail only if the observations are sensitive to a difference between these models. A good fit from \(M_1\) alone is not enough: if \(M_0\) can reproduce the same measured outputs, the data do not by themselves establish that the added mechanism is necessary.

In practice, evidence can support several different representation choices. A measured process with adequate resolution may be represented as an explicit state or process. A response that is clear but whose biological cause is not distinguished may be represented as a lumped effective process. A potentially important process with several plausible explanations may be handled through alternative scenarios or parameter ranges. A process can be left outside the model when it has no material effect on the intended outputs across the plausible conditions considered. Leaving it out is a scope decision, not a claim that the process does not exist.

Working rule: Add mechanistic detail when the available evidence can constrain that detail and the distinction matters to the decision. When the effect is decision-relevant but the mechanism is unresolved, preserve the uncertainty rather than choosing one explanation as established. When plausible versions do not change the decision-relevant predictions, a simpler boundary may be defensible.

This rule does not demand that every included process be directly measured. Some states may be inferred from multiple observations and a well-supported model structure. The key is to distinguish what the evidence directly observes, what it indirectly constrains, and what remains an assumption. Nor does the rule equate a large data set with strong mechanistic evidence: many precise measurements of one downstream output may still be unable to distinguish several causal pathways.

Worked Example: A Resolved Biomarker Response but an Unresolved Pathway

Suppose a hypothetical study measures a plasma biomarker in the same treatment group before dosing and at 2, 8, and 24 hours. The reported group means are 100, 82, 61, and 90 units, respectively. The assay’s repeatability coefficient of variation near 100 units is 5%, corresponding to about 5 units of repeatability variation. Assume the measurements are above the assay’s quantification limit and the time points are accurately recorded.

The observed fall from baseline to 8 hours is \(100-61=39\) units, and the 24-hour mean is \(100-90=10\) units below baseline. Both changes are larger than the assay’s approximate 5-unit repeatability variation. The sampling also shows that the biomarker is not simply measured once at baseline and once after recovery: an intermediate low value is observed. These data therefore resolve a suppression-and-recovery pattern in the measured plasma biomarker at the sampled times.

They do not, by themselves, resolve the intracellular mediator or tissue-specific pathway that produced this pattern. No tissue concentrations or candidate mediators were measured, and the four time points cannot establish the exact timing or shape of the unobserved response between samples. A defensible model could include a dynamic biomarker response if that response matters to the intended output. It should not treat one unmeasured intracellular pathway as established solely because it can explain the observed curve. If alternative pathways imply different decision-relevant predictions, those alternatives should remain explicit uncertainty; if they do not, a lumped response may be adequate.

Check Whether the Evidence Can Distinguish Alternatives

A short calculation can clarify the role of measurement variation. Suppose two independent groups have a measured outcome with sample standard deviations \(s_1\) and \(s_2\), and sizes \(n_1\) and \(n_2\). The standard error of the difference in group means is based on the variance of a difference of independent estimates:

SE(\bar{x}_1-\bar{x}_2) = \sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}.

This calculation helps assess whether a contrast is large relative to sampling variation under the stated design and assumptions. It does not identify the biological mechanism responsible for the contrast. A precise difference in a downstream biomarker can still be compatible with multiple mechanisms.

Worked Example: A Perturbation Resolves a Delayed Response

In a hypothetical experiment, 9 subjects per group receive either a control treatment or a perturbation intended to block a candidate feedback process. At 24 hours, the measured biomarker means are 78 units in the control group and 100 units in the perturbed group. Both groups have sample standard deviation 6 units. Assume independent subjects, approximately normal within-group measurements, and the same assay conditions in both groups.

The estimated difference, defined as perturbed minus control, is \(100-78=22\) units. Its standard error is

SE = \sqrt{\frac{6^2}{9}+\frac{6^2}{9}} = \sqrt{8} \approx 2.83\text{ units}.

For a two-sided 95% \(t\) interval with \(df=16\), use \(t^*\approx2.12\). The interval is \(22\pm2.12(2.83)=22\pm6.00\), or approximately 16.0 to 28.0 units. Under the assumptions stated, the observed group contrast is large relative to sampling variation, so the experiment supports a perturbation-sensitive difference in the 24-hour biomarker response.

The result makes a delayed process worth representing if that difference matters to the model’s decision. However, the perturbation result does not alone prove that a particular feedback pathway caused the entire difference. The perturbation could affect other processes, and the measured outcome is still a biomarker rather than a direct measurement of every causal link. The evidence can support a feedback-sensitive or otherwise perturbation-sensitive model feature while leaving its detailed biological interpretation uncertain.

Evidence resolution should be judged against the distinction the model needs to make, not against a universal threshold. If candidate mechanisms predict nearly identical observable outcomes under the available experiments, the data cannot discriminate them, even if the measurements are precise. If they predict different outcomes but observations are too sparse or noisy to separate those predictions, collecting more informative evidence may help. That evidence must be informative about the contrast: additional measurements of the same output at the same times may add little if the mechanisms differ only in an unmeasured tissue or under a different perturbation.

Use Decision Relevance to Bound Uncertainty

Weak evidence does not automatically justify either adding a detailed mechanism or excluding it. The next step is to test whether the unresolved feature can affect the model’s intended decision. Consider plausible representations of the feature, not just one preferred explanation. If the resulting predictions all lead to the same decision, detailed mechanistic representation may add complexity without useful decision information. If plausible representations lead to different decisions, the uncertainty is consequential and should not be hidden by selecting a single unsupported mechanism.

Worked Example: Sparse Tissue Evidence and a Decision-Sensitive Prediction

Suppose a hypothetical biomarker-guided dose decision depends on whether the predicted response reaches 30%. Human data measure plasma drug and the biomarker, but not drug concentration in the target tissue. A related animal experiment suggests that a tissue-mediated process may be relevant, yet the human tissue exposure is not established. Treat the following ranges as hypothetical model outputs under plausible alternatives, not as confidence intervals derived from the observed human data.

A model that omits the tissue-mediated process predicts responses from 22% to 27% across the considered dose range. The simpler model therefore remains below the 30% decision threshold, since its upper prediction is \(27\%<30\%\). Plausible versions that include the uncertain tissue process predict responses from 24% to 36%. This range crosses the threshold: \(24\%<30\%<36\%\). Thus the unresolved process can change the decision in at least some plausible representations.

The animal evidence and human plasma observations do not justify presenting a specific human tissue mechanism as established. But omitting the issue altogether would conceal a decision-sensitive uncertainty. A defensible approach is to retain alternative scenarios or a bounded effective process and show whether the dose decision changes across them. If future human evidence measures tissue exposure or otherwise distinguishes the alternatives, the representation can be narrowed. Until then, explicit uncertainty is more informative than false mechanistic precision.

Diagnostics, Alternatives, and Limitations

Several warning signs indicate that a proposed level of complexity exceeds what the evidence can support:

Proxy treated as mechanism: a downstream biomarker is measured, but a particular upstream pathway is claimed without evidence distinguishing it from alternatives.
Time scale missed: observations are too widely spaced to reveal a rapid transient, or too short to establish a delayed response or recovery.
Scale mismatch: a model separates tissues or cell types, while the evidence is only a pooled systemic measurement.
Condition mismatch: evidence comes from baseline conditions, but the model is used for a perturbation outside the studied range.
Apparent precision without discrimination: estimates are precise for an observed output, but multiple model structures reproduce that output.
Uncertainty hidden by a single choice: plausible mechanisms produce different decision outcomes, yet only one is represented without adequate evidence for preferring it.

The main alternative to an explicit mechanistic representation is not always omission. A lumped process can preserve a supported input-output relationship without asserting a detailed pathway. Scenario analysis can preserve alternative causal explanations when those explanations matter to the decision. A simpler model is preferable when the added detail is not distinguishable from existing components or does not alter decision-relevant predictions. A more detailed model is preferable when the evidence and intended use jointly justify the added distinctions.

These choices have limits. Evidence resolution is always relative to a model and a decision: a data set may be sufficient for one output but insufficient for another. A range of plausible predictions depends on the alternatives considered, so it cannot guarantee that every relevant mechanism has been included. Likewise, a lack of observed effect at the available resolution is not proof of a zero effect. Model boundaries should reflect what is known, what remains unresolved, and whether that unresolved part could alter the use of the model.

Key takeaway: Let evidence determine the level of detail, and let decision relevance determine whether unresolved detail must remain visible. A measured response can be well constrained while its biological cause remains uncertain; a mechanism may be omitted from the explicit model without being declared biologically absent.

Check Your Understanding

Answer each question using the distinction between evidence about an observed response and evidence about its mechanism.

A plasma biomarker is measured precisely at baseline and 24 hours, but a proposed pathway acts within minutes. What aspect of evidence resolution is the main concern?
Two candidate mechanisms produce the same measured clinical outcome under all observed conditions. Does a close fit to that outcome distinguish the mechanisms? Why or why not?
A poorly measured process has little effect on any decision-relevant prediction across plausible alternatives. What model treatment is defensible?
A process is not measured directly, but plausible versions move a predicted response from below to above a decision threshold. What should the model do with this uncertainty?
In the perturbation example, why does a 95% interval for the biomarker difference not establish the detailed feedback pathway?

Solutions. (1) Temporal resolution is insufficient to characterize a process that may act within minutes; two widely separated samples cannot establish its timing. (2) No. If both mechanisms imply the same observed output under the available conditions, that output does not discriminate their structures. (3) A simpler model may omit the explicit process, provided the conclusion is limited to the model’s scope and relevant conditions; omission does not assert biological absence. (4) Preserve the process as explicit uncertainty, such as alternative scenarios or a bounded effective representation, rather than selecting an unsupported mechanism or hiding its effect. (5) The interval quantifies uncertainty in the measured group contrast under the stated assumptions. It does not distinguish all biological explanations for that contrast or establish that the proposed pathway is the unique cause.