Common Errors in Cross-Model Evidence Integration

Why Integration Errors Matter

An integrated evidence chain can be numerically tidy and still be misleading. A model may pass an exposure summary to a response model, which passes an estimate to a trial-outcome model. At each handoff, a mismatch in what is being estimated, an unrecorded dependence, or discarded uncertainty can make the final prediction look more certain—or more coherent—than the underlying evidence warrants. The concern is not that different models must agree. It is that any comparison or combination must preserve the meaning and uncertainty of the quantities being connected.

This tutorial focuses on five recurring errors: combining incompatible exposure metrics, treating correlated estimates as independent, reporting unjustified parameter precision, counting the same data more than once, and losing uncertainty when passing estimates between models. These failures can overlap. For example, reusing data can create dependence, while passing only a point estimate can conceal both its uncertainty and its dependence on other inputs.

Check Whether the Quantities Match

An exposure metric is part of the definition of a model input, not merely a label for a concentration summary. AUC, measured in concentration-by-time units, and \(C_{\max}\), measured in concentration units, describe different aspects of a concentration-time profile. A response model using AUC and one using \(C_{\max}\) therefore do not have interchangeable slopes. Their coefficients have different units and generally answer different questions.

A useful diagnostic is to write the target, population, regimen or scenario, endpoint, and units beside every quantity at an interface. Then check whether the upstream model’s output is exactly the input the downstream model requires. If the quantities differ, the integration needs an explicit transformation or separate predictions on a common scale. Sensitivity to the choice of exposure definition can reveal fragility, but it does not itself establish which metric is correct.

Account for Dependence When Combining Estimates

Two model estimates may share participants, measurements, fitted parameters, or other evidence. Their errors can therefore be correlated. For a weighted sum of two estimates \(T_1\) and \(T_2\), the variance is:

The covariance term is omitted only when independence is justified, or when the term is known to be zero for another reason. Positive covariance often means that the estimates contribute less independent information than a calculation assuming independence would suggest. Negative covariance can have the opposite effect. Dependence should be determined from how the estimates were generated, not chosen to make a combined result more convenient.

For two estimates, the result follows by minimizing the variance of \(wT_1+(1-w)T_2\). The weights sum to one, so the combination remains unbiased for their shared target. Differentiating that variance with respect to \(w\), setting the derivative to zero, and solving gives the covariance-aware weights above. If the covariance matrix is misspecified, the resulting precision is also misspecified.

Do Not Manufacture Precision or Reuse Evidence

A parameter estimate passed from one model to another is uncertain unless its value is genuinely fixed by design or known without relevant uncertainty. Substituting an estimate as though it were a constant usually removes variation from the downstream result. Precision can also be overstated when a likelihood or summary is used twice as if each use came from new data. These are distinct errors: the first drops uncertainty at a handoff; the second inflates the apparent amount of information.

A point estimate is not automatically a fixed input. A sound handoff carries forward the needed uncertainty and dependence information, for example through joint parameter draws or a suitable joint covariance matrix. For nonlinear models, propagating joint draws can retain distributional features that a local approximation may miss. The choice depends on the model and available information; merely reporting more decimal places does not improve precision.

Diagnostics, Alternatives, and Limits

An integration audit begins with the information at each handoff, not with the final standard error. Check the target and units; identify overlapping data and shared model components; record whether each input is an estimate, a fixed quantity, or a distribution; and inspect how uncertainty is propagated. Compare an integrated prediction with relevant model-specific predictions and examine whether conclusions change under plausible interface or dependence assumptions. Residual patterns and predictive checks can help locate a failing interface, but a lack of obvious discrepancy does not prove that data were independent or uncertainty was fully propagated.

The main alternatives address different problems. Aligning models on a common predicted outcome can allow comparison when their internal exposure metrics differ, but does not make their coefficients interchangeable. A covariance-aware combination is preferable to an independence calculation when shared information is quantified; if dependence is unknown, sensitivity analysis is more honest than silently setting covariance to zero. Joint simulation or another propagation method can preserve input uncertainty across a handoff, whereas passing only point estimates is simpler but generally less complete. These approaches cannot repair a mismatched target, duplicated likelihood, or incorrect model specification.

Some limitations remain even after these checks. Covariance may be difficult to estimate when models were fit separately or when only summary outputs are available. Approximate propagation can be unreliable for highly nonlinear transformations, bounded parameters, or strongly skewed distributions. Model agreement is not independent confirmation when models share data or assumptions. A careful analysis therefore reports what was combined, what dependence and interface assumptions were used, and which uncertainties could not be represented.

Common pitfall: “The two models give similar numbers, so they can be averaged and the combined standard error will be smaller.” Similar numbers do not establish a common estimand, and averaging does not create independent information. First verify the target and units, then account for covariance and any reused data. If those checks cannot be made, present the outputs separately or show how the conclusion varies under explicit assumptions.