Why Parameter Flexibility Can Bias a Comparison
A QSP mechanism with more adjustable parameters can often reproduce calibration data at least as closely as a simpler alternative. That improvement is not, by itself, evidence that the mechanism is biologically better supported. The additional parameters may be capturing signal, but they may instead be absorbing noise, compensating for misspecified processes, or exploiting freedom that the competing model did not receive.
A fair comparison therefore asks more than which model has the smallest residual sum of squares or largest maximized likelihood. It asks whether the alternatives were given comparable opportunities to fit, whether the parameter count reflects the fitted model, and whether any apparent advantage survives a penalty for complexity or evaluation on data not used for calibration. The earlier material on information criteria introduces the main criteria; here the emphasis is on controlling the flexibility that enters their use.
Matching does not mean forcing biologically different mechanisms to have identical parameters. It means avoiding accidental advantages. For example, if both mechanisms predict the same biomarker observations with the same residual model, that observation model should not be more flexible in only one candidate. If one mechanism genuinely requires an additional rate constant, the added parameter should remain, but its role and contribution to fit should be visible and considered in the comparison.
Assumptions and a Complexity-Aware Fit Comparison
Information criteria such as AIC compare fitted likelihoods while accounting for model dimension. In the regular maximum-likelihood setting, AIC is defined as \(AIC=-2\ell(\hat{\theta})+2k\), where \(\ell(\hat{\theta})\) is the maximized log-likelihood under the model and \(k\) is the number of estimated parameters counted under the criterion’s assumptions. The candidate models must be fitted to the same observations using comparable likelihood definitions. The count should include estimated parameters in the likelihood, including an estimated residual variance or other error-model parameters, not just mechanistic rate constants.
The standard interpretation relies on regularity conditions: parameters are identifiable in the relevant region, the likelihood is suitably smooth, the fitted solution is an appropriate maximum, and the likelihood model is a defensible description of the observations. Boundaries, weak identifiability, heavy regularization, or highly flexible random-effects structures can make the nominal count \(k\) a poor description of effective complexity. AIC is a relative estimate of expected predictive information loss under its conditions; it is not a probability that a model is true.
The central calculation is particularly clear for nested models, in which the simpler model is obtained by fixing \(q\) parameters of the more flexible model. If both are maximized over compatible parameter spaces, the flexible model cannot have a lower maximized likelihood. Its AIC difference is:
The flexible model is preferred by AIC only if its gain in log-likelihood exceeds the complexity cost: \(\ell_{\mathrm{flex}}-\ell_{\mathrm{simple}}>q\). Equivalently, its reduction in deviance, \(2(\ell_{\mathrm{flex}}-\ell_{\mathrm{simple}})\), must exceed \(2q\). This is a penalty-based comparison, not a claim that every added parameter is biologically unnecessary when the improvement falls short.
Worked Example: A Fit Improvement That Does Not Offset Complexity
Suppose two hypothetical QSP candidates are fitted to the same 20 biomarker observations with the same Gaussian observation model. The simpler model estimates \(k=2\) parameters and has deviance \(-2\ell=15.862944\). A flexible alternative estimates \(k=6\) parameters and has deviance \(-2\ell=20\ln(1.6)\). These deviance values are on the same likelihood scale.
For the flexible model:
For the simpler model:
Thus \(AIC_{\mathrm{flex}}-AIC_{\mathrm{simple}}=21.40007258\ldots-19.862944=1.53712858\ldots\), or about 1.537 AIC units. The more flexible candidate has a better maximized likelihood, but its four additional parameters do not earn enough improvement to offset the AIC penalty. The result is modest relative support for the simpler model by AIC, not proof that its mechanism is correct.
Match the Calibration Before Interpreting the Penalty
A complexity penalty cannot repair a comparison that gave the candidates different data or likelihoods. Before comparing criteria, check that each candidate uses the same observations, weighting, error definitions, and treatment of missing or censored measurements. Confirm that shared parameters have the same biological meaning and are calibrated to the same evidence. If one candidate was calibrated with a different subset or a more accommodating residual model, its fit score is not directly comparable without an explicit justification.
The parameter count also needs care. Count every freely estimated quantity used by the likelihood, including shared nuisance and error parameters. Do not count a parameter twice merely because it appears in multiple equations, and do not omit it because it is not mechanistic. Conversely, a fixed quantity supplied externally is not an estimated parameter simply because it appears in the model. Report the convention used, especially when a model has latent variables, hierarchical components, or a constrained parameterization.
Worked Example: Checking Matched Parameterization
In a hypothetical calibration, two pathway models are fitted to 24 concentration measurements. Both use the same assay error model and estimate a baseline concentration and a residual standard deviation. The feedback model additionally estimates a feedback strength; the feedforward model instead estimates a feedforward strength. The fitted deviances are 31.2 and 27.8, respectively. Each has three estimated parameters: the two shared quantities and one mechanism-specific strength.
The corresponding AIC values are \(31.2+2(3)=37.2\) for feedback and \(27.8+2(3)=33.8\) for feedforward. Under this matched setup, the AIC difference is 3.4 units in favor of the feedforward candidate. The comparison is interpretable because the data, error model, and parameter counts are aligned. It does not establish that feedforward causally governs the biology: it indicates relative support within the specified candidates and assumptions.
Suppose instead the feedforward candidate had also been given a freely estimated time-varying assay bias, while the feedback candidate used a fixed bias. Its lower deviance would then reflect both the mechanistic difference and an extra opportunity to fit. The remedy is not automatically to delete a scientifically justified parameter. Rather, apply comparable observation-model freedom where justified, identify the difference transparently, and evaluate how much the conclusion depends on it.
Bounds, Priors, and Calibration Freedom
Bounds define the region over which calibration searches. A narrow bound can prevent a model from reaching a good fit; a broad bound can allow extreme values that compensate for an incorrect mechanism. Bounds should be scientifically justified and applied consistently to parameters with comparable meanings. A candidate whose optimum lies on a bound deserves a sensitivity check: the boundary may be active, the optimum may be constrained by the chosen range, or the data may not identify the parameter away from it. Boundary estimates are not automatically invalid, but they should not be presented as ordinary interior estimates.
Priors also affect calibration when the fitted point is a maximum a posteriori estimate. A prior-weighted objective is not the same quantity as a maximized likelihood: a parameter value can be favored because it has high likelihood, because it is favored by the prior, or both. Standard AIC uses a maximized likelihood under its regularity assumptions, not a prior-penalized objective substituted without adjustment. If prior information is used, state which parameters it constrains and report the likelihood contribution separately from the prior contribution. Using the same prior specification for shared quantities helps avoid asymmetric freedom, but it does not make the prior irrelevant to a posterior-based fit.
Worked Example: A Boundary-Driven Apparent Advantage
Consider hypothetical measurements of a response at 12 time points. Two candidates have the same number of parameters and use the same Gaussian error model with known residual standard deviation \(\sigma=2\) response units. The feedback model has a sum of squared residuals of 48 response-units-squared. A feedforward model has a value of 36 when its rate parameter is allowed up to 10 per hour, but its fitted rate is exactly 10 per hour; the prespecified biologically plausible upper bound was 4 per hour.
With known \(\sigma\), the difference in the deviance terms attributable to the residuals is the difference in squared residuals divided by \(\sigma^2\). Thus the feedforward fit’s deviance reduction relative to feedback is \((48-36)/2^2=12/4=3\). Since the parameter counts are equal, its AIC is lower by 3 units under this calculation. But the result used a bound that conflicts with the stated plausible range, and the optimum is at that bound. It would be misleading to treat this score as clean evidence favoring the mechanism.
Refit using the justified range and examine the resulting fit and parameter estimate. If the feedforward fit worsens, report that sensitivity; if its optimum remains at the new boundary, the data may be pushing against the constraint or the model may be compensating for a mismatch. The hypothetical calculation diagnoses a comparison issue, not a reason to choose bounds after seeing which model wins.
Diagnostics and Alternative Approaches
A useful diagnostic is to compare each model’s fit improvement with the number and role of its additional parameters. Inspect estimates, uncertainty, correlations, and boundary hits; an added parameter that is weakly identified or highly correlated with another may contribute nominal flexibility without yielding a stable biological interpretation. Also check residual patterns and predictive behavior. A criterion can summarize a trade-off, but it cannot reveal whether a model systematically misses a time course, dose dependence, or biomarker pattern.
AICc adds a finite-sample correction to AIC in settings where its assumptions and sample-size definition are appropriate:
The correction can matter when the number of fitted parameters is not small relative to the number of independent observations. In QSP applications with correlated repeated measures or multiple data sources, \(n\) is not automatically the number of rows in a data table; the likelihood’s sampling structure determines what constitutes independent information. Cross-validation offers a different check by assessing prediction on withheld data, but folds must preserve relevant dependence and experimental structure. A parameter penalty and predictive validation answer related, not identical, questions.
Common failure modes include selecting the model with the smallest raw residual sum of squares, counting only mechanistic parameters while ignoring estimated error terms, comparing penalized calibration objectives as if they were log-likelihoods, and interpreting a small criterion difference as decisive. AIC, AICc, and cross-validation can disagree because they embody different approximations and targets. If choices depend on a bound, prior, error model, or parameter-count convention, show that sensitivity rather than hiding it in one score.
Use the same evidence, likelihood, observation model, and calibration conventions wherever scientifically appropriate.
List estimated, fixed, bounded, and prior-constrained quantities, and state how each candidate’s complexity is counted.
Calculate the chosen criterion from the appropriate likelihood and parameter count; do not substitute a different objective without explanation.
Check boundary estimates, identifiability, residual structure, and predictive performance under defensible alternative choices.
Check Your Understanding
Answer each question, then compare with the concise solutions below.
- A nested model adds two parameters and improves maximized log-likelihood by 1.4. What is the AIC difference, flexible minus simple?
- Why should an estimated residual standard deviation generally be included in the parameter count for AIC?
- A candidate’s best-fitting rate is exactly at its upper bound. Name two checks before interpreting its fit advantage.
- Why is a prior-penalized calibration objective not automatically interchangeable with maximized likelihood in an AIC calculation?
- Two models have equal AIC but very different residual patterns. What does that tell you about relying on the criterion alone?
Solutions. (1) \(-2(1.4)+2(2)=-2.8+4=1.2\), so AIC favors the simpler model by 1.2 units. (2) The residual standard deviation is estimated from the data and contributes to the fitted likelihood, so omitting it understates fitted complexity under the standard count. (3) Check whether the bound is scientifically justified and whether the fit or estimate changes under defensible bounds; also inspect identifiability and whether the optimizer reached a reliable solution. (4) The prior contributes a term distinct from the likelihood, whereas standard AIC is based on maximized likelihood under its assumptions. (5) Equal AIC does not imply equal adequacy: inspect residual diagnostics and predictive performance because a summary criterion can conceal systematic misfit.