Comparing Mechanisms With Similar Response Curves

Why Similar Trajectories Require Careful Comparison

Different indirect-response mechanisms can produce response curves that look nearly alike over the observation window. For example, a drug may reduce response by inhibiting its production, or by stimulating its loss. If observations are sparse, noisy, or concentrated in a limited part of the trajectory, both mechanisms may fit well. The question is therefore not simply which curve looks better, but whether the available data support distinguishing the mechanisms at all.

This comparison requires three related but distinct judgments: how well each model fits the observations, how uncertain its parameter estimates are, and whether the sampling design captured times at which the models predict different responses. A preferred model under one criterion is not automatically a proven biological mechanism.

Specify the Candidates and the Observation Model

Consider a response \(R(t)\) with baseline value \(R_0=k_{\mathrm{in}}/k_{\mathrm{out}}\), where \(k_{\mathrm{in}}\) is the baseline production rate and \(k_{\mathrm{out}}\) is the baseline loss-rate constant. Let the concentration profile \(C(t)\) be available to both candidate models. For a declining response, one plausible pair is inhibition of input and stimulation of loss:

Here \(I_{\max}\) and \(S_{\max}\) are dimensionless maximum drug effects, while \(IC_{50}\) and \(SC_{50}\) have the same concentration units as \(C(t)\). The models share the untreated baseline turnover structure but represent different drug actions. Their predicted responses, denoted \(f_m(t_i;\theta_m)\) for model \(m\), are obtained by solving the corresponding differential equation with its parameter vector \(\theta_m\).

For a basic likelihood comparison, assume observations \(y_i\) at times \(t_i\) satisfy \(y_i=f_m(t_i;\theta_m)+\epsilon_i\), with independent errors \(\epsilon_i\sim N(0,\sigma^2)\). The response units must be the same across models, and both models must be fitted to the same observations using a comparable error model. If the data show proportional error, serial correlation, or another residual structure, the likelihood should represent that structure rather than treating the errors as independent with constant variance.

Under the stated Gaussian model, the log likelihood is

The parameters are estimated by maximizing this likelihood. A larger maximized log likelihood means a better fit to these data under this observation model, but it does not by itself account for differences in model complexity or establish that the mechanism is true.

Likelihood Comparison When Mechanisms Are Nonnested

Input inhibition and loss stimulation are generally nonnested alternatives: neither is obtained from the other merely by fixing one parameter to a special value. Thus, a difference between their maximized likelihoods cannot ordinarily be treated as a likelihood-ratio test with a chi-square reference distribution. The familiar nested-model likelihood-ratio test requires specific nesting and regularity conditions; those conditions should not be assumed for mechanistic rivals.

A common comparison is Akaike’s information criterion, \(AIC=-2\ell_{\max}+2k\), where \(k\) is the number of estimated parameters, including the residual variance when it is estimated. Smaller AIC indicates a better trade-off between fit and parameter count, among the models being compared. AIC differences are relative evidence for these candidates, not probabilities that a mechanism is correct.

For Gaussian errors with a separately estimated common-form variance, the maximum-likelihood estimate is \(\hat{\sigma}^2=RSS/n\), where \(RSS\) is the residual sum of squares. Substituting this estimate into the log likelihood gives the following AIC difference:

This expression makes the comparison transparent: a lower residual sum of squares favors a model, while additional estimated parameters incur a penalty. When the sample is small relative to the number of fitted parameters, the corrected criterion \(AIC_c\) may be more appropriate, provided it can be calculated. The criterion must be chosen and interpreted consistently across the candidates.

Parameter Uncertainty Is Not the Same as Model Uncertainty

A model can fit well while its parameters remain poorly determined. Within a candidate mechanism, parameter uncertainty can be examined using standard errors, confidence intervals, or profile likelihoods. A profile likelihood for one parameter fixes it at a sequence of values and re-estimates the remaining parameters at each value. A broad profile indicates that a range of parameter values fits nearly as well as the optimum.

Parameter intervals are conditional on the model being fitted. Overlap between an \(IC_{50}\) interval under input inhibition and an \(SC_{50}\) interval under loss stimulation does not demonstrate that the mechanisms are equivalent; the parameters describe different actions. Conversely, a narrow interval for one parameter does not establish that its model is better than a rival. Compare model fit and within-model uncertainty as separate questions.

Why Sampling Time Can Determine the Answer

Mechanisms may differ most during a short phase of the response, yet routine sampling may miss that phase. A useful design therefore considers the predicted separation between candidate trajectories at each possible sampling time, relative to the observation error and uncertainty in fitted parameters.

A simple justification comes from the expected log-likelihood ratio for one observation under two fixed predictions. Suppose model A predicts \(\mu_A\), model B predicts \(\mu_B\), and the observation has normal error with known standard deviation \(\sigma\). If A is the generating model, then \(Y=\mu_A+\epsilon\), where \(E(\epsilon)=0\). The log density ratio is

Taking the expectation under model A removes the term involving \(E(\epsilon)=0\), leaving \(E[\log\{f_A(Y)/f_B(Y)\}]=(\mu_A-\mu_B)^2/(2\sigma^2)\). Thus, a time with a larger prediction difference relative to the residual standard deviation can provide more expected discrimination. This calculation holds predictions fixed; when parameters are refitted, nuisance-parameter uncertainty can reduce the realized separation.

Diagnostics, Alternatives, and Limits

Before interpreting a likelihood comparison, inspect whether the fit is credible for each candidate. Plot residuals against time and predicted response, check for systematic runs or changing spread, and examine whether fitted trajectories follow the observed onset, minimum, and recovery. A nominal AIC advantage is not persuasive if the preferred model leaves a clear time-dependent residual pattern or if its optimum lies at a parameter boundary. Convergence problems and sensitivity to starting values can also make the reported maximum likelihood unreliable.

The main alternatives answer different questions. A nested likelihood-ratio test is appropriate only when one model is a regular special case of another and its assumptions hold. AIC compares relative expected predictive information loss with a complexity penalty; a Bayesian information criterion uses a different penalty and can rank models differently. Cross-validation can compare predictive performance on held-out observations, but may be unstable with a small dataset or an unrepresentative split. None of these criteria removes uncertainty due to sparse sampling, misspecified error, or omitted mechanisms.

A further limitation is that a shared concentration profile may itself be uncertain. If concentration predictions are treated as exact when they are not, apparent separation between response mechanisms can be overstated. Likewise, distinguishing the best-fitting curve among a limited set of candidates does not establish that the true biology is one of them. Independent mechanistic evidence and a design targeted to the rival predictions can strengthen interpretation.