AICc and BIC for Competing PK/PD Structures

Why Compare PK/PD Structures With Information Criteria?

Competing PK/PD structures can describe the same observations in different ways. For example, a concentration–response model with a Hill coefficient may fit differently from a model that includes an indirect-response process. A more complex structure can improve fit, but its additional parameters may also make it less useful for prediction or less strongly supported when the data are limited. AICc and BIC summarize this trade-off by combining a likelihood-based measure of fit with a penalty for model complexity.

These criteria are useful when models are not nested, or when a likelihood-ratio test is not the intended comparison. They do not establish that a model is scientifically correct. Before calculating them, confirm that the candidates use the same response data and comparable likelihoods, and that each fit has converged to a meaningful maximum. Model-set construction and likelihood-ratio tests address related but distinct questions.

All three are on a scale where smaller values are preferred within the set being compared. The term \(-2\hat{\ell}\) measures lack of fit on the likelihood scale; the remaining terms penalize model complexity. AICc is defined by this formula only when \(n-K-1>0\). The correction grows as \(K\) approaches \(n-1\), reflecting the instability of fitting many parameters with limited information.

Here \(K\) includes every freely estimated parameter contributing to the likelihood, not just the parameters that define the PD curve. Depending on the model, that may include typical PK or PD parameters, covariate effects, random-effect variances and covariances, residual-error parameters, and estimated baseline or placebo terms. Parameters fixed in advance are not counted as estimated parameters. The precise count should match the fitted likelihood and the model implementation.

What the Penalties Mean

AIC is motivated by estimating expected predictive information loss: among candidates fit to the same data, it rewards fit while penalizing the optimism that comes from estimating parameters. AICc adds a finite-sample correction to AIC. The displayed AICc correction is exact in particular settings, including standard normal linear-regression derivations under their assumptions; in general nonlinear PK/PD models it is a commonly used correction, not a universal exact formula for finite-sample bias.

BIC has a different asymptotic motivation. Under regularity conditions, it approximates a criterion based on a model’s marginal likelihood, with lower-order terms omitted. For its usual large-sample interpretation, the likelihood must be regular enough for a local quadratic approximation around the maximum, the model dimension must be fixed as sample size grows, and the prior density (in the marginal-likelihood motivation) must be positive and smooth near the maximum.

The difference in BIC’s penalty can be seen directly. If \(n\) independent contributions provide information that grows proportionally to \(n\), the log-likelihood near its maximum is approximately quadratic. Integrating the likelihood over \(K\) parameters then contributes a factor proportional to \(n^{-K/2}\), in addition to the likelihood at the maximum and terms that do not grow like \(\log(n)\). Taking minus twice the logarithm gives a leading penalty of \(K\log(n)\). This is a large-sample approximation, not an exact equality for finite samples.

In contrast, AIC’s complexity penalty is \(2K\), independent of \(n\). AICc supplements that penalty with a term that depends on both \(K\) and \(n\). Consequently, AICc and BIC need not agree, particularly when the sample is modest or the candidates have different parameter counts.

Choose the Likelihood and Sample Size Carefully

Information criteria are meaningful as a comparison only when the candidate scores are calculated on the same observations and with compatible likelihood definitions. A likelihood must include its relevant terms consistently across models. A score computed from one data subset cannot be directly ranked against a score computed from another. For comparisons that change the fixed-effects structure, likelihoods based on different estimation conventions may also be ineligible for direct comparison.

In repeated PK/PD data, \(n\) is not automatically the number of recorded concentrations or responses. Measurements within a subject may be correlated through random effects or other dependence structures. The appropriate sample-size convention depends on the likelihood and the asymptotic framework used for the criterion. In some settings, individual observations contribute conditionally to a likelihood; in others, subjects are the independent sampling units. Use the software’s documented convention, apply it consistently across candidates, and state what \(n\) represents. Counting correlated records as independent can make a penalty misleadingly small.

AICc’s denominator also provides a practical warning. If \(n-K-1\) is zero or negative, the standard AICc expression is undefined; if it is small and positive, its correction can be very large. Neither situation is repaired merely by reporting the resulting number. It signals that the chosen finite-sample formula may not be appropriate for the available information and model dimension.

Worked Comparison When the Criteria Disagree

When model structures are not nested, a criterion can be calculated for each and the scores compared, provided the data and likelihood are comparable. The following two examples use the same hypothetical comparison to show both the arithmetic and why the choice of criterion matters.

Reading Rankings Without Overclaiming

A score difference is most useful as a measure of relative support within the candidate set and under the chosen criterion. The absolute value of an information criterion has little meaning by itself. If two candidates differ by a few points, report the actual scores, the difference, and the convention for \(n\) and \(K\), rather than turning the ranking into a binary declaration that one model is correct.

For AIC-based comparisons, quantities proportional to \(\exp(-\Delta_i/2)\), where \(\Delta_i\) is a candidate’s AIC difference from the minimum, are sometimes called relative likelihoods. Normalizing these quantities produces Akaike weights. These are criterion-based summaries within the specified candidate set; they are not automatically posterior probabilities that the corresponding models are true. BIC differences likewise are not posterior model probabilities. A Bayesian interpretation requires additional assumptions and prior information, which the criterion ranking alone does not supply.

For nested models, a likelihood-ratio test can answer a different question about improvement under a null restriction. AICc and BIC can compare non-nested candidates too, but neither substitutes for inspecting scientific plausibility or predictive adequacy. Predictive validation can provide another perspective, especially when the scientific aim is prediction for new subjects.

Diagnostics, Failure Modes, and Limitations

Before accepting a ranking, check that it is not an artifact of fitting or bookkeeping:

• Nonconvergence or a poor maximum. An optimizer that stops at a local or boundary solution can give an unreliable \(\hat{\ell}\). Review convergence diagnostics, parameter estimates, and the likelihood at plausible alternative starts.
• Weak identification. A nominal parameter count does not reveal that parameters may be poorly estimated or redundant. Examine uncertainty, correlations among estimates, and the model’s residual and predictive diagnostics.
• Incorrect \(n\) or \(K\). Inconsistent treatment of repeated observations, random effects, or residual parameters changes the penalty and can change the ranking.
• Incompatible likelihoods or data. Scores from different response subsets or incomparable likelihood definitions should not be ranked as if they were calculated on the same basis.
• A deficient candidate set. The lowest score identifies a relative preference among the models considered, not adequacy. If all candidates miss important features of the data, the minimum-score candidate can still be inadequate.

Information criteria also rely on approximations and do not automatically account for all structural uncertainty, parameter constraints, or repeated searching over many candidate models. A low AICc or BIC cannot repair misspecification, establish causation, or guarantee good predictions outside the observed dose and exposure range. Use the criteria alongside scientific reasoning and the diagnostics appropriate to the PK/PD data.