Why Model Complexity Is a Decision Question
A model can be made more detailed in many ways: adding a covariate, allowing an effect to vary across groups, introducing additional compartments, or replacing a straight-line relationship with a curved one. Each addition can represent important biology or improve predictions. It can also require more data, make parameter estimates less certain, and complicate interpretation or implementation. The relevant question is not whether the more detailed model can fit the data better. It is whether its added detail improves the decision the model is intended to inform.
Building on the earlier material on choosing the required level of model resolution, this tutorial focuses on the next choice: how much complexity is warranted for that resolution and context of use. A model intended to describe an average trend may not need the same detail as one used to choose different doses for different patient groups. The decision rule and the consequences of its possible outcomes determine which differences between candidate models matter.
Compare Models Through the Intended Decision
Let \(M\) denote a candidate model, \(a\) a possible action, and \(s\) a possible state of the world relevant to the decision. For example, an action could be selecting a dose, and a state could be whether a patient attains a target without toxicity. Let \(L(a,s)\) be the loss associated with taking action \(a\) when state \(s\) occurs. Loss is a numerical representation of the decision consequences; lower loss is preferred. A model informs a decision by estimating how likely relevant states are under each action.
For a fixed model \(M\), the expected loss of action \(a\) is the sum of each state’s probability multiplied by its loss. The model-supported action is the action with the lowest expected loss, given the probabilities and losses used. Thus two models can differ in their fitted curves or parameter estimates but still support the same action. Conversely, a modest difference in predictions can matter greatly if it changes an action near a decision threshold.
Here \(R_M(a)\) is the expected loss for action \(a\) under model \(M\), and \(a_M^*\) is the action with the smallest expected loss. The calculation depends on the stated decision, possible states, probabilities, and losses. It does not make those inputs certain or objectively correct; they must be appropriate to the context of use.
A Short Justification for the Parsimony Rule
The decision-based comparison follows directly from the purpose of the model. Suppose a simpler model \(M_S\) has minimum expected loss \(R_S^*\), and a more complex model \(M_C\) has minimum expected loss \(R_C^*\). Let \(C\) represent the additional burden of using the complex model, expressed in the same units as loss if it can reasonably be quantified. The complex model is preferred on this criterion only if the reduction in decision loss exceeds that burden.
The complex model is preferred when this quantity is positive. To see why, compare total costs: using the simpler model costs \(R_S^*\); using the complex model costs \(R_C^*+C\). The latter is smaller exactly when \(R_C^*+C<R_S^*\), which rearranges to \(R_S^*-R_C^*-C>0\). If the burden cannot be credibly converted to the same units, this is a way to organize the reasoning rather than a formula for producing a precise score. In either case, uncertainty about the model’s predictions should be considered when estimating its decision value.
Worked Example: When a Covariate Enables a Better Dose Decision
Hypothetical illustration. A team is deciding whether to use a standard dose for everyone or to tailor the dose for two patient groups. A simpler model does not distinguish the groups; a more complex model represents group-specific probabilities. The decision analysis uses equal group sizes and the following stated loss: 10 loss units for failure to attain the treatment target and 4 loss units for toxicity. For each dose, assume the two outcomes’ probabilities and losses can be combined additively as specified below.
| Group | Dose | Probability of target failure | Probability of toxicity |
|---|---|---|---|
| Group A | 100 mg | 0.30 | 0.02 |
| Group A | 150 mg | 0.05 | 0.15 |
| Group B | 100 mg | 0.04 | 0.02 |
| Group B | 150 mg | 0.05 | 0.15 |
For a given group and dose, calculate expected loss as 10 times the probability of target failure plus 4 times the probability of toxicity. For Group A at 100 mg, the calculation is \(10(0.30)+4(0.02)=3.00+0.08=3.08\) loss units. At 150 mg, it is \(10(0.05)+4(0.15)=0.50+0.60=1.10\) loss units. The lower-loss choice for Group A is 150 mg.
For Group B at 100 mg, expected loss is \(10(0.04)+4(0.02)=0.40+0.08=0.48\) loss units. At 150 mg it is \(10(0.05)+4(0.15)=1.10\) loss units. The lower-loss choice for Group B is 100 mg. A group-specific model therefore supports different doses.
Under the simpler model, the dose must be the same for both equally sized groups. At 100 mg, the average target-failure probability is \((0.30+0.04)/2=0.17\), and the toxicity probability is 0.02. Expected loss is \(10(0.17)+4(0.02)=1.70+0.08=1.78\) loss units. At 150 mg, the corresponding probabilities are 0.05 and 0.15, giving \(10(0.05)+4(0.15)=1.10\) loss units. The simpler model selects 150 mg for everyone.
The complex model’s average expected loss under its tailored choices is \((1.10+0.48)/2=0.79\) loss units. The reduction relative to the simpler model’s decision is \(1.10-0.79=0.31\) loss units per patient on average. Under the hypothetical probabilities and losses, the group distinction is decision-relevant. In an actual application, the team would also need to judge whether the data support estimating that distinction well enough to use it.
When Added Complexity Does Not Help
A more flexible model may fit observed data more closely yet leave the decision unchanged. This is especially likely when the intended decision depends on a threshold and both models place their predictions on the same side of it. If no plausible prediction from the more complex model changes the action, its extra detail may have little decision value for that use, even if it improves a general fit statistic.
The following example uses a stated rule that is part of the hypothetical decision. It does not claim that the rule is appropriate for any particular drug development program. The predictions and uncertainty ranges are illustrative model outputs, not results from a real study.
Worked Example: More Detail Without a Different Decision
Hypothetical illustration. A team uses predicted target attainment at a specified exposure to decide whether a candidate dose meets a target of at least 0.80. A simple model predicts target attainment of 0.86, with a plausible range from 0.83 to 0.89. A more complex model, which adds a group interaction, predicts 0.87, with a plausible range from 0.76 to 0.95.
Using the stated rule on the point predictions, the simple model supports meeting the target because \(0.86 \geq 0.80\). The complex model also supports meeting it because \(0.87 \geq 0.80\). The point-prediction difference is \(0.87-0.86=0.01\), or 1 percentage point. It does not change the action under this rule.
The complex model’s wider range crosses the threshold: \(0.76<0.80<0.95\). That makes its decision implication less stable across the stated plausible predictions, while the simple model’s range remains above the threshold because \(0.83>0.80\). This example does not prove that the simple model is correct. It shows why a small improvement in the point prediction is not, by itself, evidence that the extra parameter helps this decision. The team would need a decision-relevant reason for the additional detail and adequate support for its estimates.
This distinction also clarifies how to think about model fit. A fit measure summarizes some aspect of agreement between a model and data; it does not automatically quantify the consequences of using that model for a specified action. A model can improve average fit in regions irrelevant to the decision, or gain flexibility by estimating relationships that the available data do not distinguish reliably. Those improvements may not justify added complexity for the stated use.
Diagnosing Whether Complexity Is Earned
No single count of parameters determines whether a model is too complex. Complexity also depends on how parameters interact, the structure imposed, the range of data supporting a prediction, and the decision the model must inform. A useful assessment asks what the extra feature is intended to represent, what evidence supports estimating it, and whether it can affect the choice of action.
Specify the actions, decision rule, and relevant consequences. Without these, “better” has no decision-specific meaning.
Name the extra covariate, interaction, compartment, or relationship and explain why it might matter for the intended use.
Consider whether the relevant groups, dose range, or outcomes are represented well enough to estimate the added feature. Sparse or uninformative data can leave extra parameters weakly determined.
Apply the same actions, losses, and decision rule to both models. Ask whether the recommendation changes and whether the change reduces decision loss.
Consider whether added uncertainty, data needs, explanation, or implementation could outweigh the decision benefit. If a recommendation changes under small reasonable changes in model output, treat that as a warning that the added detail may not be decision-robust.
Potential warning signs include implausible parameter estimates, large changes in recommendations when a small amount of information is altered, and a complex model’s decision depending on a sparsely observed subgroup. These are reasons to investigate the model and its data support, not automatic proof that the simpler model is preferable. The reverse warning also matters: if a simpler model averages over a meaningful patient difference and that average leads to avoidable harm, parsimony alone is not a sufficient defense.
Burden, Uncertainty, and a Practical Comparison
The burden of complexity can be scientific and operational. A model may need additional data, be harder to communicate, or require inputs that are unavailable when the decision is made. A more complex structure can also make predictions less certain, especially when its added parameters are weakly informed. These considerations should be connected to the intended use rather than treated as abstract objections to complexity.
The next example illustrates a transparent, hypothetical comparison in which the decision analysis already accounts for predictive uncertainty. The additional implementation burden is assigned a value in the same loss units solely to show how it affects the comparison.
Worked Example: Decision Benefit Versus Added Burden
Hypothetical illustration. For two candidate models, suppose a decision analysis that includes uncertainty in predictions gives minimum expected losses of 2.40 units for a simple model and 2.25 units for a more complex model. The complex model also requires an additional workflow step. For illustration, the team assigns that burden 0.20 loss units.
The expected-loss reduction from the complex model is \(2.40-2.25=0.15\) units. After accounting for the assigned burden, its net advantage is \(0.15-0.20=-0.05\) units. Under these stated inputs, the complex model is not preferred: its reduction in expected loss is smaller than the additional burden. If the burden were instead 0.10 units, the net advantage would be \(0.15-0.10=0.05\) units, so the complex model would be preferred on this criterion.
The arithmetic is straightforward; the judgment lies in whether the losses and burden are defensible for the context of use. If implementation burden cannot be expressed credibly in loss units, the same comparison can be stated qualitatively: does the decision improvement justify the extra work and uncertainty? The numbers in this example are not universal weights or recommended thresholds.
The main alternatives are not simply “simple” and “complex.” A team can retain a parsimonious primary model while using a more detailed candidate to examine whether an important decision-relevant feature is being omitted. It can also gather data that better distinguish the alternatives before relying on a more detailed structure. These options do not remove the need to state the intended use, but they can help avoid both unjustified complexity and unjustified simplification.
There are limits to the decision-based comparison. Its conclusions depend on the states, probabilities, actions, and losses included; an omitted consequence cannot influence the calculation. A model that has a lower expected loss under stated assumptions is not automatically reliable for every population or use. Nor does a small expected-loss difference settle whether the inputs are sufficiently supported. The comparison organizes a fit-for-purpose rationale; it does not replace careful evidence about the model and its intended application.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A more complex model improves a global fit measure, but both models recommend the same dose under the stated decision rule. What additional argument would justify the complex model for that decision?
- For an action, the probabilities of target failure and toxicity are 0.08 and 0.03. Losses are 10 units per target failure and 4 units per toxicity. Calculate expected loss under the additive-loss assumption.
- A complex model reduces expected decision loss by 0.12 units, and its added burden is assigned 0.15 units. Which model is preferred under the net-advantage criterion?
- Why might an added subgroup parameter be risky when only a few observations are available in that subgroup?
Solutions. 1. The complex model needs a decision-relevant benefit beyond global fit—for example, representing an important subgroup difference that could alter consequences or a decision not captured by the current rule. If it cannot affect the intended decision and has no other specified use, improved fit alone may not justify it. 2. Expected loss is \(10(0.08)+4(0.03)=0.80+0.12=0.92\) units. 3. Net advantage is \(0.12-0.15=-0.03\) units, so the simpler model is preferred under the stated criterion. 4. Sparse subgroup data may not determine the added parameter reliably, making its predictions and any subgroup-specific recommendation unstable. The issue is not that subgroup models are inherently unsuitable, but whether the available evidence supports the detail needed for this use.