Why Regimen Choice Must Account for Model Uncertainty
A regimen that looks best under one exposure-response model may not remain best if key parameters change or a plausible alternative model structure is used. For example, a regimen’s apparent advantage could depend on a steep exposure-response relationship, while another regimen performs more consistently if the relationship is shallow. The choice is therefore not only about which regimen has the highest predicted value under a reference model. It is also about how the decision behaves across credible alternatives.
Robust regimen choice makes that comparison explicit. It treats each candidate regimen as having outcomes under several plausible scenarios, then applies a decision rule to those outcomes. The rule might protect against the worst predicted utility, limit the worst possible regret, or favor a regimen with a high probability of meeting prespecified clinical constraints. These rules answer different questions; none is automatically the right choice for every development decision.
Utilities must be on a consistent scale so comparisons across regimens and scenarios are meaningful. They may summarize multiple outcomes according to previously specified clinical priorities, or they may be a single decision-relevant outcome. If a clinical constraint is mandatory, such as an unacceptable safety risk being ruled out, it should be stated explicitly rather than silently traded against efficacy through an arbitrary utility score. The scenario set should likewise be justified: it represents plausible uncertainty relevant to the decision, not every imaginable possibility.
The comparison assumes candidate regimens are evaluated on the same target population, outcome definitions, and time horizon. Within each scenario, the utility predictions should be calculated consistently. A scenario analysis does not remove uncertainty in the underlying evidence; it organizes how uncertainty affects the decision.
Worst-Case Utility: Protecting the Lower Bound
The maximin rule calculates the lowest utility each regimen attains across the specified scenarios, then chooses the regimen with the highest such minimum. It is appropriate when poor performance in any credible scenario is especially concerning and the scenario set is considered decision-relevant. It does not require probabilities for the scenarios.
Worked Example: Choosing the Highest Worst-Case Utility
Suppose a development team compares three hypothetical regimens using a utility scale in which larger scores are preferred. The scores are illustrative utility points, not observed clinical results. The three scenarios represent a stronger exposure-response relationship (S1), a weaker relationship with greater toxicity impact (S2), and a plausible alternative model structure with delayed response (S3).
| Regimen | S1 | S2 | S3 |
|---|---|---|---|
| A | 82 | 58 | 70 |
| B | 75 | 68 | 62 |
| C | 90 | 40 | 76 |
For each regimen, take the smallest entry in its row:
The largest of these three worst-case values is 62, so the maximin choice is regimen B. This rule favors B because it avoids the particularly low utility that C has in S2, even though C has the highest utility in S1 and S3. The conclusion is conditional on the specified scenarios and the utility scale; it is not a claim that B will achieve a particular clinical outcome.
Minimax Regret: Limiting How Much Is Given Up
Worst-case utility focuses on each regimen’s lowest outcome. Regret instead asks how much utility would be lost, in each scenario, compared with the best regimen that could have been chosen if that scenario were known in advance. The best-in-scenario benchmark is not an attainable promise under uncertainty: the decision-maker must choose a regimen before learning which scenario applies.
For any regimen and scenario, regret is nonnegative because the scenario-specific maximum includes that regimen. The minimax-regret choice has a useful guarantee: if its maximum regret is \(r^*\), then in every included scenario its utility is no more than \(r^*\) below that scenario’s best available utility. To see this, define \(r^*=\min_{d\in D}\max_{s\in S}R(d,s)\), and let \(d^*\) attain this minimum. Then, for each \(s\in S\),
This is a bound on foregone utility relative to the scenario-specific best, not a bound on clinical harm or a guarantee that the selected regimen is optimal in any particular scenario.
Worked Example: Calculating Maximum Regret
Use the hypothetical utility table in the preceding example. The best utility in S1 is 90 (regimen C), in S2 it is 68 (regimen B), and in S3 it is 76 (regimen C). Subtract each regimen’s score from the best score within its scenario:
| Regimen | Regret in S1 | Regret in S2 | Regret in S3 | Maximum regret |
|---|---|---|---|---|
| A | 90 − 82 = 8 | 68 − 58 = 10 | 76 − 70 = 6 | 10 |
| B | 90 − 75 = 15 | 68 − 68 = 0 | 76 − 62 = 14 | 15 |
| C | 90 − 90 = 0 | 68 − 40 = 28 | 76 − 76 = 0 | 28 |
The smallest maximum regret is 10, for regimen A, so minimax regret selects A. This differs from the maximin choice B. The rules answer different questions: B maximizes the lowest utility, whereas A limits the largest gap from the scenario-specific best. Neither result is a calculation error; the preferred rule depends on the decision’s priorities.
Probability of Meeting Clinical Constraints
A utility score can conceal whether a regimen meets an important clinical requirement. When efficacy and safety conditions are specified as constraints, a complementary comparison is the probability of meeting them jointly. For example, the event might require both a response measure to exceed a target and a safety risk to remain below a limit. The thresholds, population, assessment time, and joint event must be defined before calculating the probability.
Within a scenario, this probability can reflect modeled patient-level variability and outcome uncertainty. It is distinct from a probability assigned to the scenario itself. Averaging probabilities across scenarios requires scenario weights that sum to one; those weights represent an additional judgment or probability model. If no defensible weights are available, reporting the range or worst value across scenarios avoids implying that the scenarios have known probabilities.
Worked Example: Comparing Constraint-Meeting Probabilities
Consider three hypothetical regimens and the event “a patient meets both the prespecified efficacy target and safety limit.” The entries below are modeled probabilities of that joint event conditional on each scenario. They are illustrative and have no units.
| Regimen | S1 | S2 | S3 |
|---|---|---|---|
| A | 0.92 | 0.75 | 0.60 |
| B | 0.84 | 0.82 | 0.78 |
| C | 0.95 | 0.55 | 0.70 |
If the criterion is the highest probability in the least favorable scenario, the minima are 0.60 for A, 0.78 for B, and 0.55 for C. On that criterion B is preferred, with a worst-scenario modeled probability of 0.78. This does not mean that 78% of patients will meet the constraints in every real population; it is a prediction conditional on this model and scenario set.
Suppose, instead, the team has justified scenario weights of 0.50, 0.30, and 0.20 for S1, S2, and S3. The weighted probability for A is \(0.50(0.92)+0.30(0.75)+0.20(0.60)=0.805\). For B it is \(0.50(0.84)+0.30(0.82)+0.20(0.78)=0.822\). For C it is \(0.50(0.95)+0.30(0.55)+0.20(0.70)=0.780\). Under these weights, B again has the largest averaged probability. If the weights are not justified, however, this average should not be presented as an unconditional probability of success.
Applying and Checking a Robust Decision Rule
A disciplined analysis separates the construction of the scenario set from the rule used to choose a regimen. The utility definitions and any mandatory constraints should be fixed before inspecting which regimen wins. Then calculate the relevant quantity for every candidate and scenario, preserve the scenario-specific results for review, and explain why the chosen rule fits the consequences of a poor outcome.
List the candidate regimens, target population, time horizon, outcomes, and any constraints that must be met.
Include decision-relevant parameter values and credible model structures. State the rationale and avoid treating an unexamined set as exhaustive.
Use a consistent utility scale or calculate the same joint constraint event under each regimen-scenario combination.
Report scenario-specific results as well as the selected maximin, minimax-regret, or probability-based choice. Explain why that criterion reflects the decision priorities.
Diagnostics are important because a single summary can hide a fragile analysis. Check that all scenarios and regimens use the same outcome definitions, that higher utility truly means better performance, and that the regret benchmark uses the best candidate within each scenario. Examine whether one implausible extreme scenario alone determines the worst case, whether a small change in a utility trade-off reverses the ranking, and whether probability averages depend heavily on uncertain weights. These checks diagnose dependence on the analysis choices; they do not establish that one regimen is clinically safe.
A common mistake is to call the regimen with the highest utility under the reference model “robust” because its score is large. That reasoning ignores how it performs elsewhere. Robustness requires comparison across a stated set of alternatives using an explicit rule. Another mistake is to average scenario results equally and call the result a probability of success. Equal weighting is an assumption, not a consequence of having three scenarios; without justified weights, use a worst-case or range-based summary, or state clearly that the average is a decision score rather than a probability.
How the Approaches Differ—and What They Cannot Establish
A nominal or expected-utility approach chooses the regimen with the greatest utility under a reference scenario, or the greatest weighted average when scenario probabilities are available and defensible. It can be appropriate when probabilities are credible and the decision-maker wants to optimize average performance. Maximin is more protective of a poor outcome in any included scenario. Minimax regret protects against being far from the best choice that would have been made with scenario knowledge. A constraint-meeting probability focuses directly on a specified clinical event, but depends on the outcome model and, if averaged across scenarios, on scenario weights.
These approaches can select different regimens because they encode different priorities. Robustness is not a property that a regimen possesses independently of the scenario set, utility definition, constraints, and decision rule. Nor does a favorable robust summary prove that all relevant model uncertainty has been represented. An omitted mechanism, a misspecified population, or an unrealistic range of parameters can make every reported comparison misleading. Robust choice supports a transparent decision under stated uncertainty; it does not replace clinical judgment or evidence about the model’s credibility.
Check Your Understanding
Answer each question, then compare with the solutions below.
- Using the utility table for regimens A, B, and C, which regimen is selected by maximin, and what is its worst-case utility?
- For regimen C, what is the regret in S2, and what does that value mean?
- If scenario weights are unavailable, why is an equal-weight average of constraint-meeting probabilities not automatically a probability of success?
- A regimen has a high reference-scenario utility but a very low utility in one scenario considered plausible. Can the reference score alone establish that the regimen is robust? Explain.
Solutions. (1) B is selected: its minimum utility is \(\min(75,68,62)=62\) utility points, greater than A’s 58 and C’s 40. (2) C’s regret in S2 is \(68-40=28\) utility points. It is 28 points below the best available regimen in S2, which is B; it is not a probability or a clinical risk measure. (3) Equal weighting assumes each scenario has probability \(1/3\). Without evidence or a stated decision rationale for that assumption, the resulting average is not an established probability of success. (4) No. Robustness requires results across a justified scenario set and an explicit decision rule; one reference-scenario score does not show performance under the other plausible scenario.