Why Compare Designs With Scenario-Specific Benchmarks?
A sampling schedule can be excellent under one plausible set of PK parameters and mediocre under another. A robust design criterion must therefore say not only how performance varies across scenarios, but also what counts as a meaningful shortfall. Minimax regret measures each design’s performance gap from the design that would have been best if the scenario were known in advance, then chooses the design whose largest such gap is smallest.
This is different from asking which design has the best worst-case performance. The maximin criterion compares designs’ absolute performance in their least favorable scenarios; minimax regret compares them with scenario-specific attainable benchmarks. The distinction matters when scenarios have different performance levels or when the best schedule changes across scenarios.
Regret is nonnegative because each benchmark is the optimum over the same feasible set that contains \(d\). A design’s regret is zero in a scenario where it achieves that scenario’s optimum; it can be positive in other scenarios. The benchmark is an oracle comparison, not a claim that the scenario will be known when the study is designed.
In PK design, \(U_s(d)\) might be a scalar criterion calculated from the Fisher information matrix under scenario \(s\), with larger values defined as better. For example, one may use log determinant as a utility. Alternatively, an approximate variance for a specified parameter may be treated as a loss, with smaller values better. The criterion and its direction must be stated before calculating regret. Regret is measured on that criterion’s scale; it is not automatically a percentage or a probability.
Assumptions and Construction
A minimax-regret calculation is only as meaningful as its scenario set, performance criterion, and feasible designs. At minimum, specify a finite set of plausible parameter scenarios (or a clearly defined uncertainty set), a common set of allowable sampling designs, and a consistent way to calculate performance for every design-scenario pair. The design constraints—such as allowable sampling times, number of samples, and cohort allocation—must be the same when finding each scenario’s benchmark and when comparing candidate designs.
The utility or loss must represent the study objective and be comparable across designs within each scenario. If an information criterion is used, its calculation inherits the assumptions and approximations of that criterion and the PK model. For example, a local Fisher-information utility is conditional on the parameter values in each scenario and on the model used to compute information. A high value is not, by itself, a guarantee of good realized estimation performance.
The scenario-specific benchmark is found separately in each scenario, but over the same feasible design set. Only after finding these benchmarks can the regret table be formed. For a finite candidate set, the procedure is direct: calculate all performances, take the best value within each scenario, subtract each candidate’s value from that benchmark for a utility (or subtract the best loss from each candidate’s loss), then minimize the largest regret across scenarios.
List plausible PK parameter scenarios and the common feasible sampling plans to compare.
Evaluate the selected utility or loss for every plan in every scenario using one consistent modeling procedure.
For each scenario, identify the best-performing feasible plan and record its utility maximum or loss minimum.
Subtract candidate performance from each utility benchmark, or subtract each loss benchmark from candidate loss. Select a plan with the smallest largest regret.
Worked Calculations
Worked Example: Minimax Regret Across Two PK Scenarios
Suppose three hypothetical sampling schedules, A, B, and C, are evaluated under two plausible PK parameter scenarios: slow elimination and fast elimination. Their utilities are values of the same log-determinant information criterion, with larger values preferred.
| Scenario | Schedule A | Schedule B | Schedule C |
|---|---|---|---|
| Slow elimination | 8.2 | 7.7 | 6.8 |
| Fast elimination | 5.0 | 7.1 | 7.8 |
In the slow-elimination scenario, the benchmark is \(U_{\mathrm{slow}}^*=8.2\), achieved by A. The regrets are \(R_{\mathrm{slow}}(A)=8.2-8.2=0\), \(R_{\mathrm{slow}}(B)=8.2-7.7=0.5\), and \(R_{\mathrm{slow}}(C)=8.2-6.8=1.4\).
In the fast-elimination scenario, the benchmark is \(U_{\mathrm{fast}}^*=7.8\), achieved by C. The regrets are \(R_{\mathrm{fast}}(A)=7.8-5.0=2.8\), \(R_{\mathrm{fast}}(B)=7.8-7.1=0.7\), and \(R_{\mathrm{fast}}(C)=7.8-7.8=0\). The maximum regret for A is \(\max(0,2.8)=2.8\); for B it is \(\max(0.5,0.7)=0.7\); for C it is \(\max(1.4,0)=1.4\). Therefore, schedule B is minimax-regret among these candidates. Its worst shortfall from the scenario-specific best is 0.7 utility units.
Worked Example: Regret From a PK Information Determinant
Suppose the same three hypothetical schedules are evaluated by the determinant of a two-parameter Fisher information matrix. The determinants below are positive, and larger determinants are preferred. To express regret on a log scale, define utility as \(U_s(d)=\log\det I_s(d)\), using the natural logarithm.
| Scenario | Schedule A: determinant | Schedule B: determinant | Schedule C: determinant |
|---|---|---|---|
| Slow elimination | 100 | 80 | 40 |
| Fast elimination | 25 | 64 | 100 |
For slow elimination, the largest determinant is 100, so the benchmark utility is \(\log(100)=4.6052\). Schedule B’s regret is \(\log(100)-\log(80)=\log(100/80)=0.2231\), and C’s is \(\log(100/40)=0.9163\); A’s is zero.
For fast elimination, the largest determinant is also 100. A’s regret is \(\log(100/25)=1.3863\), B’s is \(\log(100/64)=0.4463\), and C’s is zero. Thus the maximum regrets are 1.3863 for A, 0.4463 for B, and 0.9163 for C. Schedule B is again minimax-regret, with a maximum log-determinant shortfall of 0.4463. On the determinant scale, this corresponds in the fast-elimination scenario to a best-to-B determinant ratio of \(100/64=1.5625\).
Taking a log makes a difference between log determinants equal to the logarithm of a determinant ratio. This can make interpretation easier than comparing raw determinants with very different magnitudes. The criterion still needs to match the design objective; transforming a utility changes the regret scale and can change the minimax choice.
How the Criterion Changes the Decision
Regret is defined relative to a chosen performance scale. For a loss, one can use absolute regret, \(L_s(d)-L_s^*\), or a relative regret such as \(L_s(d)/L_s^*-1\), provided \(L_s^*>0\). These answer different questions: absolute regret measures the difference in loss units, while relative regret measures the difference as a fraction of the scenario’s best loss. A relative form is undefined when its benchmark loss is zero and may be unstable when the benchmark is close to zero.
Worked Example: Absolute Versus Relative Loss Regret
Consider two hypothetical designs, A and B, and approximate variance of log apparent clearance as the loss, in \((\log\text{-clearance})^2\) units. Lower variance is better. The values are A: 1 and B: 2 in scenario 1; A: 190 and B: 100 in scenario 2.
The scenario benchmarks are 1 and 100. Under absolute regret, A has regrets \(1-1=0\) and \(190-100=90\), so its maximum is 90. B has regrets \(2-1=1\) and \(100-100=0\), so its maximum is 1. The absolute-regret choice is B.
Under relative regret, A’s regrets are \(1/1-1=0\) and \(190/100-1=0.9\), giving a maximum of 0.9. B’s are \(2/1-1=1\) and \(100/100-1=0\), giving a maximum of 1. The relative-regret choice is A. Neither calculation is arithmetically wrong: they encode different priorities. The decision-maker must justify which scale reflects the consequences of poorer precision.
The same caution applies to utility criteria. A strictly increasing transformation preserves which design is optimal within any single scenario, but it need not preserve differences between designs. Since regret uses those differences, minimizing maximum regret can depend on the utility scale. The log-determinant example illustrates a deliberate choice of scale, not a universal conversion rule.
Relation to Maximin Design and Practical Diagnostics
Maximin utility selects a design maximizing its worst scenario utility, \(\max_d\min_s U_s(d)\). Minimax regret instead selects the design minimizing \(\max_s[U_s^*-U_s(d)]\). The former asks, “Which design’s least favorable utility is highest?” The latter asks, “Which design is least far from the scenario-specific best?” They need not agree.
Worked Example: Maximin and Minimax Regret Can Differ
Consider three hypothetical schedules and two scenarios, with utility values to be maximized:
| Scenario | Schedule A | Schedule B | Schedule C |
|---|---|---|---|
| Scenario 1 | 10 | 100 | 0 |
| Scenario 2 | 8 | 7 | 9 |
The minimum utility for A is 8, for B is 7, and for C is 0; maximin therefore selects A. For regret, the scenario benchmarks are 100 and 9. A’s regrets are \(100-10=90\) and \(9-8=1\), giving maximum 90. B’s are \(100-100=0\) and \(9-7=2\), giving maximum 2. C’s are \(100-0=100\) and \(9-9=0\), giving maximum 100. Minimax regret selects B. The designs answer different questions: B stays closest to the scenario-specific optimum, while A has the stronger worst absolute utility.
Before accepting a result, inspect the full regret table, not only the winning maximum. Confirm that every benchmark is attainable within the stated constraints and that no candidate has negative regret; a negative value signals inconsistent benchmarks or calculations. Identify which scenario sets each candidate’s maximum regret, then check whether the selected design is driven by one extreme or implausible scenario. If numerical optimization or a discrete time grid is used, verify that the scenario-specific benchmark was adequately optimized: an artificially poor benchmark makes all regrets in that scenario appear too small.
The candidate set also matters. A minimax-regret design among three schedules is only best among those schedules; it is not necessarily optimal over all feasible sampling times. Refine the candidate grid or use a suitable optimization method, then verify that the decision does not arise merely from a coarse search. As with any scenario-based result, the choice is conditional on the scenarios considered. Adding or removing scenarios can alter both the benchmarks and the winning design.
Common Mistakes and Limitations
- Using one benchmark for every scenario. Regret is measured against each scenario’s own best feasible design, not against a single nominal design.
- Mixing feasible sets. If one scenario’s benchmark permits designs excluded from the comparison, its regret is not a valid like-for-like gap. Apply the same constraints throughout.
- Confusing regret with poor absolute performance. A small maximum regret means closeness to scenario-specific optima on the selected scale; it does not guarantee high precision or acceptable performance in absolute terms.
- Assuming scenario probabilities are used. Basic minimax regret takes the largest regret over the specified scenarios; it does not average regrets or use scenario probabilities. Probability-weighted averaging is a different decision criterion.
- Treating the selected scale as neutral. Absolute, relative, and transformed-utility regrets can favor different designs. State the criterion and why its scale is appropriate.
Minimax regret is useful when the scenarios are plausible alternatives, their probabilities are not being used, and avoiding a large shortfall from an informed scenario-specific design is the decision objective. Its limitations are equally important: it does not identify the most probable scenario, remove model uncertainty, or guarantee acceptable estimation properties. The result should be reported with the scenario set, the criterion, its units or scale, the scenario-specific benchmarks, and the worst-regret scenario.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For a utility to be maximized, what is the scenario-specific regret, and why is it nonnegative?
- Two hypothetical designs have utilities A: 6 and B: 8 in scenario 1, and A: 10 and B: 7 in scenario 2. Calculate each design’s maximum regret and identify the minimax-regret design.
- For losses to be minimized, design A has losses 4 and 20 across two scenarios, while design B has losses 7 and 12. Find their absolute regrets and the minimax-regret design.
- Why can a minimax-regret design differ from a maximin-utility design?
Solutions. (1) Regret is \(R_s(d)=U_s^*-U_s(d)\), where \(U_s^*\) is the largest utility over the common feasible set in scenario \(s\). Since \(U_s^*\ge U_s(d)\), regret is nonnegative. (2) The scenario benchmarks are 8 and 10. A’s regrets are \(8-6=2\) and \(10-10=0\), so its maximum is 2. B’s are \(8-8=0\) and \(10-7=3\), so its maximum is 3. A is minimax-regret. (3) The best losses are 4 and 12. A’s regrets are \(4-4=0\) and \(20-12=8\), so its maximum is 8. B’s are \(7-4=3\) and \(12-12=0\), so its maximum is 3. B is minimax-regret. (4) Maximin ranks designs by their lowest absolute utility, whereas minimax regret ranks them by their largest gap from scenario-specific optima. Those quantities can favor different designs, as in the worked example above.