Robustness Criteria for Decisions Under Uncertainty

What Does It Mean for a Decision to Be Robust?

A decision is made using predicted outcomes and a stated rule: for example, select a regimen that meets an efficacy target while keeping adverse-event risk acceptable. Those predictions can change when assumptions change. A practical robustness criterion makes explicit how much change the decision can tolerate before it is no longer considered satisfactory.

Robustness is therefore not a property of a regimen in isolation. It is a conclusion relative to a defined set of plausible scenarios, selected outcomes, and a decision rule. A regimen can be robust under one criterion and not another. For example, it might remain the preferred option across scenarios yet fail to keep a minimum efficacy margin above a target.

The scenario set might represent plausible alternative assumptions about exposure, response, adherence, or another decision-relevant input. Earlier material on scenario analysis and plausible scenario bounds provides ways to construct such a set. Here, the central task is to specify what decision performance across that set will count as robust.

A robustness assessment requires several choices to be stated plainly: which scenarios are included, which outcome and units are used, what makes a regimen acceptable or preferred, and whether the criterion must hold in every scenario. These choices are not technical details to fill in after seeing the results. They define the question being answered.

Two Practical Criteria

One criterion asks whether the preferred regimen is unchanged. Suppose \(U_{d,k}\) is a decision score for regimen \(d\) under scenario \(k\), with larger scores preferred. If a separate acceptability constraint is used, first exclude regimens that violate it. The preferred regimen in scenario \(k\) is then the acceptable regimen with the highest score.

Here, \(\mathcal{F}_k\) is the set of regimens satisfying the stated constraints in scenario \(k\). Preference is robust across \(\mathcal{S}\) if the same regimen is preferred in every scenario. If scores tie, the analysis needs a declared tie rule; otherwise, “the preferred regimen” may not be uniquely defined.

A second criterion asks whether efficacy stays sufficiently above a clinical threshold. Let \(E_{d,k}\) be predicted efficacy for regimen \(d\) in scenario \(k\), let \(E_{\min}\) be the minimum acceptable efficacy, and let \(\delta\) be a required additional margin, all on the same scale. The worst-case margin is the smallest scenario-specific difference between efficacy and threshold:

The regimen meets a margin criterion of \(\delta\) if \(M_d\geq\delta\). This is a worst-case requirement over the chosen scenario set, not a statement that the regimen will achieve that margin with a particular probability.

The threshold and the additional margin serve different purposes. \(E_{\min}\) identifies the minimum acceptable efficacy; \(\delta\) requires a buffer beyond it. A positive buffer may be chosen to reflect clinical importance or concern about prediction error, but its value must be justified and specified in advance.

Worked Examples

Assumptions, Diagnostics, and Alternatives

A robustness conclusion is interpretable only under clear assumptions. The scenarios should be plausible for the decision at hand, and the predicted outcomes should be calculated on comparable definitions and time horizons. The efficacy and safety thresholds must be clinically justified. Any score that combines outcomes also requires an explicit scale and weight: a numerical score does not make the trade-off self-evident.

The phrase “robust across scenarios” means only that the criterion holds across the scenarios actually examined. It does not establish that the scenario set contains every relevant possibility, or that the decision will succeed in future patients. Nor does an all-scenarios criterion assign probabilities to those scenarios. A probabilistic analysis addresses a different question by incorporating a probability distribution over uncertain inputs.

Useful checks before communicating a result include:

• Inspect the scenario set. Confirm that its bounds and combinations are credible and relevant, rather than selected only because they preserve a preferred answer.
• Check the limiting scenario. Identify which scenario determines the smallest efficacy margin or worst-case score. A single weak point can determine whether an all-scenarios criterion passes.
• Verify units and thresholds. A response rate expressed as a proportion must not be compared directly with a threshold expressed as a percentage without conversion.
• Make ties and constraints explicit. State whether a regimen must strictly outperform alternatives, how ties are resolved, and whether safety limits are hard exclusions or part of a score.
• Reconsider the decision rule. If reasonable changes to a margin, constraint, or score weight reverse the conclusion, report that dependence rather than describing the choice as unconditionally robust.

A common pitfall is to call a regimen robust because its average predicted efficacy exceeds the target. That reasoning replaces an all-scenarios requirement with an average, which can conceal a scenario that falls below the threshold. The correction is to calculate each scenario-specific margin and use the minimum when the criterion requires performance in every scenario. Another pitfall is to report a stable preferred regimen without naming the score, constraints, and scenario set; preference cannot be interpreted without those definitions.

A stable-preference criterion is easy to communicate and directly tied to the selected action, but it can conceal how close competing scores are. A minimum-margin criterion communicates a clinical buffer against a threshold but does not, by itself, rank regimens. A worst-case utility rule ranks choices by their least favorable score, but its result depends on the utility scale and weights. Probabilistic sensitivity analysis instead summarizes decisions over a distribution of uncertain inputs; it is not interchangeable with a guarantee that a condition holds in every prespecified scenario. The appropriate approach depends on the development question and should be stated alongside the result.