Trade-Offs Between Nominal and Robust Performance

Why compare nominal and robust performance?

A schedule can perform very well when the design parameters take their nominal values but lose substantial performance when those parameters differ from the nominal specification. Another schedule may sacrifice some nominal performance in exchange for more consistent performance across plausible parameter values. Neither schedule is automatically preferable: the choice depends on how much nominal performance the design team is willing to trade for robustness.

A useful way to make this trade-off explicit is to compare every candidate schedule on two criteria: its performance at the nominal parameter value and its performance averaged over parameter uncertainty. The Pareto frontier identifies schedules for which improving one criterion requires giving up some of the other. A tolerance for nominal performance loss then turns that comparison into a practical selection rule.

Both criteria must use the same underlying performance definition and orientation: higher must mean better for each. If the original criterion is a quantity to minimize, such as an estimation loss, it can be expressed as a utility that increases as that loss decreases. A comparison is meaningful only if the two performance values are sufficiently comparable to interpret together. The utility need not be the same as every other design criterion; it must be specified and applied consistently across schedules and parameter values.

The expectation in \(R(d)\) is taken over the stated uncertainty distribution. With a finite scenario set, it is a weighted average. The nominal value \(N(d)\) is evaluated at \(\theta_0\); it is not generally the same as performance in one of the uncertainty scenarios. The distribution, scenario weights, and performance calculation are part of the definition of the comparison, not incidental details.

Dominance and the Pareto frontier

Schedule \(d_1\) dominates schedule \(d_2\) if \(d_1\) is at least as good on both criteria and strictly better on at least one. In symbols, \(N(d_1)\geq N(d_2)\) and \(R(d_1)\geq R(d_2)\), with at least one strict inequality. A dominated schedule cannot be the best choice for any decision-maker who prefers more of both criteria and has no additional reason to choose that schedule.

The Pareto frontier is the set of candidate schedules that are not dominated by another candidate. On a plot with nominal performance on the horizontal axis and uncertainty-averaged performance on the vertical axis, the frontier is the upper-right boundary of the candidate points. A point on that boundary may have high nominal performance, high robust performance, or a compromise between the two. Being on the frontier does not, by itself, identify a single best schedule; a preference or decision rule is still needed.

Selecting a schedule with a nominal-loss tolerance

A decision-maker can express the acceptable trade-off as a maximum loss in nominal performance relative to the best nominal candidate. Let \(N^*=\max_d N(d)\) be the best nominal performance among the candidate schedules. If the tolerated loss is an absolute amount \(\Delta\), the acceptable set is

The selection rule is to choose the schedule in \(\mathcal{D}_{\Delta}\) with the largest \(R(d)\). This is an example of an epsilon-constraint rule: optimize one criterion while imposing an explicit bound on the other. Its advantage is that the trade-off is stated in interpretable performance units rather than hidden inside an arbitrary weighted sum.

If performance is positive and a percentage tolerance is meaningful on the utility scale, a relative nominal loss can instead be defined as

A relative tolerance of \(\delta\%\) then retains schedules satisfying \(L_N(d)\leq\delta\%\). This ratio should not be used mechanically if the utility scale has no meaningful zero or can take values near or below zero. In those cases, specify an absolute tolerance or a scientifically meaningful rescaling.

Why the selection rule works

The rule follows directly from the decision statement. The tolerance defines which schedules are acceptable, and the objective then ranks only those schedules by robust performance. A schedule outside the acceptable set cannot be selected, even if its robust score is largest overall. Among acceptable schedules, any schedule with lower robust performance than another is inferior under the stated rule.

For a finite candidate set, the procedure is complete: calculate \(N(d)\) and \(R(d)\) for each schedule, find the largest nominal score, apply the tolerance, and identify the largest robust score among the remaining schedules. If two acceptable schedules tie on robust score, the stated rule does not distinguish them; another prespecified consideration, such as implementation simplicity, can break the tie.

Dominance screening can be done before applying the tolerance because a dominated schedule cannot outperform its dominator on robust performance while meeting a looser nominal requirement. The tolerance calculation itself should still use the best nominal performance across the full candidate set.

Alternative approaches, diagnostics, and limitations

The tolerance rule is one way to resolve the trade-off, not the only one. A nominal-only rule chooses the schedule with the largest \(N(d)\), ignoring the uncertainty-averaged criterion. A worst-case rule selects according to the lowest performance over a specified uncertainty set, emphasizing protection against poor outcomes rather than average performance. A weighted-sum rule combines the two criteria, for example \(wN(d)+(1-w)R(d)\). That approach requires a defensible choice of weight \(w\), and the result can depend on the scales of \(N\) and \(R\). The tolerance rule avoids a direct weighted sum but still requires an explicit, defensible tolerance.

Several checks help reveal whether a comparison is being used appropriately:

• Check the direction. If one displayed criterion is a loss for which smaller is better, reverse or clearly transform it before interpreting an upper-right frontier.
• Check the uncertainty specification. Verify that scenario probabilities are nonnegative and sum to one, and that the scenarios represent the intended parameter uncertainty.
• Check the performance scale. Confirm that every schedule is evaluated using the same utility definition, units, parameter scenarios, and nominal value.
• Inspect scenario-specific results. An average can conceal a large decline in a low-probability scenario. Report scenario outcomes or other relevant summaries when such declines matter to the decision.
• Check numerical stability. If robust performance is estimated by simulation, assess whether Monte Carlo error could change the ranking of close candidates. A nominal-loss boundary can also be sensitive to small calculation differences.

A common mistake is to declare the schedule with the highest uncertainty-averaged score “most robust” and choose it without checking its nominal loss. The correction is to distinguish description from decision: the largest \(R(d)\) identifies the robust-score leader, but the tolerance rule selects the robust-score leader among schedules that meet the nominal requirement. Another mistake is to call every frontier schedule equally good. The frontier only rules out domination; it does not specify which trade-off is acceptable.

The frontier is limited by the schedules considered. A candidate omitted from the search could dominate the reported frontier, and a coarse grid may miss useful intermediate schedules. The uncertainty-averaged criterion also depends on the uncertainty distribution and can understate the importance of rare, severe performance losses. Finally, a two-criterion comparison does not account automatically for other considerations, such as feasibility or operational burden. Those should be addressed explicitly rather than assumed to be represented by nominal or robust performance.