Scenario-Based Robust Design

Why Evaluate a Design Across Scenarios?

A sampling time that is informative when clearance is low may be less useful when clearance is high. A local design assesses performance at one specified parameter value; when that value is uncertain, a single result can give an incomplete picture. A finite scenario set makes the uncertainty explicit: list plausible parameter values, assign each a weight, and calculate the design’s performance separately at every listed value.

The weights summarize how the scenarios contribute to the design decision. If they represent probabilities, the weighted average is an expected performance. If they instead express deliberate decision weights, the same arithmetic gives a weighted summary but should not be described as a probability-based expectation. In either case, the scenario-specific results matter: an average alone can conceal how much performance varies across the set.

A scenario is a complete parameter specification for the model being evaluated. Here, for example, a scenario will specify clearance \(CL\), while dose, volume of distribution \(V\), and residual-error standard deviation are held fixed. The scenarios are not new observations or different designs; they are alternative parameter states under which the same candidate design is assessed.

Model and Assumptions for the PK Calculation

Consider a hypothetical one-compartment intravenous-bolus model with concentration at time \(t\)

Suppose a measured concentration is \(Y(t)=C(t;CL)+\varepsilon\), where \(\varepsilon\sim N(0,\sigma^2)\). For the calculations below, assume the dose \(D\), volume \(V\), and residual-error standard deviation \(\sigma\) are known; the only unknown model parameter of interest is \(CL\); and measurement errors are independent with constant variance across sample times and scenarios. These assumptions let us use the local Fisher information for \(CL\) as the performance measure. They are deliberately restrictive: changing the error model or estimating additional parameters changes the information calculation.

For a single measurement at time \(t\), the information depends on the sensitivity of the expected concentration to clearance. Differentiating the model gives

Under the stated Gaussian, constant-variance model, the information for \(CL\) is the squared sensitivity divided by the measurement variance. This formula follows directly from the expected squared score: the score is \((Y-C)(\partial C/\partial CL)/\sigma^2\), and \(E[(Y-C)^2]=\sigma^2\). Therefore,

This derivation also shows why the nonlinear model must be evaluated separately at each scenario. Although the sensitivity contains the factor \(t/V\), its concentration term depends exponentially on \(CL\). For independent measurements, information adds, so the information for a schedule with several measurements is the sum of their individual contributions.

Build the Scenario Set Before Comparing Designs

Scenario construction is part of the design decision, not just a bookkeeping step. The values should span parameter states considered plausible for the application, and the weights should have an explicit interpretation. A small set can be useful for transparent calculations, but it only represents the uncertainty included in that set. A weighted average cannot account for parameter states that were omitted.

When the parameter is uncertain, evaluate \(U(d,\theta_k)\) at every scenario and then take the weighted sum. Do not generally replace all scenarios with their weighted-average parameter value and evaluate the model once. For a nonlinear model, the average of the performances and the performance at the average parameter need not agree. The preceding material on Bayesian Expected-Utility Design uses the same general principle of averaging utility over uncertainty; a finite scenario calculation makes the selected support points and weights explicit.

Calculate Performance at Each Scenario

Use the same hypothetical PK setup throughout the next examples: \(D=100\) mg, \(V=20\) L, and \(\sigma=0.20\) mg/L. The resulting concentration is measured in mg/L. The three clearance scenarios and their weights are those specified above. For one measurement, \(I(t;CL)\) is measured in \((\text{L/h})^{-2}\), the reciprocal of the squared units of clearance.

Compare Candidate Designs Without Hiding Scenario Results

A design can be a single sampling time or a schedule of times. To compare candidate designs using weighted average information, calculate the information under every scenario for each design, multiply by the scenario weights, and sum. Keep the scenario-specific values alongside the summary. A candidate with a larger weighted average can still perform worse in one or more scenarios.

Why the Average Parameter Is Not a Substitute

A tempting shortcut is to calculate the weighted-average clearance and evaluate the design only there. That calculation answers a different question from the weighted average of scenario-specific performance. The distinction can be checked numerically using the 16-hour design.

At the weighted-average clearance \(CL=1.05\) L/h, \(C=5e^{-1.05(16/20)}=2.159\) mg/L and \(I=(0.8(2.159))^2/0.04=74.55\ (\text{L/h})^{-2}\). But the weighted average of the three scenario-specific information values is 87.21 \((\text{L/h})^{-2}\). These differ because concentration and information are nonlinear functions of clearance. The value at \(1.05\) L/h is not the weighted average performance for the scenario set.

This does not mean the weighted average is always the only appropriate summary. It is one defined criterion: scenario performance is combined using the specified weights. Other decision goals may require examining a particular scenario or using a different measure. State which quantity is being compared before interpreting a design ranking.

Diagnostics, Alternatives, and Limitations

Several checks help prevent a scenario calculation from giving a misleading result:

• Check the weights. Confirm each is nonnegative and that their sum is one. If weights are not normalized, normalize them before interpreting the sum as an average.
• Check the scenario support. Ask whether the listed values span the parameter states relevant to the decision. A precise calculation over a narrow or poorly chosen set is still incomplete.
• Check units and the metric. Every scenario must use the same performance measure and units. Do not compare a weighted information value for one design with a concentration or standard error for another.
• Inspect scenario-specific results. A weighted average can obscure a low value or a reversal in design ranking at one scenario. Report the individual values when those differences could affect the decision.
• Check model and error assumptions. A constant additive-error standard deviation can be inappropriate if measurement variability changes with concentration. In that case, the sensitivity-based information formula must match the revised observation model.

A nominal design evaluates performance at one selected parameter value; it is simpler but does not summarize a range of scenarios. A design optimized separately for each scenario can show how the preferred sampling time changes with parameters, but it does not by itself specify one design to use before the scenario is known. A finite weighted scenario calculation provides a transparent summary for a fixed design and a specified set of weights. Its limitations are equally important: it depends on scenario selection and weights, represents only the listed parameter states, and summarizes model-based information rather than guaranteeing successful estimation in an actual study.