Why Stress-Test a Dose Recommendation?
A simulated dose recommendation is conditional: it depends on how the PK model describes concentration profiles, how variability is represented, and how exposure is linked to clinical targets. A regimen that performs best under one plausible set of assumptions may not remain preferred when those assumptions change. Sensitivity analysis makes that dependence visible before a recommendation is treated as robust.
The purpose is not to search through assumptions until a favored regimen wins. Instead, specify a defensible reference analysis, identify alternatives that are plausible from the available evidence, and apply the same decision rule to every scenario. The result is a map of which assumptions matter to the decision, not a probability that any one assumption is true.
Set Up Comparable Scenarios
Begin by stating the candidate regimens, target definitions, simulated population, and selection rule. For example, the rule might select the regimen with the largest probability of meeting both a minimum efficacy target and a maximum safety limit. The rule must be held fixed across scenarios; otherwise, a changed recommendation cannot be attributed to changed PK/PD assumptions alone.
For regimen \(d\) under scenario \(s\), define joint target attainment as the fraction of simulated individuals meeting both criteria:
Here \(N\) is the number of simulated individuals, and \(I\{\cdot\}\) equals 1 when its statement is true and 0 otherwise. The product is 1 only when both targets are met. The estimate is a simulated proportion, not a guarantee of clinical outcomes. In particular, separate efficacy and safety attainment probabilities cannot generally be multiplied to obtain joint attainment: the two outcomes may be dependent within individuals.
Keep the scenario changes interpretable. Structural alternatives might compare a one-compartment model with a two-compartment model, or alternative absorption descriptions, while preserving clinically justified parameterization. Variability alternatives might change interindividual variability or correlations between random effects. Exposure-response alternatives might change an effect threshold, an EC50 value, a Hill coefficient, or the relationship used to translate exposure into an endpoint. Change one component at a time to isolate its influence, then examine a small number of combined scenarios to see whether effects interact.
Worked Comparisons
Worked Example: A Structural PK Alternative Reverses the Choice
Hypothetical setup. Compare regimen A, 100 mg every 12 hours, with regimen B, 200 mg every 24 hours. A virtual cohort of \(N=1{,}000\) individuals is simulated under each regimen. Efficacy means time above the concentration target for at least 50% of the dosing interval; safety means peak concentration does not exceed a prespecified limit. The rule selects the regimen with the higher joint target-attainment estimate. The reference scenario uses a one-compartment PK structure. A plausible alternative uses a two-compartment structure with distribution represented explicitly. The figures below are hypothetical simulation counts, not study results.
| Scenario and regimen | Efficacy met | Safety met | Both met | Joint attainment |
|---|---|---|---|---|
| Reference, A | 720 | 900 | 680 | 680/1,000 = 0.680 |
| Reference, B | 800 | 760 | 650 | 650/1,000 = 0.650 |
| Two-compartment, A | 700 | 900 | 630 | 630/1,000 = 0.630 |
| Two-compartment, B | 850 | 820 | 710 | 710/1,000 = 0.710 |
For the reference model, A has joint attainment \(680/1{,}000=0.680\), compared with \(650/1{,}000=0.650\) for B, so A is preferred by the stated rule. Under the alternative structure, B has \(710/1{,}000=0.710\), compared with \(630/1{,}000=0.630\) for A, so the recommendation reverses. Each joint count is no larger than either corresponding marginal count, as required. The example shows why marginal efficacy or safety results alone cannot establish the joint result.
The reversal does not prove that the two-compartment model is better or that B is clinically superior. It flags a decision-relevant structural assumption. The next step is to examine which predicted concentration features differ between models and whether the available PK evidence supports one structure more strongly.
Worked Example: Variability and Correlation Change Joint Attainment
Hypothetical setup. Under a fixed PK structure and the same efficacy and safety targets, compare A and B in \(N=2{,}000\) virtual individuals. The reference scenario uses zero correlation between two individual random effects, clearance and PD sensitivity. An alternative uses a clinically defensible nonzero correlation while leaving each marginal distribution unchanged. This changes how PK and PD characteristics occur together within an individual. The following counts are illustrative simulation outputs.
| Scenario and regimen | Efficacy met | Safety met | Both met | Joint attainment |
|---|---|---|---|---|
| Zero correlation, A | 1,500 | 1,500 | 1,260 | 1,260/2,000 = 0.630 |
| Zero correlation, B | 1,600 | 1,450 | 1,220 | 1,220/2,000 = 0.610 |
| Alternative correlation, A | 1,500 | 1,500 | 1,120 | 1,120/2,000 = 0.560 |
| Alternative correlation, B | 1,600 | 1,450 | 1,280 | 1,280/2,000 = 0.640 |
Under the reference scenario, A leads by \(0.630-0.610=0.020\), or 2.0 percentage points. Under the alternative correlation, B leads by \(0.640-0.560=0.080\), or 8.0 percentage points. The marginal counts have been kept the same here to isolate how the joint distribution can change; the overlap count is not determined by the two marginal counts. In a real analysis, a proposed correlation must be justified from evidence or a defensible range, rather than selected because it produces a desired ranking.
This comparison also illustrates why variability assumptions are not only about the size of each random effect. Dependence among individual parameters can affect whether the same individuals meet both targets, and therefore affect joint attainment even if marginal summaries barely change.
Worked Example: An Exposure-Response Target Changes the Preferred Regimen
Hypothetical setup. Regimen A produces more sustained concentrations, while regimen B produces higher peaks. For an Emax relationship, the predicted effect at concentration \(C\) is \(E(C)=E_{\max}C/(EC_{50}+C)\). If the efficacy target is 50% of \(E_{\max}\), solving \(0.5E_{\max}=E_{\max}C/(EC_{50}+C)\) gives \(C=EC_{50}\). Thus, the assumed \(EC_{50}\) sets the concentration threshold for this target. With a fixed safety limit and \(N=1{,}000\), suppose the hypothetical joint counts are:
| Exposure-response assumption | Regimen A: both met | Regimen B: both met | Preferred by joint attainment |
|---|---|---|---|
| \(EC_{50}=2\) mg/L; target \(C\geq2\) mg/L | 620/1,000 = 0.620 | 590/1,000 = 0.590 | A |
| \(EC_{50}=4\) mg/L; target \(C\geq4\) mg/L | 540/1,000 = 0.540 | 610/1,000 = 0.610 | B |
For the first assumption, \(620/1{,}000=0.620\) exceeds \(590/1{,}000=0.590\), so A is preferred. For the second, \(610/1{,}000=0.610\) exceeds \(540/1{,}000=0.540\), so B is preferred. Raising the target changes which parts of the concentration profiles count toward efficacy; the safety limit has not changed. This is a sensitivity to the exposure-response assumption, not evidence that either \(EC_{50}\) value is correct.
When the two regimen outcomes are simulated for the same individuals, a paired comparison can also show whether a small difference is larger than ordinary simulation noise. Suppose, for the first target, 520 individuals meet both targets on both regimens, 100 meet them only on A, 70 only on B, and 310 on neither. These counts sum to 1,000, and the marginal joint counts are \(520+100=620\) for A and \(520+70=590\) for B. Define \(Z_i=I_{A,i}-I_{B,i}\). Then 100 values are \(+1\), 70 are \(-1\), and 830 are 0. The estimated difference is \(\bar{Z}=(100-70)/1{,}000=0.030\). The sample variance is \(s_Z^2=[170-1{,}000(0.030)^2]/999=0.1693\), giving \(SE(\bar{Z})=s_Z/\sqrt{1{,}000}=0.0130\). An approximate 95% Monte Carlo interval is \(0.030\pm1.96(0.0130)\), or 0.0045 to 0.0555. This interval describes simulation precision conditional on the scenario, not uncertainty about which exposure-response assumption is true or about clinical outcomes.
Justifying the Paired Monte Carlo Comparison
The paired calculation is useful because both regimens are evaluated in the same simulated individuals. For each individual, \(Z_i\) records the difference in target attainment, taking values \(-1\), 0, or 1. Its mean estimates the difference in simulated probabilities. Since the \(N\) virtual individuals are generated independently, the variance of their average is the variance of one \(Z_i\) divided by \(N\). Replacing that unknown variance with the sample variance gives the Monte Carlo standard error:
Using the same simulated individuals often makes this comparison more precise than simulating each regimen with unrelated random draws, because shared individual characteristics can make the paired outcomes move together. This quantifies numerical simulation error only. It does not account for misspecified structural models, uncertain PD assumptions, or other sources of scientific uncertainty. Increasing \(N\) reduces Monte Carlo error but does not remove those limitations.
Diagnosing Fragile or Misleading Results
A reversal should prompt investigation, not automatic averaging of the scenarios or selection of whichever result is more favorable. Check that each alternative is plausible, that the same population and decision rule were used throughout, and that the simulation was implemented correctly. In particular, inspect concentration-time profiles and target-attainment estimates across the dose interval, and verify that efficacy and safety are evaluated for the same simulated individuals when reporting their joint probability.
- Differences are close to Monte Carlo error. Increase the number of simulated individuals or use paired random draws, then reassess whether the apparent change is numerically stable.
- A scenario is implausible or unsupported. Do not treat an arbitrary extreme value as an equally credible alternative. Explain the evidence or rationale for the range examined.
- Only marginal outcomes are reported. If the decision depends on meeting efficacy and safety together, calculate the joint event directly rather than multiplying marginal probabilities.
- Several assumptions change at once. Add one-factor comparisons to identify which changes drive the result, then use combined scenarios to examine interactions.
- The recommendation is unchanged but attainment falls. Stability of the preferred regimen does not mean the regimen remains adequate. Check whether any candidate still meets the prespecified decision criterion.
A one-factor-at-a-time analysis is easy to interpret, but may miss interactions. A scenario grid that combines plausible alternatives can reveal such interactions, although it requires more runs and careful reporting. If defensible probabilities for competing models or assumptions are available, model averaging is another possible approach; without defensible weights, it should not be presented as though scenarios have known probabilities. Sensitivity analysis complements, rather than replaces, uncertainty analysis within a fixed model.
Check Your Understanding
Answer each question using the definitions and decision logic above. Solutions are included after the questions.
- In a simulation of 500 individuals, 340 meet efficacy, 400 meet safety, and 300 meet both. What is the joint attainment estimate, and why is it not generally valid to multiply the two marginal estimates?
- A reference scenario gives regimen A joint attainment of 0.72 and B of 0.70. A plausible variability scenario gives A 0.64 and B 0.68. State the preferred regimen in each scenario under a rule that selects the higher joint attainment.
- What kind of uncertainty does the paired Monte Carlo standard error describe, and what does it not describe?
- Why can two scenarios with the same marginal efficacy and safety attainment have different joint attainment?
Solutions. (1) Joint attainment is \(300/500=0.600\). Multiplying the marginals would assume independence; that assumption is not given, and the directly simulated overlap is 0.600. (2) A is preferred in the reference scenario because \(0.72>0.70\); B is preferred in the alternative because \(0.68>0.64\). This is a reversal under the stated rule. (3) It describes numerical error from using a finite number of simulated individuals, conditional on the model and scenario. It does not describe uncertainty about the model or assumptions, nor clinical outcome uncertainty. (4) The same individuals may meet both targets more or less often depending on dependence between outcomes, so the overlap can change while each marginal proportion stays fixed.