Why Sensitivity Analysis Matters
A selected dose is not determined by data alone. It also depends on choices made to translate evidence into a decision: what counts as a clinically meaningful response, how much safety risk matters relative to efficacy, what prior information is used, and which dose-response model is considered plausible. Sensitivity analysis asks whether reasonable changes to these inputs would change the selected dose.
The goal is not to find assumptions that preserve a preferred answer. It is to make the dependence of the decision explicit. If the same dose is selected across a defensible range of assumptions, the decision is relatively robust to those assumptions. If small changes switch the selected dose, decision-makers should know which input is driving the change and whether the evidence can resolve it.
Sensitivity analysis is conditional on what is varied and what is held fixed. A one-way analysis varies one input while fixing the others at reference values. A scenario analysis changes a coherent set of inputs together. Neither approach automatically accounts for sampling uncertainty in the estimated dose-response relationship; that uncertainty should be represented and communicated separately. The earlier material on communicating uncertainty in the selected dose addresses that distinction.
Varying a Clinical Response Threshold
A response threshold defines what outcome qualifies as a response for a target-attainment decision. Changing the threshold can change the estimated probability of response at each dose, even when the patients and fitted model are unchanged. A decision rule might select the lowest tested dose whose estimated probability of response is at least a prespecified target \(q\). The threshold defining response and the target probability \(q\) are different quantities: the first defines the event, and the second defines the required chance of that event.
Worked Example: How a Response Threshold Changes the Target-Attainment Dose
Suppose a hypothetical study evaluates doses of 5, 10, and 20 mg. A fitted model gives the following estimated response probabilities under two definitions of clinically meaningful response. The target is \(q=0.70\), and the rule is to select the lowest tested dose with estimated response probability at least 0.70.
| Dose | Less stringent response threshold | More stringent response threshold |
|---|---|---|
| 5 mg | 0.72 | 0.48 |
| 10 mg | 0.84 | 0.68 |
| 20 mg | 0.91 | 0.82 |
Under the less stringent definition, 5 mg meets the target because \(0.72 \geq 0.70\). It is the lowest candidate dose, so the selected dose is 5 mg. Under the more stringent definition, 5 mg and 10 mg fail because \(0.48 < 0.70\) and \(0.68 < 0.70\); 20 mg meets the target because \(0.82 \geq 0.70\). The selected dose is therefore 20 mg.
The decision changes because the response event changed, not because the target probability changed. This result does not establish that either response definition is clinically correct. It shows that the threshold is decision-relevant and should be justified before the dose decision is interpreted.
A threshold analysis should use definitions that clinicians and patients could plausibly regard as meaningful. Varying a threshold over arbitrary values can create a large table of dose switches without clarifying the clinical decision. It is also important to verify that the response probabilities correspond to the stated outcome definitions and population; they cannot generally be treated as interchangeable estimates.
Utility Weights and Dose-Switching Boundaries
When efficacy and safety are combined in a utility, the relative weight assigned to each outcome can determine the preferred dose. A useful sensitivity analysis does more than report the selected dose at several arbitrary weights: it derives the values at which two doses have equal utility. These values are switching boundaries.
Worked Example: Finding the Utility-Weight Ranges for Each Dose
Consider three hypothetical doses. Let \(E(d)\) be the probability of efficacy and \(A(d)\) the probability of an adverse event. Define utility as \(U(d;\lambda)=E(d)-\lambda A(d)\), where \(\lambda\) is the disutility assigned to an adverse event relative to an efficacy response. Assume the estimates are suitable for this decision and use the following values.
| Dose | \(E(d)\) | \(A(d)\) |
|---|---|---|
| Low | 0.45 | 0.05 |
| Middle | 0.65 | 0.15 |
| High | 0.75 | 0.30 |
For the middle and high doses, equality occurs when:
For \(\lambda<0.667\), the high dose has greater utility than the middle dose; for \(\lambda>0.667\), the middle dose has greater utility. The middle and low doses tie when:
The high dose and low dose tie at \(\lambda=1.2\), because \(0.75-0.30\lambda=0.45-0.05\lambda\). That tie does not make either dose optimal: at \(\lambda=1.2\), the middle-dose utility is \(0.65-0.15(1.2)=0.47\), while each tied dose has utility \(0.39\). Thus the high dose is optimal for \(0\leq\lambda<0.667\), the middle dose for \(0.667<\lambda<2\), and the low dose for \(\lambda>2\). At the boundaries, the adjacent doses tie for the maximum.
For example, at \(\lambda=1\), the utilities are \(0.45-0.05=0.40\), \(0.65-0.15=0.50\), and \(0.75-0.30=0.45\). The middle dose is preferred. The weight \(\lambda\) has no units if both outcomes are coded on the same utility scale; if the utility is defined differently, its scale and interpretation must be stated.
This calculation assumes the stated utility function is an appropriate representation of the decision-maker’s preferences. It also assumes that the efficacy and adverse-event quantities are comparable inputs to that function. The switching boundary is mathematically precise, but it does not determine which weight is clinically acceptable. That requires a defensible elicitation or justification of preferences.
Prior Sensitivity and Model-Assumption Sensitivity
A prior can affect a Bayesian decision by changing posterior summaries or the posterior probability that a dose meets a target. A prior sensitivity analysis therefore compares decisions under several defensible prior specifications, rather than treating one prior as a neutral default. The likelihood, endpoint definition, utility, and decision rule should be held constant when the purpose is to isolate prior sensitivity.
Worked Example: Prior Strength Changes the Preferred Dose
Suppose hypothetical efficacy data at two doses are \(7\) responses among \(10\) patients at dose A and \(8\) among \(10\) at dose B. For each dose, model the response probability with a binomial likelihood and a common \(\operatorname{Beta}(a,b)\) prior. The posterior mean is \((y+a)/(n+a+b)\). For this illustration, adverse-event probabilities are treated as fixed at 0.10 for A and 0.30 for B, and utility is posterior mean efficacy minus \(0.10\) times adverse-event probability.
With a \(\operatorname{Beta}(1,1)\) prior, the posterior means are \(8/12=0.6667\) for A and \(9/12=0.7500\) for B. The utilities are \(0.6667-0.10(0.10)=0.6567\) and \(0.7500-0.10(0.30)=0.7200\), respectively. Dose B is preferred.
Now use a common \(\operatorname{Beta}(30,30)\) prior. The posterior means are \(37/70=0.5286\) for A and \(38/70=0.5429\) for B. The utilities are \(0.5286-0.0100=0.5186\) and \(0.5429-0.0300=0.5129\). Dose A is now preferred. The stronger prior shrinks both efficacy estimates toward 0.5, reducing the efficacy difference enough for the assumed safety penalty to determine the ranking.
This example isolates the prior’s effect by treating the adverse-event probabilities as fixed. In an actual analysis, uncertainty in those probabilities may also matter. The strong prior must not be chosen merely because it produces a desired dose; its information source and relevance to the study population require justification.
Structural assumptions can change predictions even when the data and decision rule are unchanged. For example, a plateau model assumes that additional dose eventually yields little additional response, while a model that continues increasing over the examined range can predict more benefit at the highest dose. Compare plausible models using the same endpoint, candidate doses, and utility wherever possible, so that the result isolates structural sensitivity rather than combining unrelated changes.
Worked Example: A Plateau Assumption Versus Continued Benefit
Suppose two plausible models give the following hypothetical efficacy predictions for doses of 10, 20, and 30 mg. The adverse-event probabilities are the same under both models: 0.04, 0.10, and 0.20. Use \(U(d)=E(d)-1.5A(d)\).
| Dose | Plateau-model efficacy | Continued-increase efficacy | Adverse-event probability |
|---|---|---|---|
| 10 mg | 0.45 | 0.45 | 0.04 |
| 20 mg | 0.65 | 0.60 | 0.10 |
| 30 mg | 0.68 | 0.78 | 0.20 |
Under the plateau model, the utilities are \(0.45-1.5(0.04)=0.390\), \(0.65-1.5(0.10)=0.500\), and \(0.68-1.5(0.20)=0.380\). The 20 mg dose is preferred. Under the continued-increase model, they are \(0.390\), \(0.60-0.15=0.450\), and \(0.78-0.30=0.480\). The 30 mg dose is preferred. The selected dose depends on whether additional efficacy at 30 mg is considered plausible enough to offset the higher adverse-event probability.
These predictions are hypothetical and do not establish which model fits any real data. In an application, compare model fit and predictive adequacy as well as the decision outputs; sensitivity analysis does not make an inadequate model acceptable.
A Practical Analysis and Its Diagnostics
A clear analysis begins with a reference decision and then varies inputs in a structured way. Record the selected dose, the decision rule, the reference assumptions, and the clinically defensible ranges before interpreting results. A compact workflow is:
State candidate doses, the outcome definitions, constraints, and the rule used to select a dose.
List thresholds, utility weights, prior choices, and structural assumptions that are uncertain or debatable.
Use one-way analyses to understand individual effects, then examine plausible joint scenarios where inputs may interact.
Show where the preferred dose changes, which assumptions cause the change, and whether the change is clinically consequential.
Useful diagnostics include checking that the decision rule is applied consistently, that all candidate doses are evaluated on the same outcome scale, and that a reported switching point actually changes the maximum or feasible dose. For continuous inputs, calculate or locate switching boundaries rather than relying only on a few selected grid points. For joint changes, inspect combinations: two inputs that each have little effect alone can jointly reverse the decision.
A common mistake is to call a dose “robust” because it remains selected under several small, convenient changes to one input. The correction is to define a defensible range from clinical knowledge and plausible model alternatives, and to test interactions where relevant. Robustness is always relative to the assumptions examined; it is not proof that the dose is optimal under every possible model or preference.
Choosing a Sensitivity Approach and Recognizing Its Limits
One-way analysis is transparent and helpful for finding individual switching boundaries, but it can miss interactions. Scenario analysis evaluates coherent combinations—for example, a stricter response threshold together with a stronger safety penalty—but depends on specifying plausible scenarios. A grid or contour analysis can display dose regions across two continuous inputs. These approaches answer different questions and can be used together.
Sensitivity analysis is also distinct from probabilistic uncertainty analysis. Varying an assumed utility weight across a range shows how the decision depends on that preference; assigning a probability distribution to the weight and propagating it answers a different question about uncertainty across possible inputs. Neither should be described as though it automatically captures uncertainty in the clinical estimates. The selected dose may also be sensitive to data uncertainty, extrapolation beyond tested doses, or an unexamined structural assumption.
A useful report names both stable and unstable decisions. For example: “The 20 mg dose remains preferred across the prespecified response thresholds and prior specifications, but the preferred dose changes from 20 mg to 30 mg if the efficacy increase above 20 mg follows the continued-increase model.” This communicates what the evidence supports and what remains contingent, without presenting sensitivity as a test that proves one dose is correct.
Check Your Understanding
Answer each question, then compare with the solutions.
- A target-attainment rule selects the lowest dose with response probability at least 0.75. What must be held distinct when varying the clinical response threshold?
- For two doses, utilities are \(U_A=0.60-0.10\lambda\) and \(U_B=0.72-0.25\lambda\). Find the switching value of \(\lambda\) and identify the preferred dose on either side for \(\lambda\geq0\).
- Why can changing a prior alter a Bayesian dose decision even if the observed data are unchanged?
- A dose remains preferred across several tested utility weights, but a different plausible dose-response model selects another dose. Is the original decision robust? Explain.
Solutions. (1) Keep the response definition separate from the target probability of 0.75: the clinical threshold defines who counts as a responder, while 0.75 is the required probability of that response. (2) Set utilities equal: \(0.60-0.10\lambda=0.72-0.25\lambda\), so \(0.15\lambda=0.12\) and \(\lambda=0.800\). At \(\lambda=0\), B has greater utility; A is preferred for \(\lambda>0.800\), and they tie at 0.800. (3) A prior contributes information to the posterior, changing posterior estimates or probabilities used by the decision rule; the effect depends on the likelihood, prior, and decision rule. (4) It is robust to the examined utility weights but not to the examined model assumption. State that scope explicitly and investigate whether the alternative model is plausible and adequate.