Defining Plausible Sensitivity Sets

Why Define a Set of Assumptions?

A sensitivity analysis can show how a treatment-effect estimate changes as an assumption changes. But a collection of calculated results is difficult to interpret unless the assumptions being varied have a defensible range. The central task here is to specify which values of a sensitivity parameter are clinically and empirically plausible, and to explain why. The result is a sensitivity set: a declared range or region of assumptions for analysis, not a probability model for those assumptions.

The distinction matters. A wide range can reveal that conclusions depend on assumptions, while a narrow range can conceal important uncertainty if its limits are not justified. Conversely, an extremely broad range may include scenarios that are mathematically possible but clinically incoherent. Defining the set is therefore a substantive modeling decision, not a calculation that can be settled by the observed trial data alone.

A useful set is tied to a specific estimand, outcome scale, population, and model. If any of these change, the clinical meaning of a numerical bound may change too. The earlier material on Sampling Uncertainty Versus Assumption Uncertainty distinguishes variability from repeated sampling from uncertainty about assumptions. A sensitivity set addresses the latter: its endpoints are not automatically confidence limits, and membership does not mean that each value is equally likely.

State What the Parameter Means Before Setting Its Bounds

For illustration, suppose a model supplies a reference mean outcome for a group whose outcomes are unobserved. Define \(\delta\) as the difference, in outcome units, between the group’s actual mean and that reference mean:

This definition gives \(\delta\) a direction and units. If higher outcome scores are worse, a negative \(\delta\) means the unobserved group has a lower, more favorable mean than the reference; a positive \(\delta\) means a higher, less favorable mean. In a specified model, values of \(\delta\) may index different treatment-effect estimates, but the parameter’s definition alone does not determine which values are plausible.

Before choosing \(\mathcal{S}\), state the assumptions needed to interpret it:

• The estimand and outcome scale are fixed. Bounds on a symptom score, for example, cannot be transferred without justification to a risk difference.
• The parameter has a precise clinical interpretation, including the group it concerns, the reference quantity, the direction of subtraction, and the units.
• The reference model and the target population are defined well enough that clinical or external evidence can be judged relevant.
• The proposed bounds are supported by stated evidence or clinical constraints, and any transport from external evidence is discussed.

These conditions make the set interpretable; they do not prove that it contains the unknowable assumption value. A sensitivity set is a transparent range chosen for analysis, not a guarantee about where the truth lies.

Justifying Bounds With Clinical Knowledge and Constraints

External evidence is not the only basis for bounds. Clinical knowledge can rule out or deprioritize certain values, especially when it is stated in terms of the outcome and population rather than as an unexplained numerical preference. Physical or scale limits may also supply outer bounds. Such limits are often mathematically valid but too broad to be useful on their own.

A practical justification distinguishes at least two ideas: what is possible under the outcome scale, and what is plausible for this clinical question. For a scale from 0 to 60, a mean cannot be below 0 or above 60. But those limits may be far wider than the values supported by clinical evidence for a particular group at a particular time. A narrower range needs its own clinical or empirical rationale; it should not be presented as a consequence of the scale alone.

Bounds should be expressed in the parameter’s units and, where possible, connected back to an outcome that clinicians can understand. If reviewers can judge plausible values of the group mean more readily than plausible values of a model parameter, first state their judgment on the outcome scale and then translate it algebraically. This makes the role of the reference value and the direction of the parameter explicit.

Reconciling Evidence and Recording the Set

When several sources inform a bound, their role should be explicit. Historical data may be informative but differ from the target setting; clinicians may have strong knowledge of the setting but limited ability to quantify a shift. A defensible analysis explains how these sources were weighed rather than treating one convenient range as self-evident.

A concise record of the sensitivity set should state: the parameter definition and units; the chosen set and its endpoints; the evidence or clinical judgments supporting each endpoint; key transport assumptions; and any credible alternatives that were excluded. This record allows readers to assess the assumptions rather than infer their rationale from a final numerical result.

Alternatives, Diagnostics, and Limitations

A single reference assumption, such as \(\delta=0\), is easy to report but examines only one point. It is not a substitute for a sensitivity set when other values are plausible. At the other extreme, using every value permitted by the mathematical parameter space may be uninformative if most values are clinically irrelevant. A bounded set makes the scope of the sensitivity analysis explicit.

A probability distribution over \(\delta\) is a different modeling choice. It assigns probabilities or weights to values and requires additional justification. A set \(\mathcal{S}\) alone does neither: it does not say values near the center are more likely, that all values are equally likely, or that the endpoints have a particular coverage probability. Likewise, a confidence interval quantifies sampling uncertainty under a statistical procedure and its assumptions; it is not automatically a plausible range for an unobserved-outcome assumption.

Useful diagnostics focus on the construction, not just on the eventual treatment-effect calculations. Ask whether the units and sign are clear; whether the bounds refer to the stated group and time; whether the evidence is transportable; and whether the endpoints can be explained in clinical terms. Check whether changing the reference value would silently change the parameter’s meaning or the translated bounds. If the selected range is much narrower than the evidence, explain why. If it is so broad that it includes outcomes ruled out by the clinical setting, explain why those values remain relevant.

The principal limitation is that evidence and clinical judgment can constrain assumptions without identifying them. Different informed reviewers may define different plausible sets, especially when external populations differ or clinical experience is limited. Transparency does not eliminate that judgment; it makes the judgment visible and open to scrutiny. Nor does a well-justified set establish that its boundaries are correct. Conclusions remain conditional on the stated set and the model that maps each assumption to an estimand.

Common Pitfall

Wrong reasoning: “The sensitivity parameter is unknown, so every value in the chosen interval is equally likely; therefore this interval is a probability distribution.” An interval specifies membership, not probability. No probability weights follow from listing endpoints.

Correction: State only that values in \(\mathcal{S}\) are included for sensitivity analysis because of the stated clinical and empirical rationale. If probabilities are needed, specify and defend a separate probability model. Also avoid calling \(\mathcal{S}\) a 95% interval unless a defined procedure actually gives it that interpretation.