Why Condition the Interval on an Assumption?
A sensitivity analysis can produce a different treatment-effect estimate under each fixed assumption about outcomes that were not observed. Sampling uncertainty remains within each scenario: even if the assumption is held fixed, the estimate would vary across repeated samples. A confidence interval calculated separately at that fixed value describes this sampling uncertainty, conditional on the assumed model and sensitivity value. It does not express uncertainty about which sensitivity value is correct.
For a clinical treatment contrast, let \(\theta(\boldsymbol{\delta})\) denote the target treatment effect under a fixed sensitivity value \(\boldsymbol{\delta}\), and let \(\hat{\theta}(\boldsymbol{\delta})\) be its estimate from the observed trial data and the specified model. The sensitivity value may be a single parameter or a vector of arm-specific parameters. A confidence interval at one value is formed from that scenario’s estimate and its standard error—not from estimates across a range of scenarios.
The interval’s center and width answer different questions. Its center is the scenario-specific treatment-effect estimate. Its width is determined by the standard error and the confidence level (and, where relevant, the reference distribution and degrees of freedom). A sensitivity assumption can change the center without changing the width; refitting or changing the model’s variance estimate can also change the width. Neither feature can be inferred from the other.
Constructing an Interval at One Fixed Value
For a large-sample estimate with an approximately normal sampling distribution and a consistently estimated standard error, a two-sided Wald interval is
Here \(z_{1-\alpha/2}\) is the standard-normal quantile. For a 95% interval, it is approximately 1.96. When an analysis instead relies on a \(t\) reference distribution with specified degrees of freedom \(df\), use the corresponding \(t\) quantile rather than 1.96. The chosen procedure must match the sampling model; a normal critical value is not justified merely because an estimate is reported as a mean.
This construction requires a well-defined target effect under the fixed assumption, an estimator and standard error appropriate to that target, and a sampling distribution adequately approximated by the stated reference distribution. Any variance estimation needed for the specified analysis must be included. If, for example, the method estimates nuisance parameters or combines multiple sources of sampling variability, the reported standard error must account for them as required by that method. The sensitivity value itself is held fixed here, not estimated as an additional random quantity.
State the sensitivity value and the treatment-effect definition it implies. Do not switch assumptions partway through the calculation.
Use the analysis specified for that scenario, including the variance procedure appropriate to the model.
Use the appropriate quantile and confidence level, such as 1.96 for a two-sided 95% large-sample Wald interval.
Report the interval in the treatment-effect units and explain that its coverage is conditional on the specified scenario and model.
Worked Intervals and Their Interpretation
Worked Example: A 95% Interval at a Fixed Scenario
Suppose a hypothetical trial estimates a mean treatment-minus-control difference in a symptom score, in points, under one fixed missing-outcome assumption. The fitted conditional analysis gives \(\hat{\theta}=1.4\) points and \(SE=0.3536\) points. Assume the large-sample normal approximation is appropriate and the variance estimate includes the sampling variability required by this analysis.
For a two-sided 95% Wald interval, the margin of error is
Thus the interval is
The arithmetic uses the unrounded product for the endpoints: \(1.4-0.693056=0.706944\) and \(1.4+0.693056=2.093056\), which round to the displayed values. Under repeated sampling, if this same model and sensitivity assumption were fixed and the interval procedure’s conditions held, about 95% of intervals constructed this way would cover the corresponding scenario-specific treatment effect. This particular interval is not a 95% probability statement about the fixed effect or about the plausibility of the assumption. Since the entire interval is above zero, it is also inconsistent with a zero effect at the corresponding two-sided 5% level under this procedure; that conclusion is conditional on the scenario and model.
Arm-Specific Assumptions and Changing Centers
A useful way to see how a fixed assumption affects the center is to express the treatment contrast as an adjusted contrast. Let arm 1 be treatment and arm 0 be control. Suppose \(\delta_a\) is the assumed difference, in outcome units, between the mean for missing participants in arm \(a\) and the reference mean used for that arm. Let \(r_a\) be the proportion missing in that arm. In a simple pattern-mixture adjustment with fixed response fractions, the adjusted arm mean is shifted by \(r_a\delta_a\). The treatment-minus-control contrast therefore shifts by \(r_1\delta_1-r_0\delta_0\):
This expression explains the direction of the center’s movement in this simple setup. A positive departure in the treatment arm raises the treatment-minus-control contrast; a positive departure in the control arm lowers it. It does not, by itself, determine the standard error. The conditional analysis’s variance procedure determines that quantity. For instance, if only fixed offsets are added to the same estimated arm means and the variance calculation is unchanged, the standard error can remain the same. If the conditional model or its estimated variance changes across scenarios, the standard errors can differ.
Worked Example: Contrasts Under Two Fixed Arm Assumptions
Consider hypothetical continuous outcomes measured in points. At the reference scenario \((\delta_1,\delta_0)=(0,0)\), suppose the treatment-minus-control estimate is \(1.4\) points. The proportions missing are \(r_1=0.20\) in treatment and \(r_0=0.10\) in control. Assume each scenario’s conditional analysis uses an appropriate large-sample normal interval; its fitted standard error is reported below.
For scenario A, fix \(\delta_1=-2\) points and \(\delta_0=1\) point. The adjusted estimate is
Suppose the conditional fit gives \(SE=0.50\) points. Its 95% interval is
For scenario B, fix \(\delta_1=2\) points and \(\delta_0=-1\) point. Then
Suppose this conditional fit gives \(SE=0.30\) points. Its 95% interval is
The point estimate rises by \(1.0\) point from scenario A to B, while the interval width falls from \(1.960\) points to \(1.176\) points. The differing standard errors here are hypothetical outputs of the stated conditional fits; they are not implied by the weighted-contrast formula. These two intervals describe sampling uncertainty under their respective fixed assumptions. They do not show that scenario B is more plausible, nor does the narrower interval establish that its assumption is better supported. The zero value is inside scenario A’s interval and outside scenario B’s, so the zero-effect conclusion differs across these assumptions.
This example also shows why reporting only interval endpoints can obscure what changed. A reader should be able to identify the fixed sensitivity value, the resulting point estimate, the standard error, the confidence level, and the target effect scale for each interval.
Center, Width, and a Second Variance Calculation
Worked Example: The Same Estimate Shift Can Accompany a Wider Interval
Suppose a hypothetical analysis reports a conditional estimate of \(1.4\) points with \(SE=0.3536\) under a reference assumption, as in the first example. Under another fixed assumption, the model gives an estimate of \(0.9\) points and \(SE=0.60\) points. Assume the large-sample normal approximation is appropriate for both fits.
The reference interval has width approximately
and its endpoints are approximately \([0.7069,\ 2.093]\) points. Under the second assumption, the 95% interval is
The estimated effect falls by \(0.5\) point, and the interval widens from about \(1.386\) points to \(2.352\) points. This is a separate possible pattern from the preceding example, where a higher estimate accompanied a narrower interval. The direction and size of center and width changes depend on the conditional estimates and standard errors; there is no general rule that they move together.
In practice, verify that each interval uses the standard error belonging to its own fixed scenario. Reusing a reference-scenario standard error may be justified if the specified method makes it invariant, but it should not be done automatically. Conversely, different widths should not be attributed to the assumption without checking whether the variance estimator, sample information, or model specification also changed.
Diagnostics, Alternatives, and Limitations
A conditional interval is useful because it makes sampling precision visible at a stated sensitivity value. It can be reported in a scenario table alongside the estimate and standard error. It is not a substitute for stating which assumptions were fixed or how those assumptions relate to the clinical question. The introductory material on sensitivity analysis distinguishes sampling uncertainty from assumption uncertainty; keeping those two sources separate is essential when interpreting these intervals.
Useful checks include confirming the effect direction and units, verifying that the sensitivity value is the same for the estimate and its standard error, checking the critical value and confidence level, and recomputing both endpoints. Look for a variance estimate that is implausibly small, unstable, or based on a method whose assumptions are not met. A normal approximation may be poor with sparse data, extreme estimates, or highly skewed sampling distributions; in such cases, an analysis with a justified alternative interval procedure may be needed. The interval’s nominal coverage is not guaranteed when the model or variance assumptions fail.
| Reported quantity | What it communicates | What it does not communicate |
|---|---|---|
| Conditional estimate | The estimated treatment effect at the stated fixed assumption | How precise the estimate is, by itself |
| Conditional confidence interval | Sampling uncertainty under the stated model and fixed assumption | Uncertainty about which assumption is correct |
| Interval width | Precision as reflected by the standard error and interval procedure | Clinical importance or plausibility of the scenario |
A common alternative is to report only the interval at a single reference assumption. That can be concise, but it hides how the conditional estimate and precision behave under other assumptions being considered. Another mistake is to combine scenario-specific intervals into one interval without a defined inferential procedure. The intervals are conditional on different targets or models; taking their average, or treating their endpoints as though they formed a conventional confidence interval, does not in general preserve a stated coverage level. A confidence interval at each fixed value should be labelled as such. Any summary across a set of sensitivity values requires its own definition and method.
Finally, confidence intervals do not remove sensitivity to untestable assumptions about missing outcomes. A narrow interval can be precise conditional on a scenario while that scenario remains uncertain. Conversely, a wide interval may reflect sampling variability rather than a larger range of plausible assumptions. State the conditional interpretation explicitly so that readers do not mistake one type of uncertainty for the other.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- At one fixed assumption, an estimate is \(2.0\) points with \(SE=0.40\) points. Calculate a two-sided 95% large-sample Wald interval.
- In the weighted-contrast setup, \(r_1=0.25\), \(r_0=0.10\), and \(\hat{\theta}(0,0)=1.0\) point. Find the estimate at \(\delta_1=2\) and \(\delta_0=1\) point.
- Two scenarios have estimates of \(1.2\) and \(1.8\) points, with standard errors \(0.50\) and \(0.30\) points, respectively. Which interval is wider at the same normal critical value, and why?
- What does a 95% confidence interval conditional on a fixed sensitivity value say about uncertainty, and what does it not say?
Solutions. (1) The margin is \(1.96(0.40)=0.784\) points, so the interval is \(2.0\pm0.784=[1.216,\ 2.784]\) points. (2) The adjusted estimate is \(1.0+(0.25)(2)-(0.10)(1)=1.4\) points. (3) The second interval is narrower because its standard error is smaller: at a common critical value, width equals twice the critical value times the standard error. (4) Under repeated sampling, the stated procedure has approximately 95% coverage for the scenario-specific effect if its model and interval conditions hold. It does not express uncertainty about the sensitivity value’s plausibility or assign a 95% probability to the fixed effect lying in the realized interval.