Why an Envelope Is Needed
A conditional confidence interval answers a question under one specified sensitivity assumption. But when several assumptions remain plausible, selecting just one interval can hide how much the estimated treatment effect depends on that choice. A confidence envelope summarizes the range of values covered by the conditional intervals across a prespecified sensitivity set. Its width reflects both sampling uncertainty and variation across assumptions.
The envelope does not decide which assumption is correct, nor does it average the assumptions. It is a conservative way to report uncertainty about an effect when the true sensitivity value is not known but is taken to lie in a bounded set. The introductory material on confidence intervals conditional on each assumption supplies the underlying intervals; here the task is to combine them correctly.
The union and its hull are the same when the union is itself an interval. If \(\mathcal S\) has separated pieces, or the interval limits vary discontinuously, the union may have gaps; the hull fills those gaps. Reporting the hull is convenient and conservative, but it can include values that are not in any conditional interval. The set being enveloped must be stated: a confidence envelope over \(\mathcal S\) is not a claim about assumptions outside \(\mathcal S\).
Constructing the Envelope
Suppose the conditional interval is a two-sided Wald interval. If \(\hat\theta(\eta)\) is the estimate under assumption \(\eta\), \(\widehat{SE}(\eta)\) its standard error, and \(q\) a critical value for the intended confidence level, then \(C(\eta)=[\hat\theta(\eta)-q\widehat{SE}(\eta),\hat\theta(\eta)+q\widehat{SE}(\eta)]\). The envelope endpoints are the smallest lower limit and largest upper limit over the entire set. The critical value and standard-error method must be appropriate for the conditional inference; taking an envelope cannot repair an invalid conditional interval.
Name the treatment effect \(\theta(\eta)\), its scale and units, and the bounded sensitivity set \(\mathcal S\). Treat this set as fixed for the stated calculation.
For each \(\eta\), calculate \(L(\eta)\) and \(U(\eta)\) using a conditional confidence procedure with justified assumptions.
Find \(\inf_{\eta\in\mathcal S}L(\eta)\) and \(\sup_{\eta\in\mathcal S}U(\eta)\), checking interior points as well as boundaries when needed.
Give the envelope and explain that its coverage concerns the effect under the true value of \(\eta\), provided that value lies in \(\mathcal S\) and the conditional procedure is valid.
Why the Coverage Is Conservative
The key argument is a set-containment argument. Suppose there is a fixed true sensitivity value \(\eta_0\in\mathcal S\), and the conditional interval has coverage at least \(1-\alpha\) for \(\theta(\eta_0)\). Whenever \(C(\eta_0)\) covers that effect, the union over all \(\eta\in\mathcal S\) also covers it, because \(C(\eta_0)\) is one of the intervals in the union. Therefore:
The same containment applies to the interval hull, which contains the union. This justifies at least nominal coverage for the effect corresponding to the true sensitivity value, under the stated conditional coverage guarantee. The envelope may be wider than necessary because it includes intervals for assumptions that are not true. That extra width is the conservative price of retaining the range of plausible assumptions, not evidence that all effects in the envelope are equally plausible.
This is a pointwise guarantee for the effect at the true \(\eta_0\). It is not a guarantee that all effects \(\theta(\eta)\), for every \(\eta\in\mathcal S\), are simultaneously covered by their respective intervals with probability at least \(1-\alpha\). Simultaneous coverage is a stronger requirement and generally needs a simultaneous procedure, such as a band with appropriate joint error control. Likewise, the argument presumes the true sensitivity value belongs to the set; the envelope offers no coverage guarantee if that assumption fails.
Worked Examples
Worked Example: A Linear Effect Across a Continuous Set
Hypothetical data. Suppose an analysis estimates a treatment effect in symptom-score points as \(\hat\theta(\eta)=2.4+0.8\eta\), with standard error \(0.6\) points for every \(\eta\in[-1,1]\). Assume the conditional estimates are approximately normal, the standard error is valid under each assumption, and a 95% Wald interval uses \(q=1.96\). These are assumptions for this hypothetical calculation, not conclusions from the numbers alone.
The margin of error is \(1.96(0.6)=1.176\) points, so the conditional interval is \(C(\eta)=[2.4+0.8\eta-1.176,\ 2.4+0.8\eta+1.176]\). Since both limits increase with \(\eta\), the smallest lower limit occurs at \(\eta=-1\) and the largest upper limit at \(\eta=1\).
Thus the envelope is approximately \([0.424,4.376]\) symptom-score points. Under the assumptions above, it provides at least 95% coverage for the effect at the true \(\eta\), if that value lies in \([-1,1]\). It is wider than any single conditional interval because it spans the intervals associated with the full range of assumptions.
Worked Example: An Interior Assumption Gives an Extreme Limit
Hypothetical data. Let \(\eta\in[-1,1]\), let \(\hat\theta(\eta)=1.2+0.5\eta-0.4\eta^2\) in outcome-score points, and suppose the standard error is \(0.3\) points throughout the set. Assume valid approximately normal conditional inference and use \(q=1.96\) for a 95% interval. The margin is \(1.96(0.3)=0.588\) points.
Here \(L(\eta)=\hat\theta(\eta)-0.588\) and \(U(\eta)=\hat\theta(\eta)+0.588\). Both have the same extrema as \(\hat\theta(\eta)\). To find its extrema, differentiate: \(\hat\theta'(\eta)=0.5-0.8\eta\), which is zero at \(\eta=0.625\). The second derivative is \(-0.8<0\), so this interior point is a maximum. At the boundaries, \(\hat\theta(-1)=0.3\) and \(\hat\theta(1)=1.3\); at the interior maximum, \(\hat\theta(0.625)=1.35625\). The minimum is \(0.3\), at \(\eta=-1\).
The envelope is approximately \([-0.288,1.944]\) score points. Checking only the endpoints would give an upper limit of \(1.3+0.588=1.888\), which misses the actual maximum. This example shows why the optimization must cover the full sensitivity set, including interior stationary points.
Worked Example: A Disconnected Sensitivity Set
Hypothetical data. Suppose scientific considerations retain only two sensitivity values, \(\mathcal S=\{0,2\}\). At \(\eta=0\), an estimated treatment effect is \(-0.20\) milligrams per deciliter with standard error \(0.25\); at \(\eta=2\), it is \(1.40\) milligrams per deciliter with standard error \(0.30\). Assume valid approximately normal conditional intervals at both values and use \(q=1.96\).
At \(\eta=0\), the margin is \(1.96(0.25)=0.490\), giving \(C(0)=[-0.20-0.490,-0.20+0.490]=[-0.690,0.290]\) milligrams per deciliter. At \(\eta=2\), the margin is \(1.96(0.30)=0.588\), giving \(C(2)=[1.40-0.588,1.40+0.588]=[0.812,1.988]\) milligrams per deciliter.
The union is \([-0.690,0.290]\cup[0.812,1.988]\), which has a gap. Its interval hull is \([-0.690,1.988]\) milligrams per deciliter. The hull is a concise envelope, but values in the gap are not in either conditional interval. Assuming one of the two sensitivity values is true and the conditional intervals are valid, the union covers the effect at that true value with at least the conditional nominal coverage. The hull also covers it, but is less specific about the structure of the set.
Diagnostics, Alternatives, and Limitations
A reliable envelope calculation requires more than evaluating a few attractive scenarios. Check that the sensitivity set was justified and bounded before examining which assumptions yield favorable conclusions; that the effect and standard-error functions use the same model and scale throughout; and that the numerical optimization finds extrema over the whole set. For a smooth scalar set, inspect boundaries and interior critical points. For multiple parameters, grid calculations can help reveal the shape, but a coarse grid alone may miss extrema; use suitable optimization or verify the bounds analytically.
Also check the conditional inference itself. If standard errors change with \(\eta\), the lower and upper limits need not be optimized at the same values as the point estimate. If the conditional distribution is poorly approximated by a normal distribution, the Wald limits may be unreliable even when computed exactly. The envelope inherits those shortcomings. Report the confidence level, sensitivity set, effect units, conditional interval method, and whether the reported object is the union or its hull.
| Approach | What it reports | Main caution |
|---|---|---|
| One conditional interval | Uncertainty under one specified assumption | Does not reflect variation across other plausible assumptions |
| Confidence envelope | Union, or interval hull, of conditional intervals over a set | Can be broad; pointwise coverage is not simultaneous coverage of all effects |
| Simultaneous confidence band | Joint coverage for a family of effects indexed by assumptions | Requires a method designed for joint coverage and may be wider |
The envelope is useful when the sensitivity set is meaningful but no single assumption is privileged. Its width can be driven by a large range of assumed effects, large standard errors, or both; the envelope alone does not separate those sources. It also does not assign probabilities to sensitivity values, establish that the set is scientifically complete, or resolve missing-data assumptions. A narrow envelope over an implausibly narrow set can be less informative than a wider envelope over a defensible set.
Common Pitfall
A related error is to calculate limits only at the ends of a bounded set. That works only when the relevant limits attain their extrema there, which must be justified—for example, by monotonicity. Curved or non-monotone conditional estimates and changing standard errors can produce interior extrema.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If a conditional interval has 90% coverage at the true \(\eta_0\in\mathcal S\), what coverage statement follows for the envelope?
- For \(\hat\theta(\eta)=3-\eta^2\) and \(\eta\in[-2,2]\), with constant standard error \(0.2\) and \(q=1.96\), find the envelope endpoints.
- Why can evaluating only the endpoints of a sensitivity interval give the wrong envelope?
- Does an envelope formed from pointwise 95% intervals ensure that all indexed effects are simultaneously covered with 95% probability?
Solutions. (1) The envelope covers \(\theta(\eta_0)\) with at least 90% coverage, provided the conditional procedure is valid and \(\eta_0\) is in the set; this follows because it contains \(C(\eta_0)\). (2) The margin is \(1.96(0.2)=0.392\). The estimate has maximum \(3\) at \(\eta=0\), and minimum \(-1\) at \(\eta=-2\) and \(2\). Thus the smallest lower limit is \(-1-0.392=-1.392\), and the largest upper limit is \(3+0.392=3.392\). (3) A conditional limit can have an interior minimum or maximum, especially when the estimate or standard error varies non-monotonically. (4) No. Pointwise coverage for the effect at the true sensitivity value is not simultaneous coverage for every indexed effect; a simultaneous procedure is needed for that stronger claim.