Uniform Inference Across Sensitivity Parameters

Why inference must cover the sensitivity range

A sensitivity analysis often reports an identified set for each value of a sensitivity parameter. The resulting collection is a curve of lower and upper endpoints, not a single interval. If the analysis searches that curve for the value at which a conclusion changes, uncertainty must account for the search. A confidence interval calibrated separately at each sensitivity value does not, in general, provide the intended error control for conclusions selected after examining the whole curve.

The goal here is uniform inference: construct bands that cover the endpoint curves together over a specified range of sensitivity values. Such bands support statements like “the identified set remains positive throughout this range” or “a zero effect becomes compatible with the data by this value.” Those statements are only as reliable as the joint coverage of the curves on which they are based.

Setup and target

Let \(\gamma\) denote a sensitivity parameter in a prespecified set \(\Gamma\), such as an interval \([0,\bar\gamma]\). At each \(\gamma\), suppose the identified set for a scalar causal or structural parameter is an interval \([\mathcal{L}(\gamma),\mathcal{U}(\gamma)]\), where \(\mathcal{L}(\gamma)\) and \(\mathcal{U}(\gamma)\) are its lower and upper endpoints. The sensitivity analysis determines how the identified set changes as the assumptions indexed by \(\gamma\) change; the statistical problem is to cover both endpoint functions over all of \(\Gamma\).

The identified set at a fixed \(\gamma\) is not necessarily a confidence interval: it describes values compatible with the maintained assumptions and the population distribution. Sampling uncertainty is a separate issue. A confidence region for the identified set can be formed from endpoint bands. In particular, if the bands cover both endpoints, then \([\mathcal{L}^{-}(\gamma),\mathcal{U}^{+}(\gamma)]\) contains the true identified set at every \(\gamma\), provided the displayed outer endpoints are ordered.

We assume estimators \(\widehat{\mathcal{L}}(\gamma)\) and \(\widehat{\mathcal{U}}(\gamma)\) are available, with estimated standard errors \(\widehat{SE}_{L}(\gamma)\) and \(\widehat{SE}_{U}(\gamma)\). The sensitivity range \(\Gamma\) should be specified before inspecting the estimated curve, or any data-driven range selection must be included in the inference procedure. The analysis also requires suitable uniform regularity: informally, the endpoint estimation errors, after standardization, must be well approximated over the whole range by a joint limiting process, and the resampling method must consistently approximate that process. These conditions can fail at nonregular points, for example when a bound is determined by a changing minimum or maximum with ties; a routine smooth-process bootstrap is not automatically valid there.

Building a joint band

A common construction uses the largest absolute standardized error across both endpoint curves and all sensitivity values. The critical value is therefore calibrated to the most extreme deviation anywhere in the joint display, rather than to one endpoint at one chosen value.

Let \(c_{1-\alpha}\) be an estimate of the \(1-\alpha\) quantile of the distribution of this maximum. In practice, it is often obtained from a bootstrap or multiplier-bootstrap approximation to the joint endpoint error process. The resulting two-sided bands are

The same critical value is used because the target is joint coverage for both curves. The critical value need not equal a familiar pointwise normal quantile: it depends on the variability and dependence of errors across sensitivity values and between endpoints. A multiplier bootstrap, for example, applies common random weights to estimated influence contributions across the entire curve, computes the maximum standardized deviation for each draw, and uses the empirical \(1-\alpha\) quantile of those maxima. This preserves the estimated dependence relevant to the search.

Why the maximum gives simultaneous coverage

The basic justification is direct. If \(T_n\leq c_{1-\alpha}\), then every standardized error in its defining maximum is at most \(c_{1-\alpha}\). Thus both endpoint errors are within their bands at every \(\gamma\). Under uniform approximation and consistent critical-value estimation, choosing the quantile to satisfy \(P(T_n\leq c_{1-\alpha})\approx 1-\alpha\) gives simultaneous coverage approximately \(1-\alpha\).

This is the key distinction from pointwise inference. A pointwise interval controls the error probability at a fixed \(\gamma\), chosen without reference to the observed curve. It does not generally control the probability that at least one interval misses when many values are examined and a conclusion is reported at the most favorable one. The maximum calibration targets that joint event directly.

From endpoint bands to identified-set conclusions

The two-sided bands give a conservative outer confidence region for the identified set at each sensitivity value: \([\mathcal{L}^{-}(\gamma),\mathcal{U}^{+}(\gamma)]\). On the event of simultaneous endpoint coverage, the true lower endpoint is no lower than \(\mathcal{L}^{-}(\gamma)\) and the true upper endpoint is no higher than \(\mathcal{U}^{+}(\gamma)\). Hence the true identified set lies inside that outer region, for all \(\gamma\) together.

This construction enables statements about signs. If \(\mathcal{L}^{-}(\gamma)>0\) throughout a prespecified range, then, on the simultaneous coverage event, the entire identified set is positive throughout that range. If instead \(\mathcal{U}^{+}(\gamma)<0\), the entire set is negative. If neither condition holds, the data and assumptions do not establish that the identified set has a single sign at that sensitivity value. This is not evidence that zero is the true effect; it means zero has not been ruled out by the stated outer confidence region.

Tipping points and their uncertainty

A tipping point is a sensitivity value at which a specified conclusion changes. The definition must say which conclusion is being tracked. For a positive-effect claim, one useful inferential threshold is where the simultaneous lower confidence bound for the lower endpoint ceases to be positive. This threshold is a robustness limit for the statistical claim; it need not equal the sensitivity value where the true identified set itself first includes zero.

For example, if \(\mathcal{L}(\gamma)\) is known to decrease with \(\gamma\), its zero-crossing can be defined as the value \(\gamma_0\) satisfying \(\mathcal{L}(\gamma_0)=0\), if such a value exists. A simultaneous band supplies a confidence set for this crossing by inversion: \[ \{\gamma\in\Gamma:\mathcal{L}^{-}(\gamma)\leq 0\leq\mathcal{L}^{+}(\gamma)\}. \] Whenever the band covers the entire true curve, any true zero-crossing belongs to this set. Shape assumptions such as monotonicity can help summarize the resulting set, but they must be justified or specified rather than inferred merely from a jagged estimate.

Common mistake: treating a curve search as a single test

A frequent error is to calculate a pointwise interval at every sensitivity value and then report the value that most strongly supports the preferred conclusion, as if that value had been fixed in advance. The nominal pointwise confidence level does not generally survive this search. With many opportunities for an interval to miss, the probability of at least one miss can exceed the nominal error rate.

The correction is to calibrate a statistic that includes the search, such as the supremum of standardized errors across all sensitivity values and both endpoints. The range of values examined, the endpoints included in the maximum, and any data-driven selection rule should match the analysis being reported. A grid-based implementation must be sufficiently fine to represent the intended range; if the target is genuinely continuous, the procedure must justify how the continuum is controlled rather than silently treating a coarse grid as the entire parameter space.