Reporting Identified Intervals Without Overclaiming

What an Identified Interval Says

Partial identification does not produce a single value for the target effect. Instead, the data and stated assumptions may restrict the effect to a range of values. Reporting that range well requires more than listing two numbers: readers need to know which effect is being bounded, what assumptions yield the bounds, and how sampling uncertainty affects the reported endpoints.

Let \(\tau\) denote a scalar target effect, such as the average treatment effect in a specified target population. Under a stated set of identifying assumptions, suppose the values of \(\tau\) compatible with the population data and assumptions form the interval \(\mathcal I=[L,U]\). Here \(L\) and \(U\) are the lower and upper endpoints of the identified set. They are population quantities; in an empirical analysis, they are usually estimated by \(\hat L\) and \(\hat U\).

Definition: An identified interval \([L,U]\) is the set of values of a target parameter that are compatible with the observed-data distribution and the stated assumptions. If the set is an interval, the assumptions and data establish \(L\leq\tau\leq U\); in general, they do not establish that \(\tau\) equals either endpoint or any particular value between them.

The identified interval is not itself a confidence interval. The first describes what the population data and assumptions identify. The second describes sampling uncertainty in an estimate or set constructed from a sample. Because endpoints are estimated, an empirical report should normally make both distinctions explicit: the estimated identified interval is \([\hat L,\hat U]\), while a separate confidence procedure describes uncertainty about the population identified set.

What a Complete Report Needs

Before giving numbers, identify the target precisely. State the population, treatment contrast, outcome, time point, and effect scale. For example, “the average treatment effect on 12-month symptom score among eligible adults, in score points” is more informative than “the treatment effect.” If the target is a different population or a different effect, say so; bounds for one target do not automatically apply to another.

Next, summarize the restrictions that produce the interval. A reader should be able to tell whether the result relies on, for example, a specified limit on selection bias or a restriction on outcome values. State the restriction in interpretable terms and make clear that it is an assumption, not a fact established by the interval. A useful report also explains the data and estimation procedure used to obtain the endpoints, including any uncertainty procedure. It need not reproduce all derivations, but it should give enough information to understand what the numbers represent.

Target: name the population, effect, outcome, time point, and units.
Assumptions: identify the restrictions that define the identified set and explain their substantive meaning.
Endpoints: report \(\hat L\) and \(\hat U\), label them as estimated lower and upper endpoints, and state their units.
Uncertainty: describe the confidence procedure, its nominal level, and whether it covers both endpoints jointly or the entire identified set.
Interpretation: say which values remain compatible with the data and assumptions, without selecting one as the effect.

Worked Example: Reporting Bounds Under a Bias Limit

Suppose a hypothetical analysis targets the average treatment effect on a 12-month symptom score among eligible adults, measured in score points. The observed treatment-group contrast is estimated as 4.1 points. Under a stated sensitivity restriction, the absolute selection bias is no more than 2.7 points. Write a report that does not present a single effect as identified.

Let \(\Delta\) denote the observed contrast and \(B\) the selection bias, with the target effect represented as \(\tau=\Delta-B\). The assumed restriction is \(-2.7\leq B\leq2.7\). Substituting the estimated contrast gives the endpoint calculations:

\hat L=4.1-2.7=1.4,\qquad \hat U=4.1+2.7=6.8

Thus the estimated identified interval under this restriction is \([1.4,6.8]\) score points. A careful report would say: “For the average treatment effect on 12-month symptom score among eligible adults, the estimated identified interval is 1.4 to 6.8 points under the assumption that absolute selection bias does not exceed 2.7 points. The data and this restriction do not select a unique effect within that interval.”

This calculation treats the observed contrast as the input to the bound calculation; it does not, by itself, describe sampling uncertainty in either endpoint. The report should not imply that the bias restriction was verified from the observed contrast. It is a substantive assumption, and the interval’s interpretation is conditional on it.

Endpoint Estimates and Sampling Uncertainty

In a sample, the endpoints vary across repeated samples. Reporting \([\hat L,\hat U]\) without qualification can make the estimated interval appear more certain than it is. A confidence procedure should therefore be described separately from the identified interval. One possible summary is a confidence region \([l,u]\) for the entire population identified set \([L,U]\): its intended coverage property is

P\bigl([L,U]\subseteq[l,u]\bigr)\geq 1-\alpha.

This statement concerns the long-run performance of the procedure over repeated samples under its assumptions. It does not mean there is probability \(1-\alpha\) that the fixed population interval is inside the particular region observed in this one analysis. Nor should marginal confidence intervals for the two endpoints automatically be described as a joint confidence region: separate coverage statements do not necessarily guarantee that both endpoints are covered together.

For example, a conservative joint region can be formed from a one-sided lower confidence bound for \(L\) and a one-sided upper confidence bound for \(U\). With an error allocation of \(\alpha/2\) to each tail, if each one-sided bound has error probability at most \(\alpha/2\), the probability that either endpoint fails its required coverage is at most \(\alpha\). This follows from the union bound: the probability of at least one failure is no greater than the sum of the two failure probabilities. The resulting region contains the whole identified interval whenever both endpoint bounds hold.

P(L\geq l,\ U\leq u) \geq 1-P(L<l)-P(U>u) \geq 1-\alpha

The exact construction depends on the estimator and its sampling behavior. In particular, a normal approximation and a standard error must be justified for the setting at hand; inference near a boundary or for nonsmooth endpoints may require a different procedure. State the actual method used rather than presenting a generic confidence region as if it applied automatically. The earlier material on confidence sets for partially identified effects, inference when a bound endpoint is on a boundary, and bootstrap diagnostics for estimated bounds addresses these distinct inferential issues.

Worked Example: Adding a Joint Confidence Region

In a hypothetical study, the estimated identified interval for a mean difference in symptom scores is \([2.1,8.7]\) points. Suppose the estimated lower endpoint has standard error 0.8 points and the estimated upper endpoint has standard error 1.1 points. Assume the endpoint estimators are approximately normal, and use one-sided normal bounds with 97.5% marginal coverage for each endpoint. The critical value is approximately 1.96.

The lower confidence bound for \(L\) is:

$$ l=2.1-1.96(0.8)=2.1-1.568=0.532 $$

The upper confidence bound for \(U\) is:

$$ u=8.7+1.96(1.1)=8.7+2.156=10.856 $$

By the union-bound argument, the resulting region \([0.532,10.856]\) has at least 95% coverage for the entire identified interval, provided the stated one-sided endpoint approximations are valid. Rounded sensibly, the report could give an estimated identified interval of 2.1 to 8.7 points and a conservative 95% confidence region of 0.53 to 10.86 points for that interval. The region’s wider limits express sampling uncertainty; they do not replace the estimated endpoints or identify a new point effect.

The assumptions matter here: the normal approximations must be appropriate for both endpoints, and the reported standard errors must correspond to the endpoint estimators. If those conditions are not supported, this calculation is not justified merely because standard errors are available.

Why the Midpoint Is Not the Effect

The midpoint \(M=(L+U)/2\) is a summary of the interval’s location. Unless the assumptions or estimand specifically identify \(M\) as the target, the midpoint is not a point-identified effect. It is possible for every value in the interval to be compatible with the data and assumptions, while no one of those values is singled out. Reporting a midpoint as “the estimated effect” silently adds a selection rule that the identification analysis did not supply.

Worked Example: An Interval Center Is Only a Summary

Suppose a hypothetical analysis estimates that the average treatment effect on a continuous outcome lies in \([-3,11]\) units under its stated assumptions. Calculate the midpoint and explain how to report it.

The midpoint is:

\hat M=\frac{-3+11}{2}=\frac{8}{2}=4\text{ units}

The calculation is correct, but it does not establish that the effect equals 4 units. The identified interval still allows values from \(-3\) through 11 units. A suitable report is: “The estimated identified interval is \(-3\) to 11 units; its midpoint is 4 units, shown only as a descriptive center and not as a point-identified effect.” If a substantive decision depends on whether the effect is positive, this interval alone does not establish a positive effect, because it includes negative values as well as positive ones.

The interval and its midpoint answer different questions. The interval reports the range compatible with the analysis; the midpoint compresses that range to one descriptive number and discards information about the remaining compatible values.

Common Reporting Pitfalls

Calling the identified interval a confidence interval. An identified interval arises from the population restrictions and data distribution; a confidence procedure addresses sampling uncertainty. Name each separately.
Presenting the midpoint as the effect. The midpoint is not generally selected by the identification assumptions. Label it descriptive if it is included at all.
Leaving assumptions implicit. An interval is conditional on the restrictions that produced it. State those restrictions and avoid implying that the data alone establish them.
Giving endpoints without a target or units. A range such as “1 to 7” cannot be interpreted reliably without the effect, population, outcome scale, and time point.
Claiming joint coverage from separate intervals. State whether the uncertainty procedure covers each endpoint separately or the whole identified set jointly.
Overstating a sign or decision conclusion. If the interval includes zero, for example, the bounds do not establish that the effect is strictly positive or strictly negative.

A transparent report does not need to make the result sound more decisive than it is. It can state the range plainly, identify the assumptions on which it depends, and explain how much sampling uncertainty surrounds its estimated endpoints. That allows readers to distinguish what the analysis establishes from what remains unresolved.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

An analysis estimates an identified interval of 2 to 9 days for a target effect. What information besides the endpoint values should a report include?
An estimated identified interval is \([1,7]\) score points. A report calls 4 points “the identified treatment effect.” What is wrong with that wording?
For endpoint estimators with valid one-sided error probabilities of 0.025 each, what lower bound can be given for the probability that both endpoint bounds hold?
An estimated identified interval is \([-2,5]\) units. Does it establish a positive effect? Explain.
Why should an estimated interval and a confidence region for the population identified set be reported as different quantities?

Solutions. (1) Specify the target population, effect, outcome, time point, units, assumptions that define the interval, and the method and coverage of the uncertainty procedure. (2) The midpoint is \((1+7)/2=4\), but the assumptions have not selected it as the effect; values throughout the interval remain compatible. (3) By the union bound, the probability of either endpoint-bound failure is at most \(0.025+0.025=0.05\), so the probability both hold is at least \(1-0.05=0.95\). (4) No. The interval includes negative, zero, and positive values, so it does not establish a strictly positive effect. (5) The estimated interval summarizes the sample-based endpoint estimates, while the confidence region describes sampling uncertainty in the population identified set; they answer different questions.