Two-Sided Tests and Confidence Interval Duality

Why Two-Sided Tests and Confidence Intervals Match

A two-sided hypothesis test asks whether a specified parameter value is compatible with the data in either direction. A confidence interval gives a range of parameter values compatible with the data under a stated coverage procedure. When the test and interval are constructed from the same model and the same tail allocation, these are two ways to express the same calculation: the test rejects a proposed value precisely when that value lies outside the matching confidence interval.

The confidence level must match the test’s significance level. A level-\(\alpha\) two-sided test corresponds to a \(1-\alpha\) confidence interval, not an interval with an unrelated confidence level. The interval also depends on the chosen model and test statistic; an arbitrary interval and an arbitrary test need not have this relationship.

The interval is random before data are collected because its endpoints depend on the sample. Once the data are observed, the endpoints are fixed. The confidence level describes the long-run coverage of the procedure; it is not the probability that a fixed parameter lies in the already calculated interval.

Inverting a Two-Sided Test

Suppose a test uses a statistic \(T_{\theta_0}\) whose null distribution is the same for every value \(\theta_0\), and suppose larger absolute values indicate stronger evidence against \(H_0:\theta=\theta_0\). Let \(c\) be chosen so that, under that null model, \(P(|T_{\theta_0}|\leq c)=1-\alpha\). The test rejects when \(|T_{\theta_0}|>c\); its non-rejection region is \(|T_{\theta_0}|\leq c\).

For the observed data, collect every proposed value \(\theta_0\) that would not be rejected. That collection is the confidence set. Its coverage follows directly from the test’s acceptance probability:

The middle expression uses the true value \(\theta\) as the candidate null value. Under that value, the test statistic has its null distribution, so the probability of falling in the acceptance region is \(1-\alpha\). This proves the coverage claim when the test has exact size \(\alpha\). If a test is conservative, the resulting coverage is at least \(1-\alpha\).

Known-Variance Normal Mean

Let \(X_1,\ldots,X_n\) be independent observations from a normal distribution with mean \(\mu\) and known variance \(\sigma^2\). Then \(\bar X\) has mean \(\mu\) and variance \(\sigma^2/n\). For testing \(H_0:\mu=\mu_0\) against \(H_1:\mu\neq\mu_0\), the standardized statistic is \(Z=(\bar X-\mu_0)/(\sigma/\sqrt n)\). Under the null, \(Z\) has a standard normal distribution.

If \(z_{1-\alpha/2}\) is the standard normal quantile with cumulative probability \(1-\alpha/2\), the equal-tail test rejects when \(|Z|>z_{1-\alpha/2}\). Solving the non-rejection inequality for \(\mu_0\) gives the interval:

Unknown-Variance Normal Mean

When the population is normal but \(\sigma^2\) is unknown, replace \(\sigma\) with the sample standard deviation \(S\). For independent observations from a normal population, the statistic \(T=(\bar X-\mu_0)/(S/\sqrt n)\) has a Student \(t\) distribution with \(n-1\) degrees of freedom under \(H_0:\mu=\mu_0\). The equal-tail level-\(\alpha\) test uses the critical value \(t_{1-\alpha/2,n-1}\), and inversion gives \(\bar X\pm t_{1-\alpha/2,n-1}S/\sqrt n\).

This exact result relies on a normal population and independent observations. The \(t\) critical value accounts for estimating the standard deviation; using the standard normal critical value instead does not give this exact small-sample procedure.

Variance Tests and Asymmetric Intervals

The duality does not require a symmetric interval. Suppose \(X_1,\ldots,X_n\) are independent observations from a normal population with unknown mean and variance \(\sigma^2\), and let \(S^2\) be the usual sample variance. Under \(H_0:\sigma^2=\sigma_0^2\), the statistic \((n-1)S^2/\sigma_0^2\) has a chi-square distribution with \(n-1\) degrees of freedom. A two-sided equal-tail test rejects when the statistic is below the \(\alpha/2\) quantile or above the \(1-\alpha/2\) quantile.

Writing those quantiles as \(\chi^2_{\alpha/2,n-1}\) and \(\chi^2_{1-\alpha/2,n-1}\), respectively, and solving the non-rejection inequality for \(\sigma_0^2\) gives:

The reversed placement of the quantiles in the denominators matters: dividing by the larger quantile gives the lower endpoint. Because the chi-square distribution is not symmetric, this interval is generally not centered at \(S^2\).

What the Duality Does—and Does Not—Say

For a fixed data set and a matching procedure, interval membership gives the test decision without a second calculation. If \(\theta_0\) is inside the interval, the corresponding test does not reject \(H_0:\theta=\theta_0\). If it is outside, the test rejects. A boundary value is normally not rejected when the test rejects only for a strict inequality; endpoint conventions should follow the stated rule.

This conclusion is specific to the matching construction. The test and interval must use the same probability model, statistic, significance level, and tail allocation. For example, an equal-tail 95% interval matches a two-sided 5% equal-tail test. A one-sided test is instead associated with a one-sided confidence bound, and a different interval procedure may produce different decisions.

The interval does not show that every value inside is equally plausible, nor does a non-rejection establish that the null value is true. It says that the particular matching test does not reject those values at the stated level. Interpreting a failure to reject as acceptance is a separate issue from this decision equivalence.