Choosing Variance And Proportion Intervals In Practice

Choosing an Interval Begins With the Parameter

An interval method is not selected by habit alone. First identify the unknown parameter: a population variance, one population proportion, or a difference between two proportions. Then check the data model and sampling conditions that justify the method. A formula that is familiar but mismatched to the design can produce a precise-looking interval with no appropriate coverage interpretation.

This tutorial compares four methods: the exact chi-square interval for a variance under normal sampling, the Wald and Wilson score intervals for one proportion, and the unpooled Wald interval for a difference between two independent proportions. The interval procedures are aligned with the SOA VEE Mathematical Statistics syllabus.

Definition: A \(100(1-\alpha)\%\) confidence interval procedure is a method that, over repeated samples under its stated model and assumptions, produces intervals containing the fixed parameter with probability \(1-\alpha\). The confidence level describes the long-run behavior of the procedure; it is not a probability assigned to a fixed parameter after the data are observed.

A Practical Selection Guide

Start by naming the parameter and checking how the observations were collected. The table summarizes the main choices and their essential conditions.

Parameter and method	Model and conditions	Use with care when
Population variance \(\sigma^2\): exact chi-square interval	A random sample of independent observations from a normal population; sample variance uses denominator \(n-1\).	The population is substantially nonnormal, especially with a small sample.
One proportion \(p\): Wald interval	Independent Bernoulli observations with common probability \(p\); both observed success and failure counts should be adequate for a normal approximation.	Counts are small or \(\hat p\) is near 0 or 1; the interval can have poor coverage or leave \([0,1]\).
One proportion \(p\): Wilson score interval	Independent Bernoulli observations with common probability \(p\); obtained by inverting the score statistic.	The sample is extremely small; Wilson is usually better behaved than Wald, but discreteness still matters.
Difference \(p_1-p_2\): unpooled interval	Two independent samples, each consisting of independent Bernoulli observations with a common group-specific probability; success and failure counts are adequate in both groups.	Observations are paired, samples overlap, or either group has sparse counts.

For a proportion sample, write \(\hat p=x/n\), where \(x\) is the number of successes among \(n\) observations. A common practical check for a normal approximation is at least 10 observed successes and 10 observed failures. This is a guideline, not a theorem guaranteeing exact coverage. The Wilson interval often performs better than Wald even when the counts pass that check. If a problem specifically requests a method, follow that instruction and still state its limitations.

Variance: An Exact Interval Under Normal Sampling

Let \(X_1,\ldots,X_n\) be independent observations from \(N(\mu,\sigma^2)\), with both \(\mu\) and \(\sigma^2\) unknown. Define the sample variance by \(s^2=\sum_{i=1}^n(X_i-\bar X)^2/(n-1)\). Under this normal model, the statistic \((n-1)s^2/\sigma^2\) has a chi-square distribution with \(n-1\) degrees of freedom. This gives an exact interval for \(\sigma^2\).

Theorem: Let \(\chi^2_{\nu,\gamma}\) denote the lower-tail \(\gamma\) quantile of a chi-square distribution with \(\nu\) degrees of freedom: \(P(\chi^2_\nu\leq \chi^2_{\nu,\gamma})=\gamma\). For a normal random sample, a \(100(1-\alpha)\%\) confidence interval for the population variance is

\left( \frac{(n-1)s^2}{\chi^2_{n-1,\,1-\alpha/2}}, \frac{(n-1)s^2}{\chi^2_{n-1,\,\alpha/2}} \right).

The larger chi-square quantile goes in the lower endpoint, so the endpoints are ordered correctly. The interval for the population standard deviation \(\sigma\), if requested, is found by taking the square root of each variance endpoint. Exactness here depends on normality; the formula is not an exact, distribution-free variance interval.

Worked Example: A Variance Interval for a Normal Measurement

A hypothetical calibration study takes an independent random sample of \(n=10\) devices from a population of measurements modeled as \(N(\mu,\sigma^2)\), where both parameters are unknown. The observed sample variance, calculated with denominator \(n-1\), is \(s^2=16\) squared units. Find a 95% confidence interval for \(\sigma^2\), and explain why the method applies.

Method and assumptions. The question concerns a population variance, and the stated independent normal model permits the exact chi-square method. Here \(n-1=9\), \(\alpha=0.05\), and the relevant quantiles are \(\chi^2_{9,0.975}=19.0228\) and \(\chi^2_{9,0.025}=2.7004\).

Calculate the common numerator. It is \((n-1)s^2=9(16)=144\). Divide by the upper quantile for the lower endpoint and by the lower quantile for the upper endpoint:

\left(\frac{144}{19.0228},\frac{144}{2.7004}\right) =(7.5699,\ 53.3254).

Rounded to two decimal places, the 95% interval for \(\sigma^2\) is \((7.57,\ 53.33)\) squared units. As a check, \(7.57<53.33\); using the upper chi-square quantile in the first denominator also makes that endpoint the smaller one. The interval is wide, in part because the sample has only 10 observations.

Common mistake avoided: Reversing the quantiles would put the smaller chi-square value in the lower-endpoint denominator and produce an interval in the wrong order. The normality assumption is also essential to calling this procedure exact.

One Proportion: Wald or Wilson?

For independent Bernoulli observations with common success probability \(p\), the sample proportion is \(\hat p=x/n\). The Wald interval estimates the standard error by substituting \(\hat p\) for \(p\). For a two-sided confidence level \(1-\alpha\), let \(z_{1-\alpha/2}\) be the corresponding standard normal quantile. The Wald interval is

\hat p\ \pm\ z_{1-\alpha/2} \sqrt{\frac{\hat p(1-\hat p)}{n}}.

Its simplicity is useful, but its approximation can be poor, particularly when the observed proportion is near a boundary or the counts are small. It can even extend below 0 or above 1, although a proportion must lie in \([0,1]\). Do not treat the adequacy count check as a guarantee of coverage.

The Wilson score interval is obtained by inverting the score-statistic inequality. Writing \(z=z_{1-\alpha/2}\), its endpoints are

\frac{\hat p+\dfrac{z^2}{2n}}{1+\dfrac{z^2}{n}} \ \pm\ \frac{z}{1+\dfrac{z^2}{n}} \sqrt{\frac{\hat p(1-\hat p)}{n}+\frac{z^2}{4n^2}}.

Here is why the Wilson expression has this form. To test a candidate value \(p_0\), the score statistic uses the standard error under that candidate, \(\sqrt{p_0(1-p_0)/n}\), rather than under \(\hat p\). Inverting the condition that the statistic is no larger than \(z\) in absolute value gives the set of \(p_0\) satisfying

\frac{(\hat p-p_0)^2}{p_0(1-p_0)/n}\leq z^2 \quad\Longleftrightarrow\quad (n+z^2)p_0^2-(2n\hat p+z^2)p_0+n\hat p^2\leq 0.

The solutions of this quadratic inequality are the values between its two roots. Applying the quadratic formula and dividing the roots’ center and half-width by \(n+z^2\) gives the Wilson center and margin displayed above. Because the candidate-value standard error depends on \(p_0\), the resulting interval is generally not centered at \(\hat p\).

Worked Example: Choosing Between Wald and Wilson for One Proportion

A hypothetical quality audit independently inspects \(n=200\) units from a large process. Each unit either meets a specified criterion or does not; 34 meet it. Assume a common success probability \(p\) and independent Bernoulli outcomes. Calculate 95% Wald and Wilson intervals, then choose a practical report.

Method and conditions. The estimate is \(\hat p=34/200=0.17\); the observed counts are 34 successes and \(200-34=166\) failures. Both exceed 10, so the usual count check for Wald is satisfied. For 95% confidence, use \(z=1.96\).

Wald calculation. The estimated standard error is \(\sqrt{0.17(0.83)/200}=\sqrt{0.0007055}=0.02656\). The margin is \(1.96(0.02656)=0.05206\), so the interval is \(0.17\pm0.05206=(0.1179,\ 0.2221)\), rounded to four decimals.

Wilson calculation. The denominator for its center and margin is \(1+3.8416/200=1.019208\). Its center is \((0.17+3.8416/400)/1.019208=0.179604/1.019208=0.1762\). For the half-width, the quantity under the square root is \(0.0007055+3.8416/160000=0.00072951\). Thus the half-width is \(1.96\sqrt{0.00072951}/1.019208=0.0519\). The Wilson interval, to four decimals, is \((0.1243,\ 0.2282)\).

Choice and check. Both intervals lie within \([0,1]\), and the counts support using the Wald approximation. For a routine report where the method is not prescribed, Wilson is a sensible choice because it is generally better behaved than Wald; here it is centered above \(\hat p\), illustrating its asymmetry. The two intervals need not match. The common mistake is to assume that passing the count check makes Wald automatically preferable or exact.

Two Independent Proportions: Use Separate Standard Errors

Suppose independent samples of sizes \(n_1\) and \(n_2\) produce \(x_1\) and \(x_2\) successes. The estimates are \(\hat p_1=x_1/n_1\) and \(\hat p_2=x_2/n_2\). For the parameter \(p_1-p_2\), the unpooled Wald interval is

(\hat p_1-\hat p_2)\ \pm\ z_{1-\alpha/2} \sqrt{ \frac{\hat p_1(1-\hat p_1)}{n_1} + \frac{\hat p_2(1-\hat p_2)}{n_2} }.

This method requires independent samples and adequate success and failure counts in each group. Its standard error adds the two estimated variances because the two sample proportions are independent. It is not the same as a pooled standard error: pooling imposes a common-proportion estimate and is used for a different procedure. For an interval estimating an unrestricted difference, use the separate group estimates shown here. If observations are paired or matched, this independent-samples formula does not apply.

Worked Example: Comparing Two Independent Service Groups

A hypothetical evaluation independently samples customers from two service channels. In channel 1, 72 of 120 customers complete a requested process on the first attempt; in channel 2, 45 of 100 do so. Assume independent Bernoulli outcomes within and between channels, with probabilities \(p_1\) and \(p_2\). Find a 95% unpooled confidence interval for \(p_1-p_2\).

Method and conditions. The two samples are independent. The counts are 72 successes and 48 failures in channel 1, and 45 successes and 55 failures in channel 2, so each group passes the usual count check. The estimates are \(\hat p_1=72/120=0.60\) and \(\hat p_2=45/100=0.45\); their difference is \(0.15\).

Calculate the unpooled standard error. The two estimated variance contributions are \(0.60(0.40)/120=0.002000\) and \(0.45(0.55)/100=0.002475\). Their sum is \(0.004475\), so the standard error is \(\sqrt{0.004475}=0.06690\).

Calculate the interval. With \(z=1.96\), the margin is \(1.96(0.06690)=0.1311\). Therefore the interval is \(0.15\pm0.1311=(0.0189,\ 0.2811)\), rounded to four decimals. The lower endpoint is below the point estimate and the upper endpoint is above it, as required.

The interval estimates \(p_1-p_2\), in the stated channel order. Its positive endpoints indicate that the compatible values in this interval are positive differences; the interval is not a claim that the observed difference is exactly 0.15 in the populations. The common mistake is to pool the two proportions when calculating the interval’s standard error, rather than using each group’s own estimated variance.

Common Selection Traps

Using the variance interval without checking normality. The chi-square procedure is exact for independent observations from a normal population. It is not automatically exact for any population just because a sample variance is available.
Equating a valid range with good coverage. Truncating a Wald interval to \([0,1]\) does not fix the approximation that produced it. Consider Wilson when selecting a one-proportion interval.
Checking only the combined count. A two-proportion interval needs adequate successes and failures in each group, not merely a large total sample.
Ignoring the design. The unpooled difference interval assumes independent groups. Repeated measurements on the same people or matched pairs require a method that reflects the pairing.
Changing the parameter’s order. An interval for \(p_1-p_2\) becomes an interval for \(p_2-p_1\) by negating both endpoints and reversing their order. State the group order before interpreting the sign.

Key takeaway: Identify the parameter first, then verify the sampling model and conditions. Use the exact chi-square interval for a variance only under normal sampling; use Wilson as a strong default for one proportion; and use the unpooled interval for a difference only when the two samples are independent and their counts support the approximation.

Check Your Understanding

For each problem, select the method that fits the parameter and assumptions, then calculate or explain as requested. Use 95% confidence unless another level is stated.

A random sample of 8 independent measurements comes from a normal population with unknown mean and variance. The sample variance, using denominator \(n-1\), is 9. Using \(\chi^2_{7,0.95}=14.067\) and \(\chi^2_{7,0.05}=2.167\), find a 90% exact interval for the variance, to two decimals. (Answer: exact chi-square interval \((4.48,\ 29.07)\))
In 400 independent Bernoulli trials, 220 are successes. Find the 95% Wald interval using \(z=1.96\), to four decimals. (Answer: \((0.5012,\ 0.5988)\))
In 80 independent Bernoulli trials, 8 are successes. Using \(z=1.96\), calculate the Wilson 95% interval to four decimals. (Answer: \((0.0516,\ 0.1851)\))
Two independent groups have 56 successes among 100 observations and 36 successes among 90 observations. Find the unpooled 95% interval for \(p_1-p_2\), to four decimals. (Answer: \((0.0196,\ 0.3004)\))
A study records whether each of 25 people meets a criterion before and after a training program, with each person observed twice. Is the independent unpooled two-proportion interval appropriate? (Answer: No; the observations are paired, not independent between groups.)

What You'll Learn