What Does “95% Confident” Mean?
In Interpreting the Confidence Interval for a Proportion, you learned to describe an interval as a range of plausible values for the population proportion \(p\). The confidence level explains how the interval-producing method performs over repeated samples. It does not mean there is a 95% probability that the particular interval you calculated contains \(p\).
The key distinction is between what changes and what stays fixed. For a specified population, the true proportion \(p\) is fixed, even if it is unknown. Before collecting a sample, the sample results are uncertain. A different random sample can produce a different \(\hat{p}\), and therefore a different interval. The interval changes from sample to sample; the population proportion does not.
Imagine repeatedly taking random samples from the same population, each of the same size, and constructing a 95% one-proportion \(z\)-interval in the same way. If the method’s conditions are met, about 95% of those intervals are expected to contain the fixed true proportion \(p\). The other intervals miss it. This is a statement about the performance of the method across many repetitions—not a probability assigned to one already-computed interval.
For an individual interval, the true \(p\) either falls between its endpoints or it does not. We may not know which is the case, but the interval is fixed once the sample has been observed. Saying “there is a 95% probability that \(p\) is in this interval” treats the fixed \(p\) as if it were changing randomly. That is not the AP Statistics interpretation of a confidence level.
Think About Repeated Samples
A helpful way to picture confidence is to imagine a long row of intervals produced from repeated random samples. Draw a vertical mark at the fixed value of \(p\). Intervals that cross the mark capture \(p\); intervals that fall entirely to one side miss it.
The 95% refers to the long-run fraction of intervals that cross the mark. It does not promise that exactly 95 of every 100 intervals will capture \(p\). In a particular group of 100, the capture count may be above or below 95 because the samples themselves vary. A larger number of repetitions will generally make the observed fraction more informative about the method’s long-run performance.
- Confidence level: Describes the long-run capture rate of a method used repeatedly.
- One computed interval: Either contains the fixed population proportion \(p\) or does not.
- Interval endpoints: Vary across random samples because \(\hat{p}\) varies.
Worked Example: Interpreting One 95% Interval
A hypothetical random sample of 120 adults is selected without replacement from a town population of 5,000. In the sample, 48 adults report using a reusable water bottle every day. A 95% one-proportion \(z\)-interval is calculated. Explain what the confidence level means and find the interval to make the interpretation concrete.
Let \(p\) be the true proportion of adults in this town who use a reusable water bottle every day. The goal is to estimate \(p\) with a 95% confidence interval.
The sample is stated to be random. The 10% limit is \(0.10(5000)=500\), and \(120\leq500\). There are 48 successes and \(120-48=72\) failures, both at least 10. The conditions support using a one-proportion \(z\)-interval.
The sample proportion is \(\hat{p}=48/120=0.40\). With \(z^*=1.96\), the estimated standard error is \(\sqrt{(0.40)(0.60)/120}\approx0.04472\). The interval is approximately \(0.40\pm1.96(0.04472)\), or \((0.3123,0.4877)\).
We are 95% confident that the true proportion of adults in this town who use a reusable water bottle every day is between about 0.3123 and 0.4877. The 95% describes the long-run capture rate of this interval method, not a 95% probability for this one interval.
Here are the calculations using the one-proportion \(z\)-interval formula:
Once this particular sample has been collected, its interval has particular endpoints. The fixed \(p\) is either within those endpoints or outside them. The 95% confidence level is not a statement that there is a 95% chance that this specific interval succeeded. It describes what the method is designed to do across repeated random samples.
What a Repeated-Sampling Simulation Can Show
A simulation can make the long-run idea visible. Suppose a computer repeatedly draws random samples from a hypothetical population whose true proportion is known to be \(p=0.30\). For each sample, it constructs a 95% confidence interval using the same method. Since the simulation sets the population proportion, it can check whether each interval captures \(0.30\).
This is different from an ordinary study, where \(p\) is unknown. In a real application, we cannot usually count which of our intervals capture the truth, because the true proportion is not known. A simulation with a specified \(p\) can evaluate the method’s performance under that model.
Worked Example: A Hypothetical Run of 20 Intervals
In an invented simulation, 20 random samples are drawn from a hypothetical population with \(p=0.30\). A 95% interval is calculated for each sample. Eighteen intervals contain \(0.30\); two do not. One of the intervals that misses is \((0.31,0.49)\), which lies above \(0.30\), and the other miss lies below \(0.30\). Describe the simulation result.
The observed capture rate is:
In this particular run, 90% of the intervals captured the true proportion. That is not evidence that a 95% method is supposed to capture exactly 95% in every batch. A group of only 20 repetitions can show noticeable variation. The long-run capture rate is the target; a finite simulation run is only one set of outcomes.
The two intervals that miss illustrate what “capture” means: their endpoints do not include the fixed value \(0.30\). The interval \((0.31,0.49)\) does not capture \(0.30\), even though it is a valid interval calculated by the method. A confidence level does not guarantee that every interval will include the true proportion.
Confidence Level Is Not the Same as Interval Width
The confidence level describes a method’s long-run capture rate. For the same sample data, choosing a higher confidence level generally uses a larger critical value and produces a wider interval. The greater width is the method’s way of aiming for a higher capture rate. It does not make the population proportion more variable, and it does not change the fixed value of \(p\).
The following example uses the same sample to compare two confidence levels. The calculations show how the intervals differ; the main point is that neither level should be interpreted as the probability that its particular interval contains \(p\).
Worked Example: Comparing 90% and 95% Confidence
A hypothetical random sample of 200 students is selected without replacement from a campus population of 4,000 students. Of those sampled, 84 report using a campus shuttle at least once a week. Compare the 90% and 95% one-proportion \(z\)-intervals and explain what their confidence levels mean.
Let \(p\) be the true proportion of all students on this campus who use a campus shuttle at least once a week. The sample is random. The 10% limit is \(0.10(4000)=400\), and \(200\leq400\). There are 84 successes and \(200-84=116\) failures, both at least 10. The conditions support using a one-proportion \(z\)-interval.
The sample proportion is \(\hat{p}=84/200=0.42\). Its estimated standard error, used for both intervals, is:
For a 90% interval, \(z^*\approx1.6449\). The margin of error is approximately \(1.6449(0.03490)=0.0574\), giving:
For a 95% interval, \(z^*=1.96\). The margin of error is approximately \(1.96(0.03490)=0.0684\), giving:
The 90% method is designed to capture the true proportion in about 90% of repeated samples under its conditions; the 95% method is designed to capture it in about 95% of repeated samples. For these data, the 95% interval is wider. Neither statement says that there is a 90% or 95% probability that the corresponding particular interval contains \(p\). Each interval either contains the fixed \(p\) or misses it.
Common Mistakes and What a Full-Credit Answer Says
- “There is a 95% probability that \(p\) is in this interval.” This assigns probability to the fixed population proportion after the interval has been calculated. Instead, say that the method captures \(p\) in about 95% of repeated samples, assuming its conditions are met.
- “95% of the population is in the interval.” The interval is a range of plausible values for a population proportion. It is not a range containing 95% of individuals.
- “95% of sample proportions fall inside this one interval.” The confidence level concerns intervals produced by repeated use of the method. It does not describe how many individual sample proportions fall in the observed interval.
- “Exactly 95 out of every 100 intervals will capture \(p\).” The 95% is a long-run rate, not a guarantee for every batch of samples. A finite set of intervals can have a different capture percentage.
- “The true proportion has a 95% chance of changing into the interval.” The population proportion is fixed. The sample and the interval vary across repetitions.
- Forgetting the method’s conditions. The long-run claim depends on an appropriate sampling process and interval method. A calculator output alone does not establish that the method is suitable.
Key Takeaway
A 95% confidence level describes the long-run capture rate of a method applied repeatedly under appropriate conditions. The interval from your observed sample is one fixed result of that method. The true population proportion is fixed too, so that interval either contains it or does not.
Check Your Understanding
For each question, focus on the difference between the long-run method and one interval from one sample.
- In your own words, what does a 95% confidence level say about intervals produced from repeated random samples?
- A student writes, “There is a 95% probability that the true proportion is between 0.24 and 0.39.” What part of this statement misinterprets the confidence level?
- In a hypothetical simulation, 46 of 50 intervals capture the specified true proportion. Calculate the observed capture rate and explain why it need not equal 95%.
- For one computed 90% confidence interval, does the true population proportion have a 90% probability of being inside it? Explain the roles of the fixed proportion and the changing interval.
- Why is a 95% confidence interval generally wider than a 90% confidence interval when both are calculated from the same sample?