Why a Correct Decision Can Still Lead to a Weak Conclusion
A mean test can be calculated correctly and still be communicated poorly. “Reject \(H_0\)” reports a decision, but it does not tell a reader what the data say about the original question. A sentence such as “the sample mean is lower” describes a statistic, not necessarily the population mean the test concerns. And even a small p-value does not prove a claim with certainty.
In “Tying the Conclusion to the Original Claim,” you practiced connecting a test decision to the parameter and direction in the research question. This tutorial focuses on errors that can weaken that final communication: leaving out context, not explaining the p-value comparison, and writing as if a statistical test establishes certainty.
A useful conclusion typically names the population and quantitative variable, identifies the direction of the claim when the alternative is directional, and connects the decision to the p-value and \(\alpha\). You do not need to copy every part of the test into the final sentence. But your conclusion should make clear why you rejected or failed to reject \(H_0\), and what that decision means in the situation.
The earlier tutorials “Making a Decision From P-Value and Alpha,” “Reject \(H_0\) Wording That Earns Full Credit,” and “Fail to Reject \(H_0\) Wording That Earns Full Credit” established the decision rules. Use them here: when \(p\le\alpha\), reject \(H_0\); when \(p>\alpha\), fail to reject \(H_0\). The decision is not a statement about whether the null hypothesis is true.
Three Checks Before You Write
Does the conclusion name the population mean and the measured variable, rather than only describing the sample mean? Include units or other context where they help make the claim clear.
Have you connected the p-value to the stated significance level? State the comparison, such as \(0.02665<0.05\), and give the matching decision.
Use “provide convincing evidence” when rejecting \(H_0\), and “do not provide convincing evidence” for the alternative when failing to reject. Do not say “prove,” “accept \(H_0\),” or claim that the population mean is certainly a particular value.
These checks are separate. A conclusion can name the right population but leave out the evidence comparison. It can include a correct p-value comparison but remain too vague about the claim. It can also get both of those right and then overstate the result by saying the data prove the alternative. Revise until all three checks are satisfied.
Worked Example: The Sample Mean Is Higher, but Is the Evidence Convincing?
Worked Example: The Sample Mean Is Higher, but Is the Evidence Convincing?
A fictional parks department randomly selects 25 trail counters from a production lot of at least 250. It measures each counter’s operating time, in days, before a battery change. The sample mean is 51 days and the sample standard deviation is 10 days. The data show no severe skewness or extreme outliers. The department wants to know whether the population mean operating time exceeds 50 days. Use \(\alpha=0.05\).
Let \(\mu\) be the true mean operating time, in days, for trail counters in this production lot. The hypotheses are \(H_0:\mu=50\) days and \(H_a:\mu>50\) days.
Use a one-sample t test for a population mean. The counters were randomly selected. The 10% condition is met because the sample is no more than 10% of the lot: \(25\le0.10(250)=25\). The sample data show no severe skewness or extreme outliers, supporting use of a t procedure.
Calculate the estimated standard error and test statistic:
The data do not provide convincing evidence that the true mean operating time of trail counters in this production lot exceeds 50 days.
A weak draft might say, “The sample mean is 51 days, so the counters last longer than 50 days.” This reports the sample’s direction but skips the p-value comparison. The sample mean is above 50 days, but \(0.3108>0.05\), so the test does not provide convincing evidence for the population claim. “The data do not provide convincing evidence” is not the same as “the mean is 50 days”; failing to reject does not establish the null hypothesis.
Worked Example: A Decision Without Context Is Incomplete
Worked Example: A Decision Without Context Is Incomplete
A fictional school randomly selects 36 students from a population of at least 360 students who use a particular study room. Each student reports the time, in minutes, spent setting up materials before studying. The sample mean is 18 minutes and the sample standard deviation is 3 minutes. The data show no severe skewness or extreme outliers. The school wants to know whether the population mean setup time is less than 19 minutes. Use \(\alpha=0.05\).
Let \(\mu\) be the true mean setup time, in minutes, for students who use this study room. The hypotheses are \(H_0:\mu=19\) minutes and \(H_a:\mu<19\) minutes.
Use a one-sample t test for a population mean. The students were randomly selected. The 10% condition is met because \(36\le0.10(360)=36\). The data show no severe skewness or extreme outliers, so the Nearly Normal condition is reasonable for this sample.
The standard error is \(3/\sqrt{36}=0.5\) minute. Thus,
The data provide convincing evidence that the true mean setup time for students who use this study room is less than 19 minutes.
Suppose a draft says only, “Reject \(H_0\).” The decision is correct, but the reader cannot tell which population or claim it concerns. A stronger version adds the context and direction: “Since \(0.02665<0.05\), we reject \(H_0\). The data provide convincing evidence that the true mean setup time for students who use this study room is less than 19 minutes.” The p-value comparison explains the decision; the second sentence answers the research question.
Worked Example: A Small P-Value Does Not Prove a Claim
Worked Example: A Small P-Value Does Not Prove a Claim
A fictional community program randomly selects 16 participants from a group of at least 160 people who completed a particular training session. It measures the time, in minutes, each participant takes to complete a routine task. The sample mean is 6.5 minutes and the sample standard deviation is 2 minutes. The data show no severe skewness or extreme outliers. The program asks whether the population mean task time is less than 7.5 minutes. Use \(\alpha=0.05\).
Let \(\mu\) be the true mean task time, in minutes, for people who completed this training session. The hypotheses are \(H_0:\mu=7.5\) minutes and \(H_a:\mu<7.5\) minutes. A one-sample t test is appropriate. The data come from a random sample, and the 10% condition is met because \(16\le0.10(160)=16\). The data show no severe skewness or extreme outliers, supporting the t procedure.
The standard error is \(2/\sqrt{16}=0.5\) minute, so the test statistic is
For a left-tailed test with \(t=-2.000\) and \(df=15\), the p-value is approximately \(0.0320\), rounded to four decimal places. Since \(0.0320<0.05\), reject \(H_0\). The data provide convincing evidence that the true mean task time for people who completed this training session is less than 7.5 minutes.
A draft saying “The test proves that the true mean is less than 7.5 minutes” overstates the result. The p-value describes how unusual a test statistic at least as extreme as the observed one would be if \(H_0\) were true; it does not make the alternative certain. The appropriate conclusion is about convincing evidence, not proof. Nor does this test show that every participant takes less than 7.5 minutes: it concerns the population mean, not every individual task time.
Common Mistakes and AP Exam Tips
- Giving only the decision: “Reject \(H_0\)” or “fail to reject \(H_0\)” is not a contextual conclusion. Add what the data provide evidence about, naming the population mean and the claim’s direction.
- Omitting the p-value comparison: A conclusion should show how the evidence decision follows from the stated \(\alpha\). For example, “Since \(p=0.02665<0.05\), reject \(H_0\)” makes the comparison explicit.
- Reporting only the sample result: “The sample mean is 18 minutes” does not answer a question about the population mean. Make clear whether the inference concerns a population, and identify which one.
- Claiming the null is true after failing to reject: “Accept \(H_0\)” and “the population mean equals the null value” are not justified by a large p-value. Say that the data do not provide convincing evidence for the alternative.
- Claiming proof after rejecting: A statistically significant result is convincing evidence against \(H_0\) and for the alternative, not certainty. Avoid “proves,” “definitely,” and similar language.
- Generalizing from a mean to every individual: A test about \(\mu\) concerns an average in a population. Do not claim that every person, product, or measurement has the result described by the mean.
- Forgetting the study’s scope: Name the population represented by the sampling method. Do not extend the conclusion to people or settings not represented by the study.
For full credit, a conclusion should agree with the hypotheses, the p-value comparison, and the study context. A concise, reliable structure is: “Since \(p\) [comparison] \(\alpha\), [reject/fail to reject] \(H_0\). The data [provide/do not provide] convincing evidence that [contextual alternative claim about the population mean].” Use “do not provide convincing evidence” when you fail to reject, not “prove there is no effect.”
Check Your Understanding
For each situation, identify the conclusion error or write a corrected conclusion.
- A right-tailed test of whether a population mean exceeds 40 gives \(p=0.18\) at \(\alpha=0.05\). Write a conclusion that includes the decision and the contextual claim.
- A test gives \(p=0.02\) at \(\alpha=0.05\). Why is “the test proves the alternative hypothesis” too strong?
- A student writes only, “Reject \(H_0\)” after a test of whether the mean delivery time is less than 30 minutes. What important information is missing?
- A test fails to reject \(H_0:\mu=12\). Explain why it is not correct to conclude that the population mean is exactly 12.
- Why does evidence that a population mean is below a target not imply that every individual measurement is below that target?