From a Test Decision to a Study Conclusion
A hypothesis test produces a decision according to a specified rule. A useful written conclusion must do more than repeat “reject” or “fail to reject.” It should reconnect that decision to the question the study began with: what evidence did the sample provide about the population claim? A conclusion that omits the population or the measured outcome is hard to interpret; one that claims more than the design supports is misleading.
The conclusion has two parts. First, state the decision at the chosen significance level. Then translate it into the study’s terms: name the population quantity or relationship, describe whether the sample points above, below, or away from the null value, and state the strength of evidence indicated by the p-value. Keep the sample observation distinct from the claim about the population.
The decision rule is straightforward: reject \(H_0\) when the p-value is less than or equal to \(\alpha\); otherwise, fail to reject \(H_0\). The p-value describes how unusual data at least as extreme as those observed would be if \(H_0\) and the test model were correct. It is not the probability that \(H_0\) is true. In a conclusion, this supports wording such as “the data provide evidence against the null claim,” not “the null claim has a probability of…” or “the null is proven false.”
A Reliable Structure for the Conclusion
A concise conclusion can usually be built from four elements. They need not appear as four separate sentences, but each should be clear.
State whether the test rejects or fails to reject \(H_0\) at the specified \(\alpha\). Do not omit the threshold that determines the decision.
Refer to the population mean, proportion, difference, or relationship described by the hypotheses, in the setting’s terms.
Say whether the data provide evidence for the alternative claim and, when relevant, whether the observed sample result was higher, lower, or different in a specified way.
Do not turn evidence of an association into a causal claim, or a result for one sampled population into a universal statement.
This structure separates three things that are easily blurred: the sample’s observed result, the evidence assessed by the test, and the conclusion about a population. For example, a sample average can be above a benchmark even when the test does not reject the null. The observed direction remains a fact about the sample; it does not by itself establish the corresponding population claim.
Worked Conclusions
Worked Example: Evidence About a Population Mean
Hypothetical scenario. A clinic’s records team wants to know whether the population mean wait from check-in to appointment is different from 10 days. A random sample of 64 appointments has a mean wait of 10.8 days. For this example, assume the population standard deviation is known to be 2.4 days, the observations are independent, and the sampling distribution of the mean is normal. The two-sided hypotheses are \(H_0:\mu=10\) days and \(H_a:\mu\ne10\) days. The test will use \(\alpha=0.05\).
Under the null model, the sample mean has mean 10 days and standard error \(2.4/\sqrt{64}\) days. The standardized result is:
For a two-sided test, results at least this far from zero count as extreme in either direction. Thus the p-value is approximately \(2P(Z\ge 2.667)=0.0077\), where \(Z\) has a standard normal distribution. Since \(0.0077<0.05\), reject \(H_0\).
Conclusion in context: The sample provides statistically significant evidence, at the 5% level, that the population mean wait for appointments at this clinic differs from 10 days. The sample mean was higher than 10 days, so the observed result points toward a longer mean wait. Because the test was two-sided, the formal alternative was “different,” not specifically “greater.” This result concerns the population represented by the clinic’s appointment records; it does not establish that every appointment has a longer wait.
Worked Example: A Survey Claim About a Population Proportion
Hypothetical scenario. A district asks whether more than 55% of its adult residents use public transit at least once a week. In a random survey of eligible adults, 78 of 120 respondents answer yes, so the sample proportion is \(78/120=0.65\), or 65%. Suppose the survey’s sampling and response conditions are adequate for the intended population, an appropriate test of \(H_0:p=0.55\) against \(H_a:p>0.55\) has been carried out, and it gives a p-value of 0.014. Use \(\alpha=0.05\).
The decision comparison is \(0.014<0.05\), so reject \(H_0\). The sample proportion is above the null value: \(0.65>0.55\). Both facts belong in the conclusion, but they play different roles. The 65% is the observed sample result; the test evaluates evidence about the population proportion.
Conclusion in context: The survey provides statistically significant evidence at the 5% level that more than 55% of eligible adult residents in this district use public transit at least once a week. The surveyed proportion was 65%. This supports the population claim only to the extent that the sampling and response conditions justify generalizing from these respondents to district adults. It does not mean that more than 55% of adults everywhere use transit, nor does the survey show that transit use causes any other outcome.
Worked Example: When the Test Does Not Reject
Hypothetical scenario. Researchers randomly sample employees from a defined company to examine whether night-shift status is associated with reporting frequent fatigue. They state \(H_0\) as no association in the company’s employee population and \(H_a\) as an association. Assume the sample and analysis meet the requirements of the chosen test. The test gives a p-value of 0.18, and the researchers set \(\alpha=0.05\).
Because \(0.18>0.05\), the test fails to reject \(H_0\). The result does not meet the study’s rule for sufficiently strong evidence against no association. It does not prove that the population has no association, that the groups are identical, or that any association is too small to matter.
Conclusion in context: At the 5% significance level, the study does not provide sufficient evidence of an association between night-shift status and frequent fatigue among employees of this company. The data have not established that no association exists. Since this is an observational study, even a statistically significant association would not, by itself, show that night shifts cause fatigue.
Why the Decision Language Is Justified
The meaning of the decision follows from the test’s rule, not from a declaration that one hypothesis is certainly true. If the p-value is at most \(\alpha\), the observed result falls within the set of outcomes the decision rule designates as sufficient evidence against \(H_0\). That is why “reject \(H_0\)” is translated as evidence against the null claim and in favor of the alternative, conditional on the test assumptions. It is not a guarantee that the alternative is true.
If the p-value exceeds \(\alpha\), the result does not cross that rule’s threshold. The test therefore fails to reject \(H_0\). The reasoning stops there: not crossing a threshold is not proof that the null is correct. A study may have limited information to distinguish the null from other possibilities, and the p-value alone does not tell us which explanation is true.
The significance threshold and the p-value also play distinct roles. The threshold is the decision rule selected for the test; the p-value is a measure of how extreme the data are under the null model. A conclusion should name the decision made at the threshold, rather than treating “statistically significant” as a description of how large or important an effect is. Statistical significance does not automatically imply practical importance.
Keep the Scope of the Claim Proportional to the Study
A test addresses the population quantity or relationship represented by its hypotheses. It does not automatically answer a broader question. Generalizing from a sample depends on how observations were selected and on issues such as nonresponse or missing data. If the sample consists only of volunteers, for example, a small p-value does not make those volunteers representative of everyone in the target population.
Causal conclusions require particular care. A test may find evidence of a difference or association, but the p-value does not say whether one variable caused another. In an observational study, other differences between groups may help explain the pattern. A causal statement needs support from the design and relevant assumptions, not merely a small p-value. Even a randomized study’s result should be stated for the participants, treatment, outcome, and conditions actually studied, unless broader generalization is justified.
Common Mistakes
- Repeating only the decision. “Reject \(H_0\)” does not tell a reader what the study found. Add the population claim and outcome in context.
- Claiming the null is true after failing to reject. The test did not provide sufficient evidence against \(H_0\) at the chosen threshold; it did not establish \(H_0\).
- Turning the sample result into a population fact. A sample proportion of 65% is an observed sample proportion, not a guarantee that exactly 65% of the population has the characteristic.
- Using a two-sided result to claim a one-sided alternative. If \(H_a\) says “different,” describe evidence of a difference. You may report the sample’s direction, but do not rewrite the alternative after seeing the data.
- Claiming causation from an association. A significant result does not, by itself, show that one variable caused another.
- Calling a result important because it is significant. Significance describes evidence under the test and decision rule. Practical importance depends on the size and consequences of the difference or relationship.
Check Your Understanding
For each question, focus on how the decision should be translated into a conclusion in context.
- A test uses \(\alpha=0.05\) and gives a p-value of 0.03. What decision follows, and what should the conclusion say about the population claim?
- A test fails to reject \(H_0\). Why is “there is no difference” stronger than the test result warrants?
- A two-sided test concerns whether a population mean differs from a benchmark, and the sample mean is below that benchmark. How can a conclusion describe the direction without changing the alternative hypothesis?
- A survey finds a statistically significant association between two characteristics. What additional issue must be considered before generalizing to the target population, and why does the p-value not establish causation?
- In a conclusion, how should statistical significance be distinguished from practical importance?