Why the Consequences Matter
In “Two Possible Errors in a Significance Test,” you learned that a Type I error is rejecting a true null hypothesis, while a Type II error is failing to reject a false null hypothesis. In “Significance Level as the Probability of a Type I Error,” you saw that \(\alpha\) describes the probability of a Type I error when the null model is true. This tutorial asks a different question: if an error occurs, what harm could it cause?
The answer depends on what the hypotheses mean in the situation and who is affected by the decision. A false claim that a drug works may expose patients to a treatment that provides no benefit. But failing to recognize that an effective drug works may also harm patients by delaying access to a useful treatment. The same two error definitions apply in both cases; the consequences do not have to be equal.
A useful first step is to connect each test decision to the real-world action. “Reject \(H_0\)” might mean approve a drug or flag an email. “Fail to reject \(H_0\)” might mean do not approve the drug or let the email reach an inbox. Then ask what happens if that decision is wrong.
Translate the Errors Into Consequences
The error labels alone do not tell you what happens in a particular setting. First state the null and alternative hypotheses in words. Then match each incorrect decision with the actual condition that would make it an error.
| Actual situation | Test decision | Result |
|---|---|---|
| \(H_0\) is true | Reject \(H_0\) | Type I error |
| \(H_0\) is true | Fail to reject \(H_0\) | Correct decision |
| \(H_0\) is false | Reject \(H_0\) | Correct decision |
| \(H_0\) is false | Fail to reject \(H_0\) | Type II error |
For example, suppose \(H_0\) says that a treatment does not provide a real benefit and \(H_a\) says that it does. Rejecting a true \(H_0\) amounts to concluding that the treatment benefits patients when it does not: a Type I error. Failing to reject a false \(H_0\) means not finding convincing evidence of benefit when the treatment really is beneficial: a Type II error.
The conclusion “Type I is more serious” is not a general statistical rule. It is an argument about a setting. A careful argument identifies the people affected and the kind of harm at stake. It may also recognize that different groups can reasonably weigh those harms differently.
Worked Examples: Comparing Consequences in Context
Worked Example: Deciding Whether to Approve a Drug
A review panel is considering a new drug for a serious condition. Let \(H_0\) mean that the drug does not provide a meaningful benefit for patients, and let \(H_a\) mean that it does. The panel must decide whether the evidence is strong enough to approve the drug for use. In a hypothetical comparison, if an ineffective drug is approved, 300 patients could face serious side effects without receiving the intended benefit. If an effective drug is not approved, 80 patients could miss a treatment that would meaningfully help them. Which error seems more serious under these assumptions?
Translate the decisions: Approving the drug corresponds to rejecting \(H_0\). If \(H_0\) is true, that approval decision is a Type I error: the panel concludes that the drug provides meaningful benefit when it does not. Not approving the drug corresponds to failing to reject \(H_0\). If \(H_0\) is false, that decision is a Type II error: the panel does not establish benefit even though the drug really is beneficial.
Compare the stated consequences: Under the scenario’s assumptions, a Type I error could expose 300 patients to serious side effects without the expected benefit. A Type II error could leave 80 patients without a beneficial treatment. If the comparison gives greatest weight to the number of patients facing a serious avoidable health consequence, the Type I error appears more serious here.
Qualify the argument: The count alone does not settle every ethical or medical comparison. The severity and likelihood of side effects, the importance of the treatment benefit, other available treatments, and the condition’s urgency could change the judgment. The figures are invented for this example, not claims about an actual drug. The defensible conclusion is conditional: given these assumptions and this measure of harm, Type I appears more serious.
Worked Example: A Personal Spam Filter
A spam filter treats \(H_0\) as “this email is legitimate” and flags an email when the evidence leads it to reject \(H_0\). A Type I error is therefore a legitimate email flagged as spam; a Type II error is spam that the filter fails to flag. Suppose a hypothetical day has 98,000 legitimate emails and 2,000 spam emails. The filter flags \(0.2\%\) of legitimate emails by mistake and misses \(5\%\) of spam. Estimate the number of each type of error, then discuss which might matter more to a person expecting an important message.
Calculate false flags: A Type I error happens to a legitimate email. The estimated count is
So about 196 legitimate emails would be flagged that day under the stated rate. As a check, \(0.2\%=2\) out of every 1,000, and \(98{,}000\div1{,}000=98\); \(98(2)=196\).
Calculate missed spam: A Type II error happens when spam is not flagged. The estimated count is
So about 100 spam emails would reach inboxes. As a check, \(5\%=1\) out of every 20, and \(2{,}000\div20=100\).
Compare consequences, not just counts: The filter makes more Type I errors than Type II errors in this scenario: about 196 versus 100. But if one of the falsely flagged legitimate emails is an urgent medical referral or a time-sensitive job offer, that Type I error could matter more to the recipient than an ordinary spam message getting through. On the other hand, if the missed spam contains a convincing harmful link, a Type II error could cause greater damage. The rates and counts describe how often errors occur under the stated assumptions; the specific consequences determine how serious they are.
Worked Example: Screening for a Contagious Infection
A clinic uses a screening test to decide which patients should be isolated while waiting for follow-up testing. Let \(H_0\) mean that a patient does not have the infection and \(H_a\) mean that the patient does. A positive screening result leads the clinic to reject \(H_0\) and isolate the patient. In a hypothetical group of 1,000 patients, suppose 20 actually have the infection. The screening decision misses 3 of those infected patients and incorrectly flags 30 patients who are not infected. Which error could be more serious during an outbreak?
Identify the errors: The 30 uninfected patients who are incorrectly flagged represent Type I errors: the clinic acts as if they have the infection even though \(H_0\) is true. The 3 infected patients who are missed represent Type II errors: the clinic fails to identify the infection even though \(H_0\) is false.
Relate errors to possible harm: During an outbreak, a missed infection could allow an infected patient to expose others, so the Type II error may have consequences beyond that patient. Incorrectly isolating an uninfected patient can also cause harm, such as unnecessary separation from family or delayed care for another condition. The number of errors is not the only consideration: the potential for transmission and the costs of isolation matter too.
Make a context-based judgment: If the infection is highly contagious and a missed case could expose many vulnerable people, it is reasonable to argue that Type II errors are more serious in this setting. If isolation is exceptionally harmful or resources are so limited that unnecessary isolation prevents care for patients who need it, the Type I errors may deserve greater concern. The given counts help describe this hypothetical situation, but the argument depends on the consequences attached to each error.
Probability and Consequence Are Different Questions
The probability of an error and the harm caused by an error are separate ideas. Alpha concerns the probability of a Type I error under the null model. For a specified alternative, \(\beta\) is the probability of a Type II error when that alternative is true, and the power of the test is \(1-\beta\). Neither \(\alpha\) nor \(\beta\), by itself, says how harmful an error would be.
It is also important not to confuse a higher error rate with a more serious error. In the spam example, the filter made more Type I errors than Type II errors because the legitimate-email group was much larger. That fact did not prove that a Type I error was more harmful to a person. The consequences depend on what the incorrectly classified message is and what the recipient loses or risks.
A comparison can change when the goal or perspective changes. A patient might focus on avoiding exposure to an ineffective drug; a clinician might emphasize the cost of withholding an effective treatment. A person might be most concerned about missing a crucial email, while a company might focus on the security risks of spam reaching employees. A strong response makes its perspective and assumptions clear instead of presenting a value judgment as a mathematical fact.
How to Build a Defensible Comparison
When asked which error is more serious, do not stop at “Type I” or “Type II.” Connect the error definitions to the real decision, explain the consequences, and give a reasoned conclusion. You can use the following sequence:
Identify the null condition and the alternative in the setting.
Describe the false conclusion or missed finding associated with a Type I error and a Type II error.
Explain who could be affected and what could happen after each incorrect decision.
Say which error seems more serious under the stated assumptions, and identify what could change that comparison.
For example: “A Type II error would mean failing to identify an infected patient who could transmit the illness. During a severe outbreak, that could expose other vulnerable people, so I would consider Type II errors more serious, assuming transmission risk outweighs the harms of temporary unnecessary isolation.” The statement defines the error, names the consequence, gives a conclusion, and states the assumption behind it.
Common Mistakes and AP Exam Tip
- Claiming one error type is always worse: There is no universal ranking. State why an error is more serious in the particular context.
- Only repeating the definition: “A Type I error is rejecting a true null” does not explain its consequences. Translate it into the decision and outcome, such as approving a drug that provides no benefit.
- Using error counts as a complete comparison: A larger number of errors does not necessarily mean greater harm. Consider the seriousness of each event and who bears its costs.
- Confusing probability with harm: Alpha and beta describe error probabilities under specified conditions; they are not measures of how painful, costly, or dangerous the errors are.
- Assuming that failing to reject proves the null: A Type II error is possible precisely because failing to reject \(H_0\) does not establish that \(H_0\) is true.
- Making an unsupported judgment: Full-credit reasoning should state the possible consequence and connect it to the conclusion. If the answer depends on an assumption—such as an outbreak being highly contagious—say so.
Check Your Understanding
For each question, identify the error in context and explain how its consequences affect your comparison.
- A test evaluates whether a new medicine reduces symptoms. Explain what a Type I error and a Type II error would mean if the medicine is approved only after rejecting \(H_0\).
- A spam filter flags a legitimate message as spam. Which type of error is this if the null hypothesis is that the message is legitimate? Explain.
- In a screening setting, why might a missed case be more serious during an outbreak than when the illness cannot spread between people?
- A student says, “There were twice as many false positives, so Type I errors are twice as harmful.” Explain why the conclusion does not follow from the count alone.
- Write a two- or three-sentence argument for which error is more serious when a missed email could be a time-sensitive medical appointment reminder. State an assumption your argument depends on.