Three Ways a Probability Evaluation Can Go Wrong
A probability calculation can be correct and still lead to a poor conclusion. That happens when someone uses a model without checking its conditions, stretches a result beyond the situation it represents, or misstates what the calculated probability means. These errors often occur after the arithmetic is finished, when the result is being interpreted.
In Evaluating Claims Based on Probability, you learned to compare an observed result with what a claim’s model would predict. That comparison depends on the model being reasonable for the chance process. As discussed in Model Validity Versus Calculation Accuracy, correct arithmetic answers a question about the model used; it does not by itself show that the model fits reality.
A useful evaluation separates three questions. First, what conditions does the model require, and are there reasons to believe they hold? Second, what population, process, and time period does the model describe? Third, what exact event does the probability refer to? Keeping these questions separate helps prevent a correct number from being used to support an unjustified claim.
Error 1: Using a Model While Ignoring Its Conditions
Many models rely on assumptions about how outcomes are produced. For a binomial model, for example, there must be a fixed number of trials, two outcomes per trial, a constant probability of success, and independent trials. Earlier tutorials, including Assumptions Behind a Probability Model and Independence Assumptions in Real Settings, explain what these assumptions mean. The common error is not forgetting their names; it is calculating as if they hold when the setting gives reason to doubt them.
A calculation can still be useful as a description of what a proposed model implies. But if an important condition is questionable, the result should be described as conditional on that model, not confidently presented as the chance in the real process. Look for shared influences: outcomes produced in the same tray, on the same machine, or during the same weather conditions may be related rather than independent.
Worked Example: Seeds Sharing One Growing Tray
A garden club proposes that each seed in a particular batch has a 0.70 probability of germinating. A student uses a binomial model to calculate the chance that all 10 seeds germinate. The seeds are planted together in one tray. Evaluate the calculation and its interpretation.
State. Let \(X\) be the number of seeds, out of 10, that germinate. The proposed model is \(X\sim\operatorname{Binom}(10,0.70)\). The event “all 10 germinate” is \(X=10\).
Plan. Check the binomial conditions before treating the calculated probability as a chance for this tray. There are a fixed 10 seeds, and each seed has two outcomes: it germinates or it does not. The model assigns the same probability, 0.70, to each seed. However, independence is uncertain: all 10 seeds share a tray, so excess moisture, poor drainage, or another tray-wide condition could affect several seeds together. The calculation can show what the proposed binomial model predicts, but the shared environment gives a reason to question whether that prediction accurately represents the real process.
Do. Under the proposed model, all 10 seeds germinate only when every seed succeeds:
As a check, \((0.70)^5=0.16807\), and squaring gives \((0.16807)^2=0.0282475249\), the same result. The model therefore assigns about a 2.82% chance to all 10 germinating.
Conclude. If each seed truly has a 0.70 germination probability and the seeds’ outcomes are independent, the chance that all 10 germinate is about 0.0282. Because a shared tray may make the outcomes dependent, this is not automatically a reliable probability for the actual tray. A careful evaluation identifies that limitation instead of treating the calculator output as proof that the real chance is 2.82%.
Error 2: Extending a Result Beyond Its Scope
A model or a set of observations refers to a particular population and process. Extending its conclusion to a different group, time period, or setting requires a reason to believe the relevant conditions remain similar. A result from one busy week, for example, may not represent a typical week. A result from one location may not represent every location.
This is not a warning against making any broader claim. It is a reminder to match the strength and scope of the conclusion to the evidence. Ask whether the data came from a random, representative selection and whether the chance process is stable over the target period. Even a well-calculated probability under a proposed model does not guarantee that its inputs apply to future cases.
Worked Example: Generalizing from One Week of Support Tickets
A fictional software team reviews 60 support tickets from a particularly busy product-launch week and finds that 18 were answered late. A team member says, “The chance any ticket will be late next month is 30%, so in the next four tickets the chance of at least one late answer is about 76%.” Assess the reasoning.
State. The observed late-ticket proportion in the reviewed group is \(18/60\). For the team member’s proposed future model, let \(X\) be the number of late answers among the next four tickets. The proposed model treats each ticket as late with probability \(p=0.30\).
Plan. First check the arithmetic and then check whether the data justify the future model. The 60 reviewed tickets came from a busy launch week, not a stated random sample of next month’s tickets. The mix of ticket difficulty, staffing, and workload could differ. For the binomial calculation, four is a fixed number of trials and each ticket has two outcomes, late or not late. But a constant late-answer probability of 0.30 for next month and independence between ticket outcomes are assumptions, not established facts. So the calculation can be evaluated conditionally, while the broad claim about next month remains unsupported by this information alone.
Do. The observed proportion is:
The arithmetic is also checked by \(60(0.30)=18\). If, hypothetically, the next four tickets each had a 0.30 probability of being late and their outcomes were independent, then the probability of at least one late answer would be the complement of no late answers:
As a check, \((0.70)^4=(0.49)^2=0.2401\), so the complement is \(0.7599\), or about 76.0%. This verifies the calculation under the proposed model; it does not verify that 0.30 is an appropriate probability for next month.
Conclude. If the next four tickets each have a constant 0.30 probability of being late and the outcomes are independent, the chance of at least one late answer is about 0.7599. The reviewed launch-week tickets do not, on their own, establish those conditions or show that next month’s tickets are comparable. The team member should present 76.0% as a conditional model result, not as a confirmed prediction for next month.
The distinction is between using data to propose a model and having enough evidence to apply that model elsewhere. As covered in Limitations of Probability Models, the quality and representativeness of the inputs matter. A larger calculation does not repair a mismatch between the data and the population being discussed.
Error 3: Misreading What a Probability Represents
A probability belongs to a clearly specified event and a stated model. It describes the model’s chance for that event, not a promise about what will happen. A probability of 0.85 for one weekday does not mean that exactly 85% of any small group of days must have no delays. And a probability calculated using a model is not the probability that the model itself is correct.
When interpreting a result, name the random variable or event, the relevant number of trials or individual, and the assumptions behind the model. Be especially careful with wording such as “the probability the claim is true,” “the chance it happened by chance,” or “there is a 76% chance the next four will include a late answer.” These phrases may confuse a model-based event probability with certainty about the model or an individual outcome.
Worked Example: Interpreting a Weekday Delay Model
A transit planner proposes that a particular bus route has a 0.15 probability of a delay on each weekday. For a five-weekday period, a student calculates the chance of no delays and says, “There is a 44% chance the model is correct.” Identify the error and give a correct interpretation.
State. Let \(X\) be the number of weekdays, out of five, on which the route is delayed. Under the planner’s proposed binomial model, \(X\sim\operatorname{Binom}(5,0.15)\). The event of no delays is \(X=0\).
Plan. The model uses a fixed five weekdays, two outcomes per weekday, and a stated 0.15 delay probability. It also assumes the delay probability is constant across those weekdays and that delay outcomes are independent. Those assumptions should be considered: for instance, the same storm could affect several days. For now, calculate what the proposed model says, then distinguish the probability of the event from the probability that the model is correct.
Do. No delays means all five weekdays are not delayed, each with model probability \(1-0.15=0.85\):
A direct check is \(0.85^2=0.7225\), \(0.85^4=0.52200625\), and \(0.85^5=0.52200625(0.85)=0.4437053125\). The model therefore assigns about a 44.37% chance to no delays in the five weekdays. The complementary event, at least one delay, has probability \(1-0.4437\approx0.5563\), using the unrounded value before rounding.
Conclude. If the proposed delay model and its assumptions are appropriate, the chance of no delays on the five weekdays is about 0.4437. This is not a 44% chance that the model is correct. It is the model-based probability of a specific five-day event. Whether the model is appropriate requires separate evidence about the route’s delay process and the assumptions of constant probability and independence.
A Practical Scope Check
Before accepting an interpretation, use a scope check. It does not replace checking a model’s particular conditions; it helps keep the conclusion attached to the question the calculation actually answers.
State exactly what outcome the probability describes, such as all 10 seeds germinating or at least one of four tickets being late.
Say which probabilities and conditions the calculation relies on. If a condition is uncertain, make the conclusion conditional and explain why.
Identify the individuals, process, place, and time period represented. Do not quietly extend the result to a different group or future period.
Describe it as a chance under the stated model. Do not call it the chance that the model is true or a guarantee about what will happen.
This check also helps distinguish three different statements: what the model predicts, how well the model fits the evidence, and what can be concluded about a broader population or future process. They are related questions, but one answer does not automatically settle the others.
Common Mistakes and AP Exam Communication
A strong AP response makes the reasoning visible. It does not have to reject a model whenever an assumption is imperfectly known, but it should not ignore an assumption that could materially affect the conclusion.
- Writing “the model is valid because the calculation works.” Correct arithmetic establishes only that the model’s implication was calculated correctly. Full-credit communication separately addresses whether the model conditions fit the setting.
- Claiming independence without considering shared influences. Outcomes from one tray, one production run, or a short period of unusual weather may be connected. Name a plausible source of dependence and explain how it affects the model’s appropriateness.
- Turning a sample proportion into a guaranteed future probability. An observed proportion can help suggest a value for a model, but it does not automatically establish that the same probability applies to a different group or time period. Discuss whether the sample represents the target situation.
- Confusing a conditional probability with a probability that the model is true. Say, “If the model assumptions hold, the chance of this event is…” rather than assigning a probability to the truth of the model.
- Interpreting a probability as a required percentage in a small group. A 0.70 chance for each trial does not require 70% of a small set of trials to succeed. Random outcomes vary.
- Leaving the event vague. “The probability is 0.44” does not identify what 0.44 measures. State the event, the number of trials or unit, and the model conditions in the conclusion.
Key Takeaway
When evaluating a probability model, keep the calculation, the model assumptions, and the conclusion’s scope separate. A correct probability can still be misused if the conditions are doubtful, the result is extended to a different setting, or the number is described as the chance that the model itself is true.
Check Your Understanding
For each situation, identify the error or explain what additional qualification a careful interpretation needs.
- A model says each of 8 seedlings independently has probability 0.60 of sprouting. All 8 are planted in the same container. What condition might be questionable, and what shared influence could matter?
- A random sample from one month shows that 12% of orders arrived late. What would you want to know before applying that percentage to orders during the next holiday season?
- A model assigns probability 0.20 to a particular event. Explain why this does not mean the model has a 20% chance of being correct.
- A student calculates the probability of at least one late answer among four tickets under a model with \(p=0.30\). What assumptions should be stated before interpreting the result as a prediction?
- Rewrite this conclusion more carefully: “The probability is 0.44, so there is a 44% chance the bus-delay model is true.”