Checking the Success-Failure Condition

Why Check the Successes and Failures?

A one-proportion z interval uses a normal approximation to describe how a sample proportion varies from sample to sample. That approximation is most dependable when the sample contains enough observations in both outcome categories. A large sample size by itself is not always enough: if successes are rare, a sample can be large while still containing very few successes.

The success-failure check is a diagnostic for the normal approximation used by the interval. It does not establish that the sample was randomly selected or that observations are independent; those are separate requirements. Here, the focus is whether the observed counts give reasonable support for approximating the sampling distribution by a normal distribution.

Definition: For a sample of size \(n\) with a binary outcome, let \(x\) be the number of successes and \(n-x\) the number of failures. The observed sample proportion is \(\hat{p}=x/n\). The success-failure condition is commonly checked using the counts \(n\hat{p}=x\) and \(n(1-\hat{p})=n-x\). A widely used guideline for a one-proportion z interval is that both counts should be at least 10.

The threshold of 10 is a practical rule of thumb, not a mathematical boundary at which a normal approximation suddenly becomes valid. Some courses or references use a less stringent threshold, often 5. The convention in this tutorial is the 10-and-10 guideline: report whether each observed count reaches 10, and treat a failure of either count as a warning that the usual z interval may be unreliable.

What the Counts Tell Us

When each observation is a success or a failure, the sample proportion is the number of successes divided by the sample size. If the population proportion is \(p\), then under independent Bernoulli sampling the number of successes has a binomial distribution with parameters \(n\) and \(p\). Its mean is \(np\), and its variance is \(np(1-p)\). The normal approximation is used because, under suitable conditions, the distribution of the sample proportion is approximately bell-shaped.

For an interval, the population proportion \(p\) is unknown. The usual z interval estimates the standard error by substituting \(\hat{p}\) for \(p\):

SE=\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

The success-failure check uses the corresponding estimated counts: \(n\hat{p}\) estimated successes and \(n(1-\hat{p})\) estimated failures. Since \(\hat{p}=x/n\), these are exactly the observed counts \(x\) and \(n-x\). Checking the counts is therefore a direct way to look for a very sparse category before relying on the normal approximation.

Practical rule: Before using a one-proportion z interval, calculate \(x=n\hat{p}\) and \(n-x=n(1-\hat{p})\). Under the 10-and-10 guideline, proceed with the normal approximation only if both counts are at least 10, while also checking the interval’s other assumptions separately.

Worked Checks

Worked Example: Both Counts Are Large Enough

Suppose a hypothetical survey asks 100 randomly selected participants whether they support a proposed change. There are 42 “Yes” responses and 58 “No” responses. For this example, assume the sampling design and independence requirements are satisfied; assess only the success-failure condition.

The sample size is \(n=100\), the success count is \(x=42\), and the failure count is \(100-42=58\). The sample proportion is

\hat{p}=\frac{42}{100}=0.42

The estimated success and failure counts are

n\hat{p}=100(0.42)=42,\qquad n(1-\hat{p})=100(0.58)=58.

Both 42 and 58 are at least 10, so the sample passes the 10-and-10 guideline. The normal approximation for a one-proportion z interval is reasonable on this diagnostic. This check supports using the approximation; it does not guarantee that the interval will contain the population proportion or that other assumptions have been met.

Worked Example: A Rare Success

In a hypothetical sample of 30 devices, 2 fail a quality test and 28 pass. Let “fails the test” be the success category of interest. Assume the separate sampling and independence requirements are satisfied, and check the normal approximation condition.

Here \(n=30\), \(x=2\), and \(n-x=28\). Thus

\hat{p}=\frac{2}{30}=0.0667,\qquad n\hat{p}=30(0.0667)\approx 2,\qquad n(1-\hat{p})\approx 28.

The failure count is well above 10, but the success count is only 2. Because both counts must meet the guideline and 2 is less than 10, this sample fails the success-failure check. A one-proportion z interval may be unreliable even though \(n=30\) might initially sound substantial.

The estimated standard error is

SE=\sqrt{\frac{(2/30)(28/30)}{30}}\approx 0.0455.

The calculation is possible, but that does not make the normal approximation appropriate. With so few observed successes, the approximate bell-shaped sampling distribution is questionable. The count check warns against interpreting a z interval as dependable just because its formula produces a number.

Worked Example: A Balanced but Small Sample

Suppose a hypothetical group of 18 participants has 9 successes and 9 failures. Check the counts under the 10-and-10 guideline.

The sample proportion is \(\hat{p}=9/18=0.50\). The estimated counts are

n\hat{p}=18(0.50)=9,\qquad n(1-\hat{p})=18(0.50)=9.

Neither count reaches 10, so this sample does not satisfy the chosen guideline. The counts are balanced, but balance alone is not enough: the guideline asks whether there are enough observations in each category, not whether the categories have similar sizes.

A reference using a threshold of 5 would reach a different conclusion for these counts. That difference reflects the fact that the threshold is a rule of thumb rather than a theorem. When applying a procedure, use the stated convention consistently and make the actual counts visible so readers can judge the diagnostic.

Worked Example: Large Sample, Still Too Few Successes

A hypothetical screening program records 8 positive results among 1,000 people. Take a positive result as success. Check whether the sample size alone makes the normal approximation suitable.

The success and failure counts are

x=8,\qquad n-x=1000-8=992.

The sample proportion is \(\hat{p}=8/1000=0.008\), so the estimated counts are \(1000(0.008)=8\) successes and \(1000(0.992)=992\) failures. Although \(n=1{,}000\), the success count is less than 10. The sample therefore fails the 10-and-10 check. A large \(n\) cannot compensate automatically for an outcome category represented by only a few observations.

Why the Guideline Is Sensible

The connection between the counts and the shape of the sampling distribution can be seen from the binomial model. Let each observation be 1 for a success and 0 for a failure, with success probability \(p\). The sum of \(n\) such independent outcomes is the success count. Its mean is \(np\), and its variance is \(np(1-p)\). When one expected category count is very small, the possible values of the count are concentrated near a boundary and its distribution can be strongly asymmetric.

A brief skewness calculation makes this connection more precise. For a single binary outcome \(Y\), the third central moment is

$$ E[(Y-p)^3] =p(1-p)^3+(1-p)(-p)^3 =p(1-p)(1-2p). $$

For a sum of \(n\) independent such outcomes, the third central moment is \(np(1-p)(1-2p)\), while the variance is \(np(1-p)\). Dividing the third central moment by the variance raised to the power \(3/2\) gives the standardized skewness:

\frac{np(1-p)(1-2p)}{[np(1-p)]^{3/2}} =\frac{1-2p}{\sqrt{np(1-p)}}.

The denominator involves the expected success and failure counts, \(np\) and \(n(1-p)\). When either is small, the standardized skewness need not be small, so a symmetric normal approximation may be poor. In an interval problem \(p\) is unknown, which is why the observed counts based on \(\hat{p}\) serve as a practical diagnostic. This reasoning explains the guideline but does not derive 10 as a universal cutoff.

How to Use a Failed Check

If either observed count is below 10, report which count is small and avoid presenting the ordinary one-proportion z interval as automatically reliable. Depending on the setting, an interval method designed for sparse or boundary-near data may be more suitable, such as an exact binomial interval or a Wilson score interval. Choosing among alternative methods is a separate decision; the essential point here is to recognize when the normal approximation deserves caution.

Passing the check is also not a complete approval of the analysis. It supports only the normal-approximation component of the one-proportion z interval. The sample must still meet the procedure’s other conditions, including appropriate random sampling and independence. Those checks address how the data were obtained, rather than whether the observed outcome counts are large enough.

Common Mistakes

Checking only the total sample size. “There are 1,000 observations, so the approximation must be fine” ignores a possible count such as 8 successes. Check successes and failures separately.
Checking only one category. A count of 40 successes does not satisfy the guideline if there are only 3 failures. Both counts must meet the threshold.
Treating 10 as a guarantee. Passing the check does not prove that a z interval is accurate. It is evidence that one part of the approximation is reasonable, not proof that all requirements hold.
Treating 10 as a sharp mathematical boundary. A count of 9 does not imply that the approximation is always unusable, nor does a count of 10 ensure perfection. The cutoff is a convention for screening cases that warrant caution.
Confusing observed counts with the unknown population counts. The calculation uses the sample’s observed successes and failures, not a claim that the population contains those same counts.

Key takeaway: For a one-proportion z interval, calculate \(x=n\hat{p}\) and \(n-x=n(1-\hat{p})\). Under the common 10-and-10 guideline, both must be at least 10. Treat the result as a normal-approximation diagnostic, not a guarantee or a substitute for checking the procedure’s other conditions.

Check Your Understanding

For each situation, identify the success and failure counts and decide what the 10-and-10 guideline says.

A hypothetical poll has 64 “Yes” responses and 36 “No” responses. Does it pass the guideline?
A hypothetical sample of 75 items has 6 defective items. Which count fails the guideline, and what does that imply for a one-proportion z interval?
Explain why a sample with 9 successes and 9 failures can fail the 10-and-10 guideline despite being balanced.
A researcher says, “The sample size is 2,000, so checking the category counts is unnecessary.” What is wrong with this reasoning?
Does meeting the success-failure guideline by itself establish that a one-proportion z interval is appropriate? Explain what the check does and does not address.