Testing a Claimed Percentage

What Does It Mean to Test a Claimed Percentage?

A public statement such as “55% of residents support the proposal” makes a claim about a population proportion, not necessarily about the percentage in any one sample. A survey sample may produce a percentage above or below 55% simply because it includes only some members of the population. A hypothesis test assesses whether the observed difference from the claimed percentage is large relative to the sampling variability expected if the claim were correct.

When the question is whether the true percentage differs from the stated claim in either direction, the appropriate alternative is two-sided. The sample could suggest a population percentage higher than the claim or lower than it; both departures count as evidence against the equality in the claim.

Definition: In a one-proportion z test, $p$ is the population proportion of individuals with the outcome of interest, $p_0$ is the proportion specified by the claim, $n$ is the sample size, and $\hat{p}=x/n$ is the sample proportion when $x$ of the $n$ sampled individuals have that outcome. For a two-sided test of a claimed percentage, $$ H_0:p=p_0 \qquad\text{and}\qquad H_a:p\ne p_0. $$

The null hypothesis represents the claimed value for purposes of calculating how unusual the sample result would be. The alternative says that the population proportion is not equal to that value. It does not specify a direction, so the test must count results at least as far from $p_0$ as the observed result on either side.

Model, Conditions, and Test Statistic

The one-proportion z test uses a binary outcome: each sampled individual either has the outcome being counted (“success”) or does not. Under the null hypothesis, if observations are independent and have the same probability $p_0$ of success, the count $X$ of successes has a binomial model with parameters $n$ and $p_0$. The usual z procedure approximates the sampling distribution of $\hat{p}$ by a normal distribution.

Before calculating the test, check that the data and approximation support this model. The sample should come from a suitable random process, such as a random sample or randomized assignment, and the observations should be independent. For sampling without replacement from a finite population, a common check for approximate independence is that the sample is no more than 10% of that population. The outcome must be recorded consistently as success or failure. Finally, under the null hypothesis, both expected counts should be sufficiently large; the usual introductory check is at least 10 expected successes and 10 expected failures.

The expected counts for this check are calculated using $p_0$, not $\hat{p}$. If these conditions are reasonable, the null standard error and test statistic are

SE_0=\sqrt{\frac{p_0(1-p_0)}{n}}, \qquad z=\frac{\hat{p}-p_0}{SE_0}.

The subscript in $SE_0$ emphasizes that this is the standard error under the null value. For the two-sided test, the p-value is the probability, assuming $H_0$ is true and the model applies, of a z statistic at least as far from zero as the observed statistic:

\text{p-value}=2P(Z\ge |z_{\text{obs}}|) =2\left[1-\Phi(|z_{\text{obs}}|)\right], \qquad Z\sim N(0,1).

Here $\Phi$ is the cumulative distribution function of the standard normal distribution. The absolute value makes the calculation depend on the distance from zero, and the factor of 2 accounts for equally extreme results in both tails. The p-value is not the probability that the null hypothesis is true.

Why the Null Standard Error Is Used

The test asks what sample proportions would be expected if the claimed value were correct. That question determines which standard error belongs in the test statistic. Under $H_0:p=p_0$, the binomial count has expected value $np_0$ and variance $np_0(1-p_0)$. Since $\hat{p}=X/n$, dividing the count by $n$ divides its variance by $n^2$.

E(\hat{p})=p_0, \qquad \operatorname{Var}(\hat{p}) =\frac{np_0(1-p_0)}{n^2} =\frac{p_0(1-p_0)}{n}, \qquad SE_0=\sqrt{\frac{p_0(1-p_0)}{n}}.

Thus $z$ measures the observed difference $\hat{p}-p_0$ in units of the standard deviation expected under the null model. A difference of 0.02, for example, cannot be judged by its size alone: it may be small compared with the standard error in a small sample and comparatively large in a much larger one.

Worked Tests of Public Percentage Claims

Worked Example: A Survey Claim of 55%

Suppose a hypothetical public report claims that 55% of eligible residents support a proposed service. A random sample of 400 eligible residents is surveyed, and 228 say they support it. Assume the sampling process is appropriate, responses are independent, and the eligible population is much larger than 4,000. Test whether support differs from the reported 55% at $\alpha=0.05$.

Hypotheses. Let $p$ be the proportion of all eligible residents who support the service. Since the question asks whether the proportion differs from the claim in either direction, $H_0:p=0.55$ and $H_a:p\ne0.55$.

Conditions. The hypothetical sample is random and independent, and 400 is less than 10% of the population. Under the null, the expected number supporting the service is $np_0=400(0.55)=220$; the expected number not supporting it is $n(1-p_0)=400(0.45)=180$. Both exceed 10.

Sample proportion and standard error. The observed proportion is $\hat{p}=228/400=0.57$. The null standard error is

SE_0=\sqrt{\frac{0.55(0.45)}{400}} =\sqrt{0.00061875} \approx0.02487.

Test statistic and p-value. To keep the arithmetic consistent, calculate the statistic using the unrounded standard error expression:

z=\frac{0.57-0.55}{\sqrt{0.55(0.45)/400}} \approx0.8040.

The two-sided p-value is $2[1-\Phi(0.8040)]\approx 0.4214$. If the null claim were correct, a standardized difference at least this far from zero in either direction would not be unusual.

Conclusion. Since $0.4214>0.05$, we do not reject $H_0$. These data do not provide statistically significant evidence at the 5% level that the population support percentage differs from 55%. This does not establish that the true percentage is exactly 55%; the sample simply does not show a sufficiently unusual departure from that value.

Worked Example: The Sample Percentage Falls Below the Claim

In a second hypothetical survey, a public statement claims that 40% of customers would recommend a service. A random sample of 250 customers is asked, and 89 say they would recommend it. Assume the sample is independent, the customer population is much larger than 2,500, and the test is conducted at $\alpha=0.05$. Is there evidence that the population percentage differs from 40%?

Hypotheses. Let $p$ be the proportion of all customers who would recommend the service. The question concerns a difference in either direction, so $H_0:p=0.40$ and $H_a:p\ne0.40$.

Conditions and observed proportion. The hypothetical random sample is less than 10% of the customer population, and the individual responses are treated as independent. Under $H_0$, the expected success and failure counts are $250(0.40)=100$ and $250(0.60)=150$, respectively; both are at least 10. The sample proportion is $\hat{p}=89/250=0.356$.

Standard error and statistic.

SE_0=\sqrt{\frac{0.40(0.60)}{250}} =\sqrt{0.00096} \approx0.03098, \qquad z=\frac{0.356-0.40}{\sqrt{0.40(0.60)/250}} \approx-1.420.

The negative sign indicates that the sample proportion is below the claimed proportion. Because the alternative is two-sided, the p-value includes outcomes as far below the claim as this result and outcomes equally far above it: $2[1-\Phi(1.420)]\approx0.1556$.

Conclusion. The p-value is greater than 0.05, so we do not reject the null hypothesis at the 5% significance level. The sample does not provide statistically significant evidence that the population proportion of customers who would recommend the service differs from 40%. It would be incorrect to conclude that the data show a statistically significant decrease merely because $\hat{p}$ is below 0.40.

Worked Example: A Sample Percentage Well Above the Claim

Suppose a hypothetical announcement claims that 30% of users complete a training program. In a random sample of 500 users, 180 completed it. Assume the sample is independent and comes from a population much larger than 5,000. Test whether the completion percentage differs from 30% at $\alpha=0.05$.

Hypotheses and conditions. Let $p$ be the proportion of all users who complete the program. The hypotheses are $H_0:p=0.30$ and $H_a:p\ne0.30$. The stated random sampling and large population support the independence check. Under $H_0$, the expected success and failure counts are $500(0.30)=150$ and $500(0.70)=350$; both are at least 10.

Calculation. The observed proportion is $\hat{p}=180/500=0.36$. The null standard error and test statistic are

SE_0=\sqrt{\frac{0.30(0.70)}{500}} =\sqrt{0.00042} \approx0.02049, \qquad z=\frac{0.36-0.30}{\sqrt{0.30(0.70)/500}} \approx2.928.

The two-sided p-value is $2[1-\Phi(2.928)]\approx0.00341$. This is the probability, under the null model, of a z statistic at least 2.928 units from zero in either direction.

Conclusion. Since $0.00341<0.05$, reject $H_0$ at the 5% level. The data provide statistically significant evidence that the population completion percentage differs from 30%. The observed sample percentage is higher than 30%, but this test does not prove that the population percentage is exactly 36% or establish why it differs from the claim.

Making the Conclusion Match the Test

A complete conclusion links the decision to the public claim and the population in question. When the p-value is below the chosen significance level $\alpha$, the result is statistically significant evidence against $H_0$, in the direction of the two-sided alternative: the population proportion differs from $p_0$. The sample direction can be stated as context, but it should not change the two-sided hypotheses or the p-value calculation.

When the p-value is at least $\alpha$, the appropriate conclusion is that the data do not provide sufficient evidence to reject the claimed proportion at that significance level. A large p-value does not verify the public claim, show that the population proportion equals $p_0$, or mean that the null hypothesis has a high probability of being true. It says that the observed result is not sufficiently unusual under the null model to reject it using the selected rule.

The test also does not, by itself, establish the size or practical importance of a difference, nor does it establish a causal explanation. It evaluates compatibility between the sample data and a specific null model. Its conclusion is only as credible as the sampling process, outcome measurement, and model assumptions.

Common mistake: A learner sees $\hat{p}>p_0$, uses only the upper tail, and calls the result a test of whether the percentage “differs” from the claim. That changes the question to whether the proportion is greater. For a two-sided claim of a difference, use $H_a:p\ne p_0$ and double the one-tail probability beyond $|z_{\text{obs}}|$. Also avoid saying “accept the claim” when a result is not significant; the correct conclusion is that the data do not provide sufficient evidence against it.

Define the population proportion.
State what counts as success and identify the population to which the claim refers.

Write two-sided hypotheses.
For a claim of an exact percentage $p_0$, use $H_0:p=p_0$ and $H_a:p\ne p_0$ when the question asks whether it differs.

Check the model conditions.
Assess the sampling process, independence, binary outcome, and null expected success and failure counts.

Calculate and interpret.
Find $\hat{p}$, use the null standard error to calculate $z$, obtain the two-sided p-value, and conclude in context at the chosen $\alpha$.

Why Two-Sided Testing Matters

Public percentages are often presented as single numbers, but a sample can contradict such a claim in either direction. A two-sided test treats an observed proportion above the claim and one below it symmetrically when their standardized distances are equal. Choosing only one tail after seeing which direction the sample went would not answer the same question and would understate the evidence against equality.

The practical discipline is to set the hypotheses from the question before interpreting the sample result. For a claim that a population percentage is exactly $p_0$, the two-sided test provides a structured comparison: it quantifies how unusual the observed departure would be if that exact claim and the model assumptions were true. It does not turn the sample into certainty about the whole population.

Check Your Understanding

Answer each question using the two-sided one-proportion test framework.

A statement claims that 62% of households use a particular service. What hypotheses test whether the population percentage differs from this claim?
Why are the null expected success and failure counts calculated using $p_0$, rather than the observed $\hat{p}$?
A sample proportion is below the claimed proportion, but the stated alternative is $p\ne p_0$. Which tail or tails contribute to the p-value, and why?
If a test gives a p-value of 0.18 and $\alpha=0.05$, what conclusion is warranted? What conclusion is not warranted?
In a two-sided test, $z_{\text{obs}}=-2.1$. Express the p-value using the standard normal cumulative distribution function.

What You'll Learn