Rounding Sample Size and Checking Precision

Why Rounding Is Part of the Calculation

A sample-size formula can produce a decimal, but a study cannot collect a fraction of an observation. The decimal result is a threshold: it tells us how large the sample must be to meet a chosen margin-of-error target under a stated planning model. Rounding down—even when the decimal part is small—can put the actual planned margin of error above that target.

For a confidence interval for a population proportion, the required sample size depends on the confidence level, the desired margin of error, and the proportion used to plan. Once the calculation is rounded up, recomputing the margin of error provides a direct check that the whole-number sample size meets the target under that same plan.

The Planning Formula and Its Assumptions

Definition: Let $E$ be the desired maximum margin of error, let $z^*$ be the positive standard-normal critical value for the chosen two-sided confidence level, and let $p_{\text{plan}}$ be the proportion assumed for planning. The Wald-based planned margin of error at sample size $n$ is $$ E_{\text{plan}}=z^*\sqrt{\frac{p_{\text{plan}}(1-p_{\text{plan}})}{n}}. $$ Solving $E_{\text{plan}}\le E$ for $n$ gives the required sample-size threshold $$ n_{\text{raw}}=\frac{(z^*)^2p_{\text{plan}}(1-p_{\text{plan}})}{E^2}. $$ The whole-number sample size is the smallest integer at least as large as $n_{\text{raw}}$.

The planning proportion may come from earlier relevant data, such as a pilot study, or from a deliberately chosen conservative value. If no useful prior estimate is available, $p_{\text{plan}}=0.5$ is conservative for this formula because $p(1-p)$ is largest at $p=0.5$. The margin of error $E$ is expressed in proportion units: a target of 4 percentage points means $E=0.04$, not $4$.

This formula plans for the usual two-sided Wald interval for a proportion. Its use assumes a random or otherwise appropriate sample, independent observations (or a suitable correction when sampling a substantial fraction of a finite population), and conditions adequate for the normal approximation. For planning, check that the expected counts $np_{\text{plan}}$ and $n(1-p_{\text{plan}})$ are sufficiently large; the course’s success-failure condition is one standard check. These checks support the planned approximation; they do not make a Wald interval suitable in every situation.

Why the Sample Size Must Be Rounded Up

The formula for $n_{\text{raw}}$ comes from requiring the planned margin of error not to exceed $E$. Since $n$ is in the denominator inside a square root, increasing $n$ decreases the planned margin of error. The required integer is therefore the ceiling of $n_{\text{raw}}$: the smallest whole number that is no smaller than the threshold.

E_{\text{plan}}\le E \quad\Longleftrightarrow\quad z^*\sqrt{\frac{p_{\text{plan}}(1-p_{\text{plan}})}{n}}\le E \quad\Longleftrightarrow\quad n\ge \frac{(z^*)^2p_{\text{plan}}(1-p_{\text{plan}})}{E^2}.

The equivalence follows by dividing by the positive $z^*$, squaring both nonnegative sides, and rearranging the resulting inequality. Because the margin decreases as $n$ increases, the ceiling meets the requirement; if the raw result is not an integer, the integer immediately below it does not.

Worked Example: A 95% Plan With No Pilot Estimate

A hypothetical poll seeks a margin of error no larger than 0.04 at 95% confidence. No prior estimate is available, so the planners use $p_{\text{plan}}=0.50$. For 95% confidence, take $z^*=1.96$. Find the whole-number sample size and check its planned margin of error.

Calculate the threshold. Substitute the planned values into the formula: $$ n_{\text{raw}}= \frac{(1.96)^2(0.50)(1-0.50)}{(0.04)^2} = \frac{3.8416(0.25)}{0.0016} = 600.25. $$

Round up. The smallest integer at least 600.25 is $n=601$. Rounding to 600 would round down, so it cannot be assumed to meet the target.

Check the rounded value. $$ E_{\text{plan}}= 1.96\sqrt{\frac{(0.50)(0.50)}{601}} = 1.96\sqrt{\frac{0.25}{601}} \approx 0.03998. $$ The planned margin is about 0.03998, or 4.0 percentage points to three significant figures, so the target of 0.04 is met. At $n=600$, the planned margin is approximately $1.96\sqrt{0.25/600}=0.04001$, slightly above the target. This confirms why rounding down would not suffice.

Recomputing the Margin of Error

After rounding up, put the chosen integer $n$ back into the planned margin-of-error formula. Do not simply report the unrounded threshold as though it were the margin achieved by the whole-number sample size. The check should use the same confidence critical value and the same planning proportion as the sample-size calculation; changing either one changes the quantity being checked.

Specify the plan.
Record the target $E$, confidence level and its critical value $z^*$, and the planning proportion $p_{\text{plan}}$.

Find the threshold and round up.
Calculate $n_{\text{raw}}=(z^*)^2p_{\text{plan}}(1-p_{\text{plan}})/E^2$, then choose the smallest whole number at least that large.

Check precision and conditions.
Substitute the chosen integer into $E_{\text{plan}}$, compare it with $E$, and check that the planned approximation’s assumptions are reasonable.

Worked Example: Using a Pilot Proportion

In a hypothetical pilot poll, 32% of respondents select an option of interest. Planners use $\hat p_{\text{pilot}}=0.32$ as $p_{\text{plan}}$, and want a margin of error no larger than 0.03 at 90% confidence. Use $z^*=1.645$. Find the rounded sample size and verify the target.

Calculate the threshold. $$ n_{\text{raw}}= \frac{(1.645)^2(0.32)(0.68)}{(0.03)^2} = \frac{2.706025(0.2176)}{0.0009} \approx 654.26. $$ Thus the smallest whole-number sample size that reaches the threshold is $n=655$, not 654.

Recompute the planned margin. $$ E_{\text{plan}}= 1.645\sqrt{\frac{(0.32)(0.68)}{655}} = 1.645\sqrt{\frac{0.2176}{655}} \approx 0.02998. $$ This is just under 0.03, so the rounded sample size meets the target under the pilot-based plan. With $n=654$, the same calculation gives approximately 0.03001, just above the target.

The check concerns the planning value 0.32, not a promise that the final sample will have that proportion. A pilot estimate is useful planning information, but it is itself an estimate and may differ from the population proportion.

Planned Precision Versus the Observed Margin

The recomputation above verifies the planned margin using $p_{\text{plan}}$. Once data are collected, the conventional Wald interval uses the observed sample proportion $\hat p$, and its estimated margin of error is $$ z^*\sqrt{\frac{\hat p(1-\hat p)}{n}}. $$ That observed margin can differ from the planned one because $\hat p$ may differ from $p_{\text{plan}}$. Thus, a rounded sample size satisfies the planning calculation; it does not guarantee that every possible sample will produce an interval whose observed margin is at most the target.

Worked Example: The Observed Margin Can Differ

Suppose a hypothetical survey is planned at 99% confidence with $p_{\text{plan}}=0.40$ and target $E=0.05$. Use $z^*=2.576$. After collecting the planned sample, suppose 306 of 638 respondents choose the option, so $\hat p=306/638\approx0.4796$. Compare the planned and observed margins.

Plan and round. $$ n_{\text{raw}}= \frac{(2.576)^2(0.40)(0.60)}{(0.05)^2} = \frac{6.635776(0.24)}{0.0025} \approx 637.03. $$ The required whole-number sample size is $n=638$.

Verify planned precision. $$ E_{\text{plan}}= 2.576\sqrt{\frac{0.24}{638}} \approx 0.04996. $$ This meets the target of 0.05 under the planning value.

Calculate the observed margin. Since $\hat p=306/638\approx0.4796$, the observed Wald margin is $$ 2.576\sqrt{\frac{(0.4796)(1-0.4796)}{638}} \approx 0.05095. $$ This observed margin is slightly greater than 0.05. The sample-size arithmetic was not incorrect: the planned value 0.40 and the observed value about 0.4796 give different estimated proportions and therefore different margins. A planning calculation is not a guarantee about the realized sample.

Common Mistakes and Practical Checks

Rounding to the nearest integer. A raw result of 654.26 does not justify using 654. The requirement is a minimum, so round upward to 655.
Rounding the sample size but skipping the check. Substituting the integer into the margin formula confirms what precision that integer supports under the chosen planning values.
Mixing percentage points and proportions. A target of 3 percentage points is $E=0.03$. Using $E=3$ would produce a nonsensical sample-size calculation for this context.
Treating the planned margin as guaranteed. The planned check uses $p_{\text{plan}}$. After collection, the margin calculated with $\hat p$ can differ.
Ignoring the model conditions. Rounding correctly does not by itself establish that the sampling design is appropriate or that the normal approximation is adequate. Check random sampling, independence, and the planned success-failure counts.

These distinctions matter when a study has limited time or resources. Rounding down can miss a stated precision goal even under the planning model; rounding up and checking makes the calculation transparent. At the same time, describing the result as planned precision—not guaranteed realized precision—keeps the conclusion honest about sampling variability and assumptions.

Key takeaway: Calculate the raw sample-size threshold, round up to the smallest adequate integer, and recompute the planned margin of error with that integer. Then interpret the check as a result under the stated planning assumptions, not as a guarantee about the observed sample.

Check Your Understanding

For each question, distinguish the sample-size threshold from the check using the rounded integer.

A sample-size calculation gives $n_{\text{raw}}=412.08$. What whole-number sample size should be planned, and why is 412 insufficient under the formula?
A target margin of error is 2.5 percentage points. What value of $E$ belongs in the sample-size formula?
Why should the planned margin of error be recomputed after rounding $n_{\text{raw}}$ upward?
If the observed $\hat p$ differs from $p_{\text{plan}}$, must the observed Wald margin equal the planned margin? Explain.
Name two sampling or approximation conditions that should be considered in addition to rounding the calculated sample size.