Refining Sample Size With a Pilot Estimate

Why a Pilot Estimate Needs Refinement

Planning the sample size for a mean interval requires an estimate of how variable individual measurements are in the population. Usually, the population standard deviation \(\sigma\) is not known before data collection. A small pilot sample can provide a preliminary estimate, its sample standard deviation \(s\), to use in planning. The resulting sample size is a reasoned plan, not a guarantee: the pilot estimate may differ from the variability in the eventual study.

For a two-sided t confidence interval for a population mean, the margin of error depends on the sample standard deviation, the sample size, and a t critical value. The critical value itself depends on the degrees of freedom, which depend on the sample size being chosen. This dependence is why a normal-based calculation is useful as a starting point but must be refined for a t interval.

Definition: A pilot estimate is a preliminary estimate of a quantity needed for planning a study. For planning a mean interval, the pilot sample standard deviation \(s_{\text{pilot}}\) is used as an estimate of the standard deviation that may be observed in the planned sample. It is a planning input, not a known value of the population standard deviation.

The Planning Condition for a t Interval

Suppose the goal is a two-sided confidence interval with confidence level \(C\) and a margin of error no greater than a chosen target \(E\). For a one-sample t interval, the planned margin of error, using the pilot standard deviation, is

ME_{\text{planned}}=t^*_{n-1}\frac{s_{\text{pilot}}}{\sqrt{n}}.

Here \(t^*_{n-1}\) is the positive t critical value for confidence level \(C\) with \(n-1\) degrees of freedom. The planning condition is

t^*_{n-1}\frac{s_{\text{pilot}}}{\sqrt{n}}\leq E.

Unlike a formula with a fixed critical value, this condition has \(n\) in two places: in the square root and in the degrees of freedom used to find \(t^*\). Therefore, a t-based plan is found by trying sample sizes and checking the inequality, rather than by solving a single closed-form equation.

Planning principle: Use a normal critical value to get an initial sample-size estimate if useful, then check the t-based margin of error at the candidate \(n\), using \(df=n-1\). Increase \(n\) and check again if the margin exceeds the target.

Why the Normal Calculation Is Only a Starting Point

If a normal critical value \(z^*\) is used in place of the t critical value, the planning condition can be rearranged directly:

z^*\frac{s_{\text{pilot}}}{\sqrt{n}}\leq E \quad\Longleftrightarrow\quad \sqrt{n}\geq\frac{z^*s_{\text{pilot}}}{E} \quad\Longleftrightarrow\quad n\geq\left(\frac{z^*s_{\text{pilot}}}{E}\right)^2.

This gives a useful initial estimate. But for finite degrees of freedom, the t distribution has heavier tails than the standard normal distribution, so its critical value is larger. The t critical value also depends on \(n-1\): as the planned sample size increases, the degrees of freedom rise and the t critical value moves closer to the normal critical value. Both the decrease in \(1/\sqrt{n}\) and the change in \(t^*\) affect the margin.

The refinement is straightforward: evaluate the t-based margin at the initial candidate; if it is too large, test a larger sample size. Continue until the planning condition is met. For a fixed confidence level and positive \(s_{\text{pilot}}\), the t critical value decreases as degrees of freedom increase, while \(1/\sqrt{n}\) also decreases. Thus the planned margin decreases as \(n\) increases, making this search well behaved.

Worked Example: Refining a 95% Plan

Suppose a hypothetical pilot sample gives \(s_{\text{pilot}}=12\) units. The planned study seeks a two-sided 95% t interval with a margin of error no greater than \(E=2.5\) units. Find the smallest sample size that meets the t-based planning condition.

For a 95% interval, the normal critical value is approximately \(z^*=1.96\). The initial estimate is

n\geq\left(\frac{1.96(12)}{2.5}\right)^2 =\left(9.408\right)^2 =88.5105\ldots

This suggests beginning with a sample size near 89. Now check a candidate using the t critical value, rather than assuming the normal approximation is exact. At \(n=90\), \(df=89\) and \(t^*\approx1.987\), so

ME_{\text{planned}} =1.987\frac{12}{\sqrt{90}} =\frac{23.844}{9.48683\ldots} \approx2.513\text{ units}.

This exceeds the 2.5-unit target. At \(n=91\), \(df=90\) and \(t^*\approx1.9867\):

ME_{\text{planned}} =1.9867\frac{12}{\sqrt{91}} =\frac{23.8404}{9.53939\ldots} \approx2.499\text{ units}.

The condition is met at \(n=91\), whereas it fails at \(n=90\). Since the planned margin decreases as \(n\) increases, 91 is the smallest sample size that meets the target under this pilot-based calculation.

A Refinement Procedure

A practical calculation can be organized into a short sequence. The initial normal estimate does not replace the t check; it only gives a sensible place to start. Software or a t table can supply the critical values, provided the degrees of freedom correspond to each candidate sample size.

Specify the target.
Choose the two-sided confidence level and the largest acceptable margin of error \(E\), in the measurement’s units.

Choose a pilot estimate.
Use the pilot sample standard deviation \(s_{\text{pilot}}\) as a preliminary estimate of variability for the planned sample.

Find a starting candidate.
Calculate \(n\approx(z^*s_{\text{pilot}}/E)^2\) to identify an approximate sample size.

Check the t margin.
For each candidate \(n\), use \(df=n-1\), find the corresponding t critical value, and calculate \(t^*s_{\text{pilot}}/\sqrt{n}\).

Refine as needed.
If the calculated margin is greater than \(E\), test a larger \(n\). Stop at the first candidate that satisfies the condition.

Worked Example: A Higher Confidence Level

A hypothetical planning team has a pilot estimate of \(s_{\text{pilot}}=8\) minutes and wants a 99% t interval with a margin of error no greater than \(2\) minutes. Find the first sample size that meets the planned t condition.

The 99% normal critical value is approximately \(z^*=2.576\). The normal-based starting estimate is

n\geq\left(\frac{2.576(8)}{2}\right)^2 =(10.304)^2 =106.1724\ldots

This gives a starting candidate around 107. Checking the t-based margin shows that a larger sample is needed. At \(n=109\), \(df=108\), and \(t^*\approx2.622\):

ME_{\text{planned}} =2.622\frac{8}{\sqrt{109}} =\frac{20.977}{10.4403\ldots} \approx2.009\text{ minutes}.

This is slightly above the target. At \(n=110\), \(df=109\), and \(t^*\approx2.6217\):

ME_{\text{planned}} =2.6217\frac{8}{\sqrt{110}} =\frac{20.974}{10.4881\ldots} \approx1.9997\text{ minutes}.

The planned margin at 110 is at most 2 minutes, while it is still above 2 minutes at 109. Therefore, 110 is the smallest sample size meeting this pilot-based t condition. The critical values are close at these neighboring degrees of freedom, but the check matters because the target is tight.

Another Example: A 90% Interval

Worked Example: Checking a Lower Confidence Level

Suppose a hypothetical pilot sample has \(s_{\text{pilot}}=6\) hours. A future study aims for a two-sided 90% t interval with margin of error no greater than \(1.5\) hours. Determine the sample size by refining the normal estimate.

For a 90% interval, \(z^*\approx1.645\). The initial calculation is

n\geq\left(\frac{1.645(6)}{1.5}\right)^2 =(6.58)^2 =43.2964.

Begin by checking nearby candidates. At \(n=44\), \(df=43\), and \(t^*\approx1.681\):

ME_{\text{planned}} =1.681\frac{6}{\sqrt{44}} =\frac{10.086}{6.63325\ldots} \approx1.520\text{ hours}.

The margin is too large. At \(n=45\), \(df=44\), with \(t^*\approx1.680\):

ME_{\text{planned}} =1.680\frac{6}{\sqrt{45}} =\frac{10.080}{6.70820\ldots} \approx1.503\text{ hours}.

This also exceeds the target. At \(n=46\), \(df=45\), and \(t^*\approx1.679\):

ME_{\text{planned}} =1.679\frac{6}{\sqrt{46}} =\frac{10.074}{6.78233\ldots} \approx1.486\text{ hours}.

Thus, 46 is the first of these candidates to meet the planned margin requirement. This example shows why simply taking the normal estimate of about 44 observations can be insufficient: the t critical value adds a little width at these degrees of freedom.

Assumptions and Limits of the Plan

A pilot-based sample-size calculation is only as relevant as its inputs and the conditions for using the eventual t interval. Before relying on the plan, check the following points:

Representative variability: The pilot observations should come from a population and measurement process relevant to the planned study. If the pilot is unusually homogeneous or unusually variable, its \(s_{\text{pilot}}\) may give a poor planning estimate.
Sampling and independence: The planned data should come from an appropriate random sample or design, with observations reasonably independent. For sampling without replacement from a finite population, check the 10% condition when it applies.
Shape of the measurement distribution: The t interval relies on a suitable distribution of the sample mean. For small samples, the population measurements should be reasonably close to normal and free of severe outliers; larger samples can make inference more robust, but do not correct every design or data problem.
Uncertainty in the pilot standard deviation: A small pilot sample can estimate variability imprecisely. The calculated sample size should therefore be understood as conditional on the chosen pilot estimate, not as a promise that the realized interval will have exactly the planned margin.

The planning expression substitutes \(s_{\text{pilot}}\) for the standard deviation that will be observed in the full study. The actual sample standard deviation can be smaller or larger. Consequently, the achieved interval can be narrower or wider than the planned margin even if the planned sample size is followed. If missing observations or unusable measurements are anticipated, that practical issue also needs to be accounted for in the overall plan.

Common Mistake

A common mistake is to calculate \(n\) using \(z^*\), round to a convenient candidate, and treat that number as the finished t-based answer. For instance, in the 95% example, a normal calculation gives a value near 89, but the t margin at \(n=90\) is about 2.513 units, which is above the target of 2.5 units. The correction is to use the normal result only as a starting estimate, calculate the t margin with \(df=n-1\), and keep checking until the inequality is satisfied.

Another error is to interpret the pilot standard deviation as if it were the known population standard deviation. It is an estimate. A plan based on \(s_{\text{pilot}}\) is informative only to the extent that the pilot provides a reasonable guide to variability in the future data.

Key takeaway: A pilot sample standard deviation supplies a practical estimate of variability for planning a mean interval. Use it to obtain a starting sample size, then evaluate the t-based margin at the candidate degrees of freedom. The resulting plan targets a margin under stated assumptions; it does not ensure that the realized interval will attain that exact width.

Check Your Understanding

Try these questions without looking back at the worked calculations.

Why does the t-based sample-size condition have \(n\) in both the denominator’s square root and the degrees of freedom?
A normal-based calculation gives a candidate sample size. What must be checked before treating that candidate as adequate for a t interval?
Explain why a pilot standard deviation is a planning estimate rather than a known population standard deviation.
If the t-based planned margin at a candidate \(n\) is larger than the target, what should be done next, and why does that change help?
Name two conditions or limitations that should be considered before relying on a pilot-based sample-size plan.