Why Sample Size Controls Precision
When an estimator is based on more independent observations, its uncertainty generally decreases. For regular parametric models, Fisher information quantifies this relationship: the information in a sample of size \(n\) is \(n\) times the information in one observation. Consequently, the approximate standard error of a maximum likelihood estimator decreases in proportion to \(1/\sqrt{n}\), not \(1/n\). This square-root relationship is central to planning an observation count for a desired level of precision.
A sample-size calculation is only as useful as its assumptions and target. A standard error target is not the same as a confidence-interval half-width target, and both depend on a planning value for the unknown model parameter. The calculations below make these distinctions explicit.
From Information to a Sample-Size Rule
The basic planning rule follows by setting the approximate standard error no greater than a chosen tolerance \(\varepsilon>0\). Because information is positive in an identifiable regular model, the inequality can be rearranged directly. The smallest admissible sample size is the ceiling of the resulting bound, since the number of observations must be an integer.
The parameter value in this rule is usually unknown before data collection. A planning calculation therefore uses a specified value, such as a value from earlier experience, a pilot study, or a conservative choice over a plausible range. The resulting count is conditional on that planning value; it does not guarantee that the realized standard error will meet the target in every sample.
If the goal is instead a two-sided approximate confidence interval with confidence level \(1-\alpha\) and half-width at most \(h\), use the normal critical value \(z_{1-\alpha/2}\). The approximate half-width is that critical value multiplied by the standard error, so the sample-size rule includes the square of the critical value.
This interval-based rule is appropriate when the normal approximation is adequate and the interval construction being planned is the stated normal approximation. It is not a substitute for a method designed for small samples, boundary parameters, or strongly skewed estimator distributions. In particular, a half-width is not itself a standard error: omitting the critical value can materially understate the required sample size.
Why Information Adds
For independent observations, the joint likelihood is a product of the individual likelihoods. Taking logarithms turns that product into a sum, so the log-likelihood curvature contributed by each observation accumulates. Taking expectations gives the additivity of expected Fisher information.
For independent observations with individual log-likelihood contributions \(\ell_i(\theta)\), the joint log-likelihood is \(\ell_n(\theta)=\sum_{i=1}^n\ell_i(\theta)\).
The negative second derivative is \(-\ell_n''(\theta)=\sum_{i=1}^n[-\ell_i''(\theta)]\).
Each observation has the same expected information \(I_1(\theta)=E[-\ell_i''(\theta)]\). By linearity of expectation, \(I_n(\theta)=E[-\ell_n''(\theta)]=\sum_{i=1}^n I_1(\theta)=nI_1(\theta)\).
For a regular scalar model, the large-sample variance approximation is the reciprocal of total information, \(1/I_n(\theta)=1/[nI_1(\theta)]\). Taking the square root yields the standard-error rule.
This argument relies on independent observations from the same model and on regularity conditions that justify the information and maximum-likelihood approximations. With dependent observations, the information need not equal \(nI_1(\theta)\). Likewise, a singular or non-identifiable model cannot be handled by simply applying the scalar reciprocal-information rule.
Worked Sample-Size Calculations
Worked Example: Precision for an Exponential Loss Mean
A hypothetical portfolio is modeled as having independent, identically distributed positive losses \(X_1,\ldots,X_n\), each exponential with mean \(\theta\). For planning, use \(\theta=4{,}000\) dollars. Find the minimum number of losses for the approximate standard error of the maximum likelihood estimate of \(\theta\) to be at most \(400\) dollars. Check the chosen count and the count immediately below it.
Model and method. The density is \(f_X(x)=\theta^{-1}e^{-x/\theta}\) for \(x>0\) and \(\theta>0\). Under this model, the maximum likelihood estimator is \(\hat{\theta}=\bar{X}\). For one observation, the log-likelihood is \(\ell(\theta)=-\log\theta-X/\theta\), so \(\ell''(\theta)=1/\theta^2-2X/\theta^3\). Since \(E[X]=\theta\), the expected information is \(I_1(\theta)=-E[\ell''(\theta)]=-(1/\theta^2-2\theta/\theta^3)=1/\theta^2\).
Calculate the bound. The information rule gives \(n\geq 1/[I_1(\theta)\varepsilon^2]=\theta^2/\varepsilon^2\). With the stated planning values, \(n\geq (4{,}000)^2/(400)^2=16{,}000{,}000/160{,}000=100\). Thus \(n_{\min}=100\).
Check. At \(n=100\), the approximate standard error is \(4{,}000/\sqrt{100}=400\) dollars, which meets the target exactly. At \(n=99\), it is \(4{,}000/\sqrt{99}\approx402.02\) dollars, which exceeds the target. The minimum is therefore confirmed. This model also gives \(\operatorname{Var}(\bar{X})=\theta^2/n\) exactly, although the information-based result is stated as an asymptotic standard-error rule.
Method and trap avoided. This is a per-observation information calculation using a specified planning mean. The count is rounded up, not down; rounding 99.01 or any other noninteger bound down would fail the precision requirement.
Worked Example: Standard-Error Target for a Bernoulli Probability
A hypothetical claim-screening process records whether each independent claim has a specified feature. Model each result as Bernoulli with success probability \(p\). Using planning value \(p=0.30\), find the minimum number of observations for the approximate standard error of the maximum likelihood estimate \(\hat p\) to be no greater than \(0.03\).
Model and information. For one Bernoulli observation \(X\in\{0,1\}\), the log-likelihood is \(X\log p+(1-X)\log(1-p)\). Its expected Fisher information is \(I_1(p)=1/[p(1-p)]\). Thus the approximate variance of \(\hat p\) is \(p(1-p)/n\).
Calculate the bound. The planning value gives \(p(1-p)=0.30(0.70)=0.21\). Requiring \(\sqrt{0.21/n}\leq0.03\) gives \(n\geq0.21/(0.03)^2=0.21/0.0009=233.333\ldots\). Therefore the minimum integer sample size is \(234\).
Check. At \(n=234\), the approximate standard error is \(\sqrt{0.21/234}=\sqrt{0.0008974359}\approx0.02996\), which is below \(0.03\). At \(n=233\), it is \(\sqrt{0.21/233}=\sqrt{0.0009012876}\approx0.03002\), which is above \(0.03\). Hence \(234\) is the minimum.
Method and trap avoided. The target here is a standard error, not a confidence-interval half-width, so no normal critical value is included. The calculation also uses \(p(1-p)\), not its reciprocal, after rearranging the information formula.
Worked Example: Confidence-Interval Half-Width for a Bernoulli Probability
A hypothetical audit estimates the proportion \(p\) of records with a specified coding issue. Assume independent Bernoulli observations and use planning value \(p=0.20\). Find the minimum sample size for an approximate 95% normal confidence interval for \(p\) to have half-width no greater than \(0.10\). Use \(z_{0.975}=1.96\).
Set up the half-width condition. The approximate standard error is \(\sqrt{p(1-p)/n}\). The required half-width is \(1.96\sqrt{0.20(0.80)/n}\), which must be at most \(0.10\).
Calculate the bound. Squaring both positive sides gives \(n\geq1.96^2(0.20)(0.80)/(0.10)^2\). Here \(1.96^2=3.8416\), \(0.20(0.80)=0.16\), and \((0.10)^2=0.01\). Therefore \(n\geq3.8416(0.16)/0.01=0.614656/0.01=61.4656\), and \(n_{\min}=62\).
Check. At \(n=62\), the half-width is \(1.96\sqrt{0.16/62}=1.96\sqrt{0.0025806452}\approx1.96(0.05080)=0.09957\), which meets the target. At \(n=61\), it is \(1.96\sqrt{0.16/61}=1.96\sqrt{0.0026229508}\approx1.96(0.05122)=0.10038\), which does not. The minimum is confirmed.
Method and trap avoided. This is a confidence-interval half-width calculation, so it includes \(1.96\). Using only the standard-error formula would give \(n\geq0.16/0.01=16\), which is not enough for a 95% interval: at \(n=16\), the approximate half-width is \(1.96\sqrt{0.16/16}=0.196\), almost twice the target.
Interpreting and Using the Result
The square-root relationship gives a quick way to compare designs. If a model and planning value remain unchanged, reducing the standard error by a factor of two requires multiplying the sample size by four: \(1/\sqrt{4n}=(1/2)(1/\sqrt n)\). More generally, multiplying the sample size by \(c\) divides the approximate standard error by \(\sqrt c\). The relationship is gradual; doubling the sample size does not halve the standard error.
A reliable calculation should identify what is being estimated, state the assumed model and planning parameter value, and distinguish a standard-error target from an interval-width target. It should then keep the unrounded bound until the final ceiling operation and verify that the selected integer satisfies the inequality. These checks catch both conceptual and arithmetic errors.
- Use a plausible planning value. Information can depend on the unknown parameter. Report the assumption and examine whether a different plausible value would change the count.
- Do not interpret an approximation as a guarantee. Actual precision depends on the data and model. Small samples, boundary values, and model misspecification can make normal approximations unreliable.
- Check the observation structure. The simple \(nI_1(\theta)\) formula assumes independent observations with the same model. Dependence or unequal information per observation requires a different calculation.
Check Your Understanding
For each problem, state the model and use the stated planning values. Round the required observation count up to the next integer.
- Independent losses follow an exponential distribution with mean \(\theta=1{,}200\) dollars. What sample size gives an approximate standard error for the maximum likelihood estimate of \(\theta\) no greater than \(100\) dollars? (Answer: 144 observations)
- Independent outcomes are Bernoulli with planning probability \(p=0.40\). What sample size gives an approximate standard error for \(\hat p\) no greater than \(0.04\)? (Answer: 150 observations)
- Independent outcomes are Bernoulli with planning probability \(p=0.25\). Using a 90% normal interval with \(z_{0.95}=1.645\), what sample size gives an approximate half-width no greater than \(0.05\)? (Answer: 203 observations)
- Observations are independent and normally distributed with known variance \(\sigma^2=360{,}000\). Using the maximum likelihood estimator of the mean, what sample size gives a standard error no greater than \(75\)? (Answer: 64 observations)
- Independent outcomes are Bernoulli with planning probability \(p=0.50\). Using a 95% normal interval with \(z_{0.975}=1.96\), what sample size gives an approximate half-width no greater than \(0.04\)? (Answer: 601 observations)