Sample Variance and the N Minus One Convention

Why Do Two Variance Formulas Use Different Denominators?

Variance describes how spread out quantitative observations are around their mean. If the data set contains every value in the population of interest, the population variance divides the sum of squared deviations by the number of values. If the data are a sample used to learn about a larger population, the usual sample variance divides that sum by one fewer than the sample size. The difference is small to state but important to calculate correctly.

The reason for the adjustment is that the sample mean is calculated from the same observations whose spread is being measured. It tends to sit near those observations, making their squared deviations from it smaller, on average, than deviations from the population mean would be. Dividing by \(n-1\) corrects this systematic tendency when the sample is random and observations have finite variance.

The expressions in the numerators have the same structure: find the mean, subtract it from each observation, square each difference, and add those squares. The denominator depends on what the data represent. Use \(N\) when the values are the entire population being described; use \(n-1\) for the usual sample variance when the sample mean is estimated from the sample.

Calculating Sample Variance

A reliable calculation follows a fixed sequence. Keep the squared deviations as a separate total before dividing; this makes it easier to check arithmetic and avoid mixing up the denominator. The sample mean must be used to calculate the deviations in the sample variance formula.

For comparison, if those exact four values were the complete population under description, their mean would still be 5 and their squared deviations would still sum to 20. The population variance would then be \(20/4=5\), not \(20/3\). The observations have not changed; the question about whether they are a sample or a whole population determines which variance is appropriate.

Why the Sample Formula Uses \(n-1\)

The denominator has a statistical justification beyond convention. Under a random-sampling model, the sample variance with denominator \(n-1\) is an unbiased estimator of the population variance. “Unbiased” means that if we repeatedly drew samples in the same way and calculated \(s^2\) each time, the average of those sample variances would equal the population variance.

For the derivation, suppose \(X_1,\ldots,X_n\) are independent observations from the same population, with mean \(\mu\) and finite variance \(\sigma^2\). Their sample mean is \(\bar{X}\). The identity below holds for every set of observations: $$ \sum_{i=1}^{n}(X_i-\bar{X})^2 = \sum_{i=1}^{n}(X_i-\mu)^2-n(\bar{X}-\mu)^2. $$

To see the identity, write \(X_i-\bar{X}=(X_i-\mu)-(\bar{X}-\mu)\), square, and sum: $$ \begin{aligned} \sum_{i=1}^{n}(X_i-\bar{X})^2 &=\sum_{i=1}^{n}(X_i-\mu)^2 -2(\bar{X}-\mu)\sum_{i=1}^{n}(X_i-\mu) +n(\bar{X}-\mu)^2. \end{aligned} $$ Because \(\sum_{i=1}^{n}(X_i-\mu)=n(\bar{X}-\mu)\), the middle term is \(-2n(\bar{X}-\mu)^2\). Combining it with the final term leaves \(-n(\bar{X}-\mu)^2\), which proves the identity.

Now take expectations. Each squared deviation from \(\mu\) has expected value \(\sigma^2\), so the first sum has expected value \(n\sigma^2\). Independence gives the sample mean variance \(\operatorname{Var}(\bar{X})=\sigma^2/n\), and therefore \(E[(\bar{X}-\mu)^2]=\sigma^2/n\). It follows that $$ E\left[\sum_{i=1}^{n}(X_i-\bar{X})^2\right] =n\sigma^2-n\left(\frac{\sigma^2}{n}\right) =(n-1)\sigma^2. $$ Dividing this sum by \(n-1\) gives $$ E[s^2]=E\left[\frac{\sum_{i=1}^{n}(X_i-\bar{X})^2}{n-1}\right] =\frac{(n-1)\sigma^2}{n-1} =\sigma^2. $$ Thus the \(n-1\) denominator corrects the average downward bias that would result from dividing the squared deviations around the sample mean by \(n\).

Conditions, Interpretation, and Common Mistakes

The sample formula can be calculated for any list of at least two numbers. Its interpretation as an unbiased estimate of a larger population variance, however, depends on the sampling model used in the derivation: observations should be independent and drawn from the same population, and that population must have a finite variance. These assumptions are often plausible for a random sample, but they do not follow just from having a list of numbers. A biased or dependent sampling process can make the estimate systematically unrepresentative.

If the data include every member of the population being described, use the population formula instead. If the goal is to estimate the variance of a broader population using a sample, use the sample formula. The formulas are not rival versions of one fixed answer; the intended population and the role of the data must be clear.

• Using \(n\) for a sample variance around \(\bar{x}\). This generally underestimates the population variance on average under the random-sampling model. Use \(n-1\) for the usual sample variance.
• Subtracting one from the data. The \(n-1\) convention changes the denominator; it does not mean subtracting one from each observation or from the sum of squares.
• Using \(n-1\) for a complete population. If the data are the whole population of interest, divide by \(N\) to describe that population's variance.
• Trying to calculate sample variance with one observation. For \(n=1\), \(n-1=0\), so the sample variance formula is undefined. One observation cannot show how values vary around a sample mean.

The central decision is therefore conceptual before it is computational: decide whether the data are the full population or a sample used to estimate features of a larger population. After that, the denominator follows directly from the definition.