Why Averaging Reduces Variability
A sample mean is often used to estimate a population mean. Its value changes from sample to sample, so the estimate has a sampling distribution and a corresponding variance. Understanding how that variance depends on sample size explains why larger independent samples tend to produce more stable averages—and why this stability is connected to the weak law of large numbers.
The key result is precise: for independent, identically distributed observations with finite variance, the sample mean has variance equal to the population variance divided by the sample size. This is a statement about variability across repeated samples. It does not say that every realized average is close to the population mean, or that the sample variance itself equals the variance of the sample mean.
Deriving the Variance of the Sample Mean
The derivation uses two standard variance rules: multiplying a random variable by a constant \(c\) multiplies its variance by \(c^2\), and the variance of a sum is the sum of variances plus twice the pairwise covariances. Independence makes those covariances zero.
The third line has \(n\) variance terms, each equal to \(\sigma^2\). Every covariance term is zero because the observations are independent. Identical distributions ensure that every observation has the same variance. The same assumptions give \(E[\bar{X}_n]=\mu\), since expectation is additive and scaling by \(1/n\) gives \(\frac{1}{n}(n\mu)=\mu\).
Variance and standard error describe the same sampling variability on different scales. Variance is measured in squared units of \(X_i\); standard error is measured in the original units. If the sample size is multiplied by four, the variance is divided by four, while the standard error is divided by two. In general, the standard error decreases at the rate \(1/\sqrt{n}\), not \(1/n\).
Worked Example: Precision of a Sample Defect Rate
A hypothetical production process has a fixed defect probability \(p=0.20\). For each inspected item, let \(X_i=1\) if it is defective and \(X_i=0\) otherwise. Assume items are inspected independently under the same process. Find the variance and standard error of the sample defect rate for \(n=25\), and determine the sample size needed to make its variance no greater than \(0.0016\).
Method. Each \(X_i\) is Bernoulli with parameter \(p=0.20\), so \(E[X_i]=p=0.20\) and \(\operatorname{Var}(X_i)=p(1-p)=0.20(0.80)=0.16\). The sample mean \(\bar{X}_n\) is the sample defect rate.
For \(n=25\), \(\operatorname{Var}(\bar{X}_{25})=0.16/25=0.0064\). Its standard error is \(\sqrt{0.0064}=0.08\), or 8 percentage points.
For the requested variance target, solve \(0.16/n\leq0.0016\). Multiplying by positive \(n\) and dividing by \(0.0016\) gives \(n\geq0.16/0.0016=100\). Thus the smallest qualifying integer sample size is 100.
Check. At \(n=100\), the variance is \(0.16/100=0.0016\) and the standard error is \(\sqrt{0.0016}=0.04\). The variance target is met exactly. The common mistake is to divide the standard error by \(n\); it is the variance that is divided by \(n\), while standard error is divided by \(\sqrt{n}\).
How Variance Shrinkage Supports the Weak Law
The weak law of large numbers concerns probabilities, not just variances. In the IID finite-variance setting, it says that for every fixed \(\varepsilon>0\), the probability that the sample mean differs from \(\mu\) by more than \(\varepsilon\) tends to zero as \(n\) increases:
This is called convergence in probability to \(\mu\). The sample mean is centered at \(\mu\), and its variance \(\sigma^2/n\) tends to zero. Thus its sampling distribution becomes increasingly concentrated around its expected value. A variance-based probability inequality makes that concentration statement quantitative; applying such an inequality gives the formal weak-law proof. The shrinking variance is the essential input, but a variance by itself is not a probability statement.
For example, saying that a sampling variance is small does not assert that a particular realized sample mean must be close to \(\mu\). It describes the distribution of the statistic over repeated samples. Nor does variance tending to zero, in isolation, establish convergence to an arbitrary target: the means must also be centered at, or approach, that target. Here \(E[\bar{X}_n]=\mu\) for every \(n\), which supplies the required center.
Worked Example: A Discrete Population and a Larger Sample
Suppose an observation \(X_i\) takes values \(0,2,4\) with probabilities \(0.25,0.50,0.25\), respectively. Assume observations are IID. Find the mean and variance of one observation, then find the variance and standard error of the sample mean for \(n=8\). State how the variance changes if the sample size is increased to 32.
Method. First calculate the population mean: \(E[X_i]=0(0.25)+2(0.50)+4(0.25)=0+1+1=2\). Next calculate the second moment: \(E[X_i^2]=0^2(0.25)+2^2(0.50)+4^2(0.25)=0+2+4=6\). Therefore \(\operatorname{Var}(X_i)=E[X_i^2]-(E[X_i])^2=6-2^2=2\).
For \(n=8\), \(\operatorname{Var}(\bar{X}_8)=2/8=0.25\), so the standard error is \(\sqrt{0.25}=0.5\). For \(n=32\), \(\operatorname{Var}(\bar{X}_{32})=2/32=0.0625\). The sample size has been multiplied by four, so the variance has been divided by four. The corresponding standard error is \(\sqrt{0.0625}=0.25\), half the earlier value.
Check. The sample mean has expected value 2 at both sample sizes. The calculated variances are \(0.25\) and \(0.0625\), and the standard errors are \(0.5\) and \(0.25\). A common error is to use \(E[X_i^2]\) as the variance; subtracting \((E[X_i])^2\) is necessary.
When the Formula Does Not Apply
Independence is not a cosmetic condition. For observations with the same variance \(\sigma^2\) but nonzero covariances, the relevant identity is \(\operatorname{Var}(\bar{X}_n)=\frac{1}{n^2}\left(n\sigma^2+2\sum_{i<j}\operatorname{Cov}(X_i,X_j)\right)\). Positive covariances add variability; the result need not shrink at the IID rate. If variances differ, independence instead gives \(\operatorname{Var}(\bar{X}_n)=\frac{1}{n^2}\sum_{i=1}^n\operatorname{Var}(X_i)\), which is not generally \(\sigma^2/n\) for one common \(\sigma^2\). Finally, if the observations do not have finite variance, the stated formula cannot be used with a finite \(\sigma^2\).
Worked Example: A Shared Source of Variation
Suppose repeated measurements follow \(X_i=10+Z+\epsilon_i\), where \(Z\sim N(0,1)\) is a shared random effect and the \(\epsilon_i\) are mutually independent \(N(0,9)\) variables, independent of \(Z\). Find the variance of the average of \(n\) measurements for \(n=9\) and \(n=81\), and compare it with the variance that would result if the measurements were independent with the same individual variance.
Method. Each measurement has mean 10 and variance \(\operatorname{Var}(Z)+\operatorname{Var}(\epsilon_i)=1+9=10\). For the average, \(\bar{X}_n=10+Z+\frac{1}{n}\sum_{i=1}^n\epsilon_i\). The shared \(Z\) remains unchanged when the measurements are averaged. The independent errors average down, giving \(\operatorname{Var}(\bar{X}_n)=1+9/n\).
At \(n=9\), the variance is \(1+9/9=2\). At \(n=81\), it is \(1+9/81=1+1/9=10/9\), approximately \(1.1111\). If the measurements were independent with individual variance 10, the IID formula would instead give \(10/9\) at \(n=9\), and \(10/81\) at \(n=81\). The shared random effect prevents the actual variance from decreasing below 1 as \(n\) grows.
Check. The covariance between distinct measurements is \(\operatorname{Var}(Z)=1\), not zero. At \(n=9\), \(10/9\) under the independence calculation is less than the correct value 2; at \(n=81\), \(10/81\) is less than the correct \(10/9\). The mistake is treating measurements that share \(Z\) as independent.
Common Exam Traps
- Dividing by the wrong quantity. For IID observations, \(\operatorname{Var}(\bar{X}_n)=\sigma^2/n\), whereas the standard error is \(\sigma/\sqrt{n}\). These are not interchangeable.
- Using the formula without checking dependence. Identical marginal distributions do not imply independence. Correlated observations require covariance terms.
- Confusing sample variance with sampling variance. The sample variance describes spread among observed \(X_i\) values. The variance of \(\bar{X}_n\) describes the spread of the sample mean across repeated samples.
- Claiming a guarantee for one sample. Variance shrinkage describes a sampling distribution and supports a probability limit; it does not ensure that every realized sample mean is close to \(\mu\).
Check Your Understanding
For each problem, identify the assumptions needed before applying a variance formula.
- Independent Bernoulli observations have success probability \(0.30\). Find the variance and standard error of the sample proportion for \(n=36\). (Answer: variance \(0.0058333\ldots\); standard error approximately \(0.0764\).)
- IID observations have mean 5 and variance 12. Find the expected value, variance, and standard error of their sample mean for \(n=48\). (Answer: mean \(5\); variance \(0.25\); standard error \(0.5\).)
- An IID population has values \(1\) and \(5\), each with probability \(0.5\). Find the population variance and the variance of the sample mean for \(n=16\). (Answer: population variance \(4\); sample-mean variance \(0.25\).)
- Independent observations have variances \(4,9,16\). Find the variance of their average. (Answer: \((4+9+16)/3^2=29/9\).)
- Measurements have the form \(X_i=7+Z+\epsilon_i\), where \(Z\) has variance \(4\), the independent errors \(\epsilon_i\) each have variance \(12\), and \(Z\) is independent of every error. Find the variance of the average for \(n=12\), and its limiting variance as \(n\) increases. (Answer: \(4+12/12=5\); limiting variance \(4\).)