Why Study the Variability of a Sample Variance?
A sample variance is itself a random variable: a different sample from the same population will generally produce a different value. Knowing its expected value tells us whether it systematically overestimates or underestimates the population variance. Knowing its variance tells us how much the estimate fluctuates from sample to sample. These are separate properties: an estimator can be unbiased and still be highly variable.
For a normal sample, the chi-square sampling law gives both quantities directly. The key is to use the correct degrees of freedom and to distinguish the variance of the observations from the variance of the statistic \(S^2\).
Expectation: Why \(S^2\) Is Unbiased
A chi-square random variable with \(k\) degrees of freedom has expectation \(k\) and variance \(2k\). Apply these facts to \(Q=\frac{(n-1)S^2}{\sigma^2}\), which has \(k=n-1\) degrees of freedom. Since \(S^2=\frac{\sigma^2}{n-1}Q\), the expectation follows by multiplying the expectation of \(Q\) by the constant \(\frac{\sigma^2}{n-1}\).
Thus \(S^2\) is an unbiased estimator of \(\sigma^2\) under the stated normal-sample model. Unbiasedness means that the average of \(S^2\) over repeated samples equals \(\sigma^2\); it does not mean that every observed \(S^2\) is close to \(\sigma^2\), or that the probability of overestimating equals the probability of underestimating.
Worked Example: Finding the Mean and Variance for a Small Sample
A hypothetical laboratory models a measurement \(X\) as \(N(50,9)\), with measurements in the same units and \(9\) in squared units. Six independent measurements are collected. Find the expectation, variance, and standard deviation of \(S^2\).
Method. Use the normal-sample chi-square law with \(n=6\) and \(\sigma^2=9\). Its degrees of freedom are \(n-1=5\), so \(\frac{5S^2}{9}\sim\chi^2_5\).
The expectation is $$ E[S^2]=\sigma^2=9. $$
The variance formula, derived below, gives $$ \operatorname{Var}(S^2)=\frac{2\sigma^4}{n-1} =\frac{2(9^2)}{5} =\frac{162}{5} =32.4. $$ The standard deviation is therefore $$ \sqrt{\operatorname{Var}(S^2)}=\sqrt{32.4}\approx 5.6921. $$
So the estimate has expected value \(9\), but its standard deviation is about \(5.6921\) squared units. As a check, \(E[(S^2)^2]=\operatorname{Var}(S^2)+(E[S^2])^2=32.4+81=113.4\), which is consistent with the chi-square second moment. The common mistake avoided here is treating unbiasedness as if it implied small sample-to-sample variation.
Variance: Quantifying Sampling Variability
For a constant \(c\) and random variable \(Q\), \(\operatorname{Var}(cQ)=c^2\operatorname{Var}(Q)\). The square on the constant matters: \(S^2\) is \(\frac{\sigma^2}{n-1}\) times \(Q\), so its variance involves \(\sigma^4\), not \(\sigma^2\).
This also gives a direct way to compare sample sizes when the population variance is fixed. The variance decreases as \(n-1\) increases, and the standard deviation is \(\sigma^2\sqrt{\frac{2}{n-1}}\). The standard deviation of \(S^2\) is measured in squared units, just like \(S^2\) and \(\sigma^2\).
A useful dimensionless measure is the coefficient of variation of \(S^2\), its standard deviation divided by its expectation. Under this model, $$ \frac{\sqrt{\operatorname{Var}(S^2)}}{E[S^2]} =\frac{\sigma^2\sqrt{2/(n-1)}}{\sigma^2} =\sqrt{\frac{2}{n-1}}. $$ This relative variability depends only on the sample size, not on \(\mu\) or \(\sigma^2\). It decreases as the sample grows, although the decrease is proportional to \(1/\sqrt{n-1}\), not \(1/(n-1)\).
Worked Example: Comparing the Variability at a Given Sample Size
A hypothetical sensor population is modeled as \(N(12,25)\), with \(25\) in squared measurement units. For a sample of \(n=8\) independent sensor readings, calculate \(E[S^2]\), \(\operatorname{Var}(S^2)\), the standard deviation of \(S^2\), and its coefficient of variation.
Method. Apply the normal-sample formulas with \(\sigma^2=25\), \(\sigma^4=25^2=625\), and \(n-1=7\).
First, $$ E[S^2]=25. $$ Next, $$ \operatorname{Var}(S^2)=\frac{2(625)}{7} =\frac{1250}{7} \approx178.5714. $$ Thus, $$ \operatorname{SD}(S^2)=\sqrt{\frac{1250}{7}} \approx13.3631. $$ Finally, the coefficient of variation is $$ \frac{13.3631}{25}\approx0.5345, $$ or about \(53.45\%\). The estimate remains unbiased, but its standard deviation is substantial relative to its expected value.
As a check, the dimensionless formula gives \(\sqrt{2/7}\approx0.5345\), agreeing with the ratio just calculated. The common mistake avoided is reporting \(178.5714\) as the standard deviation: it is the variance of \(S^2\), so it must be square-rooted to obtain the standard deviation.
Sample Size and a Denominator Trap
The denominator \(n-1\) in \(S^2\) is tied to its definition and to the chi-square degrees of freedom. Replacing it with \(n\) defines a different statistic. For instance, let \(V_n=\frac{1}{n}\sum_{i=1}^n(X_i-\bar X)^2\). Since \(V_n=\frac{n-1}{n}S^2\), it follows under the same normal model that $$ E[V_n]=\frac{n-1}{n}\sigma^2, \qquad \operatorname{Var}(V_n)=\left(\frac{n-1}{n}\right)^2\frac{2\sigma^4}{n-1} =\frac{2(n-1)\sigma^4}{n^2}. $$ Thus \(V_n\) is biased downward for \(\sigma^2\). The formula \(\frac{2\sigma^4}{n-1}\) is for \(S^2\), not for every statistic called a sample variance.
Worked Example: Finding the Minimum Sample Size for Relative Precision
A hypothetical normally distributed process has unknown mean and known population variance \(\sigma^2=16\) squared units. How large must an independent sample be so that the coefficient of variation of \(S^2\) is at most \(0.4\)? Verify that the chosen integer sample size meets the requirement and that the next smaller size does not.
Method. Under the normal-sample model, the coefficient of variation is \(\sqrt{\frac{2}{n-1}}\). The target requires $$ \sqrt{\frac{2}{n-1}}\leq0.4. $$ Both sides are nonnegative, so squaring preserves the inequality: $$ \frac{2}{n-1}\leq0.16, \qquad n-1\geq\frac{2}{0.16}=12.5. $$ Since \(n\) must be an integer, the smallest possibility is \(n=14\).
For \(n=14\), the coefficient of variation is \(\sqrt{2/13}\approx0.3922\), which is at most \(0.4\). For \(n=13\), it is \(\sqrt{2/12}\approx0.4082\), which exceeds \(0.4\). Therefore \(14\) is the minimum sample size. The value \(\sigma^2=16\) does not affect this relative-precision calculation; it would affect the variance in squared units, which is \(\frac{2(16^2)}{13}=\frac{512}{13}\approx39.3846\) for \(n=14\).
The check at \(n=13\) prevents rounding the threshold down to an insufficient sample size. The common mistake avoided is solving for \(n\) and rounding down even though the inequality requires a minimum.
What the Formulas Do—and Do Not—Say
Unbiasedness and variability answer different questions. \(E[S^2]=\sigma^2\) describes the center of the sampling distribution. \(\operatorname{Var}(S^2)=\frac{2\sigma^4}{n-1}\) describes its spread. Neither statement says that a particular observed sample variance equals the population variance. The standard deviation helps put that spread on the same scale as \(S^2\); the coefficient of variation expresses the spread relative to the mean.
The formulas also depend on the stated model: independent observations from one normal population with finite positive variance, and the usual sample variance using denominator \(n-1\). The chi-square sampling law is what makes these exact finite-sample results available. If the data do not satisfy the normal model, the expectation and variance formulas above should not be asserted solely from the sample size and population variance.
Check Your Understanding
For each problem, use the normal-sample model and the sample variance with denominator \(n-1\).
- A hypothetical calibration population is \(N(30,12)\). For \(n=5\), find \(E[S^2]\) and \(\operatorname{Var}(S^2)\). (Answer: \(12\) and \(72\).)
- Independent readings come from \(N(8,9)\), and \(n=7\). Find the standard deviation of \(S^2\). (Answer: \(\sqrt{27}\approx5.1962\).)
- For a normal population with variance \(20\) and sample size \(11\), find \(\operatorname{Var}(S^2)\) and the coefficient of variation of \(S^2\). (Answer: \(80\) and \(\sqrt{0.2}\approx0.4472\).)
- What is the smallest integer \(n\) for which the coefficient of variation of \(S^2\) is at most \(0.4\)? (Answer: \(14\).)
- For \(n=8\) from a normal population with variance \(25\), define \(V_n=\frac{1}{n}\sum_{i=1}^n(X_i-\bar X)^2\). Find \(E[V_n]\). (Answer: \(\frac{175}{8}=21.875\).)