Deriving the Chi-Square Law for Sample Variance

Why the Sample Variance Has a Chi-Square Distribution

The sample variance measures how much the observations vary around their sample mean. For a normal random sample, that variability has an exact sampling distribution: a suitable scaling of the sample variance follows a chi-square distribution. The result is useful because it turns a random measure of spread into a standard distributional quantity, but the scaling and degrees of freedom must be handled carefully.

The key feature is that the sample mean is estimated from the same observations used to calculate the variance. The deviations from that estimated mean satisfy a constraint: they sum to zero. As a result, only \(n-1\) independent directions of variation remain.

Definition: Let \(X_1,\ldots,X_n\) be an independent random sample from a normal distribution \(N(\mu,\sigma^2)\), where \(\mu\) is the population mean and \(\sigma^2>0\) is the population variance. Define the sample mean and sample variance by

\bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i, \qquad S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2.

Here \(S^2\) uses the denominator \(n-1\). Under these assumptions,

\frac{(n-1)S^2}{\sigma^2}\sim\chi^2_{n-1}.

A chi-square random variable with \(k\) degrees of freedom is the sum of the squares of \(k\) independent standard normal random variables. Thus the theorem says that the scaled sample variance can be represented as a sum of \(n-1\) independent standard normal squares. It does not say that \(S^2\) itself has a chi-square distribution without scaling.

Derivation: The Lost Degree of Freedom

Write the observations as a vector and first standardize them using the population parameters. Independence and normality imply that the vector of standardized observations has a multivariate standard normal distribution. We then separate the direction corresponding to the sample mean from the directions corresponding to deviations around that mean.

Standardize the sample.
Set \(Z_i=(X_i-\mu)/\sigma\) and \(Z=(Z_1,\ldots,Z_n)^\mathsf{T}\). Since the observations are independent and each is \(N(\mu,\sigma^2)\), the components of \(Z\) are independent \(N(0,1)\) variables.

Represent deviations from the sample mean.
Let \(\mathbf{1}\) be the \(n\)-vector of ones and define \(H=I-\frac{1}{n}\mathbf{1}\mathbf{1}^\mathsf{T}\). Multiplication by \(H\) removes the component in the direction of \(\mathbf{1}\), leaving the centered deviations.

Rotate to independent normal coordinates.
Choose an orthogonal matrix \(U\) whose first column is \(\mathbf{1}/\sqrt n\). The remaining columns form an orthonormal basis perpendicular to \(\mathbf{1}\). The transformation separates the mean direction from the \(n-1\) directions of centered variation.

Identify the sum of squares.
In the rotated coordinates, the contribution from the mean direction is zero after centering. The other \(n-1\) coordinates are independent standard normal variables, so their squared sum has a chi-square distribution with \(n-1\) degrees of freedom.

Here is the algebra behind those steps. The centering matrix \(H\) is symmetric and satisfies \(H^2=H\), so it is a projection. It maps \(\mathbf{1}\) to zero and leaves every vector perpendicular to \(\mathbf{1}\) unchanged. Its rank is therefore \(n-1\). Since \(X_i-\bar{X}=\sigma(Z_i-\bar{Z})\), the centered sum of squares satisfies

\frac{1}{\sigma^2}\sum_{i=1}^{n}(X_i-\bar{X})^2 =Z^\mathsf{T}HZ.

Let \(W=U^\mathsf{T}Z\). An orthogonal transformation preserves the standard multivariate normal distribution: its density depends on \(Z\) through \(Z^\mathsf{T}Z\), which is unchanged by the transformation, and the absolute Jacobian determinant is one. Therefore, the components of \(W\) are again independent \(N(0,1)\) variables. In these rotated coordinates, the projection has diagonal form:

U^\mathsf{T}HU=\operatorname{diag}(0,1,\ldots,1), \qquad Z^\mathsf{T}HZ=\sum_{j=2}^{n}W_j^2.

The final sum contains exactly \(n-1\) independent standard normal squares, so it follows a \(\chi^2_{n-1}\) distribution. From the definition of \(S^2\),

\frac{(n-1)S^2}{\sigma^2} =\frac{1}{\sigma^2}\sum_{i=1}^{n}(X_i-\bar{X})^2 \sim\chi^2_{n-1}.

This establishes both the scaling and the degrees of freedom. The number \(n-1\) is not an arbitrary convention: centering the observations at \(\bar{X}\) removes the one direction used to estimate the mean.

Using the Result Correctly

The theorem is about the random variable \((n-1)S^2/\sigma^2\). Rearranging it gives the equivalent distributional statement

S^2\sim\frac{\sigma^2}{n-1}\chi^2_{n-1}.

This notation means that \(S^2\) has the same distribution as \(\sigma^2/(n-1)\) times a chi-square random variable with \(n-1\) degrees of freedom. It does not mean that the sample variance has \(n-1\) degrees of freedom without any scale factor.

The assumptions matter. The observations must be independent and identically distributed normal variables with a common mean and a common positive variance. The result is exact under those assumptions; normality is not merely a convenient approximation for this derivation. For nonnormal observations, the same statistic is not generally chi-square distributed exactly. Also, the variance in the denominator is the population variance \(\sigma^2\), not the sample variance.

Worked Example: Standardizing a Sample Variance

A hypothetical analyst models the amounts of a particular type of claim adjustment as an independent sample from \(N(\mu,4)\), where the variance is \(4\) squared units. Four observations are recorded: \(10,12,14,16\). Using the sample-variance definition with denominator \(n-1\), calculate the observed chi-square statistic and state its sampling distribution.

Method. Calculate the sample mean and sample variance, then apply \((n-1)S^2/\sigma^2\). The model has \(n=4\) and \(\sigma^2=4\).

First, the sample mean is

\bar{X}=\frac{10+12+14+16}{4}=\frac{52}{4}=13.

The deviations from the mean are \(-3,-1,1,3\). Their squared values sum to

$$ (-3)^2+(-1)^2+1^2+3^2=9+1+1+9=20. $$

Thus the sample variance, with denominator \(4-1=3\), is

S^2=\frac{20}{3}.

The observed statistic is

\frac{(n-1)S^2}{\sigma^2} =\frac{3(20/3)}{4} =\frac{20}{4} =5.

By the theorem, before observing the sample this statistic has a \(\chi^2_3\) distribution. For the data given, its observed value is \(5\). As a check, substituting \(S^2=20/3\) and \(\sigma^2=4\) into the statistic reproduces \(5\).

Common mistake avoided: dividing the observed sample variance by the population variance alone would give \((20/3)/4=5/3\), which is not the theorem's statistic. The required factor \(n-1=3\) gives \(5\).

Worked Example: Converting a Scaled Variance Distribution

A hypothetical study takes an independent sample of four measurements from \(N(\mu,9)\). The sample variance \(S^2\) uses denominator \(3\). State the distribution of \(S^2/3\), and then express the distribution of \(S^2\) itself.

Method. Apply the theorem with \(n=4\) and \(\sigma^2=9\):

\frac{(n-1)S^2}{\sigma^2} =\frac{3S^2}{9} =\frac{S^2}{3} \sim\chi^2_3.

Therefore \(S^2/3\) has a chi-square distribution with \(3\) degrees of freedom. Multiplying both sides of this distributional relationship by \(3\) gives

S^2\sim 3\chi^2_3.

The answer is not \(S^2/3\sim\chi^2_3/3\). The factor \(3\) in the theorem's numerator cancels with the population variance \(9\) to produce \(S^2/3\) directly. As a check, multiplying \(S^2/3\sim\chi^2_3\) by \(3\) returns \(S^2\sim3\chi^2_3\).

Common mistake avoided: scaling the chi-square variable by an extra \(1/3\) after the cancellation. Always simplify \((n-1)S^2/\sigma^2\) before rearranging.

Worked Example: Recovering the Statistic From a Residual Sum of Squares

A hypothetical portfolio review records eight independent measurements of a process, modeled as \(N(\mu,4)\). The sum of squared deviations from their sample mean is \(28\). Find \(S^2\), calculate the observed chi-square statistic, and identify its degrees of freedom.

Method. First divide the residual sum of squares by \(n-1\) to obtain \(S^2\); then scale by \((n-1)/\sigma^2\). Here \(n=8\), \(\sigma^2=4\), and the residual sum of squares is \(28\).

The sample variance is

S^2=\frac{28}{8-1}=\frac{28}{7}=4.

The observed statistic is

\frac{(n-1)S^2}{\sigma^2} =\frac{7(4)}{4} =\frac{28}{4} =7.

The random statistic has a \(\chi^2_{7}\) distribution because its degrees of freedom are \(n-1=7\). For the reported data, the statistic equals \(7\). Checking from the residual sum directly gives \(28/4=7\), the same value.

Common mistake avoided: assigning \(8\) degrees of freedom because there are eight measurements. The statistic is based on deviations from an estimated sample mean, so one degree of freedom is used in centering.

Common Exam Traps and the Role of the Assumptions

The most frequent errors are usually small algebraic or modeling errors rather than difficulty with the chi-square definition. Use the following checks before accepting a result.

Using \(n\) instead of \(n-1\). The sample variance in this theorem is defined with denominator \(n-1\). If a problem defines a different variance-like quantity, translate it explicitly before applying the result.
Scaling in the wrong direction. The chi-square statistic is \((n-1)S^2/\sigma^2\). Dividing \(S^2\) by \(\sigma^2\) alone omits the factor \(n-1\).
Confusing a distribution with an observed value. The statistic is a random variable with a chi-square distribution under the model. A particular sample produces one observed numerical value of that statistic.
Using the sample variance in the denominator. The scaling uses the population variance \(\sigma^2\). Replacing it with \(S^2\) would make the expression a constant, not the chi-square random variable in the theorem.
Ignoring normality or independence. The exact result follows from the independent normal coordinates in the derivation. A similar-looking sample from a nonnormal or dependent model does not automatically satisfy it.

The main practical value of the result is that it identifies the full sampling distribution of the normal-sample variance, not just a typical value. The degrees of freedom record how many independent residual directions remain after estimating the mean, while the factor \(\sigma^2/(n-1)\) sets the scale. Keeping those two roles separate makes the theorem easier to apply and helps catch incorrect distributional statements.

Key takeaway: For an independent sample from \(N(\mu,\sigma^2)\), with \(S^2=\sum_{i=1}^{n}(X_i-\bar{X})^2/(n-1)\), the exact result is \((n-1)S^2/\sigma^2\sim\chi^2_{n-1}\). The numerator factor, population variance, and \(n-1\) degrees of freedom must all match.

Check Your Understanding

For each problem, apply the normal-sample theorem and keep the definition of the variance-like quantity in view.

An independent sample of six observations comes from \(N(\mu,9)\), and \(S^2\) uses denominator \(n-1\). State the distribution of \(5S^2/9\). (Answer: \(\chi^2_5\))
Four independent observations come from \(N(\mu,9)\), with \(S^2\) defined using denominator \(3\). State the distribution of \(S^2/3\), and express \(S^2\) as a scaled chi-square variable. (Answer: \(S^2/3\sim\chi^2_3\); \(S^2\sim3\chi^2_3\))
An independent sample of ten observations comes from \(N(\mu,16)\). If the observed sample variance is \(24\), calculate the observed chi-square statistic and state its degrees of freedom. (Answer: \(13.5\), with \(9\) degrees of freedom)
Six independent observations come from \(N(\mu,4)\), and their squared deviations from the sample mean sum to \(20\). Find \(S^2\) and the observed chi-square statistic. (Answer: \(S^2=4\); statistic \(=5\), with \(5\) degrees of freedom)
For an independent sample of seven observations from \(N(\mu,\sigma^2)\), define \(T=\frac{1}{7}\sum_{i=1}^{7}(X_i-\bar{X})^2\). State the distribution of \(7T/\sigma^2\). (Answer: \(\chi^2_6\))

What You'll Learn