The Chi-Square Distribution as a Sum of Normal Squares

Why Squared Normal Variables Matter

In normal-sample problems, deviations from a mean are often standardized and squared. Squaring removes the sign of a deviation, while standardizing puts observations on a common scale. When the standardized quantities are independent standard normal variables, their sum of squares has a particular distribution: the chi-square distribution. This distribution is a building block for calculations involving normal samples.

The key to recognizing a chi-square variable is not merely seeing squares in a formula. One must check that the terms being squared are independent standard normal variables. The number of such terms determines the degrees of freedom.

Definition: Let $Z_1,\ldots,Z_k$ be independent random variables, each with the standard normal distribution $N(0,1)$, where $k$ is a positive integer. Then $Q=\sum_{j=1}^{k} Z_j^2$ has a chi-square distribution with $k$ degrees of freedom, written $Q\sim\chi^2_k$. The integer $k$ is the degrees of freedom.

A chi-square random variable is therefore a nonnegative sum of squared random quantities. Its possible values lie in $[0,\infty)$, and it is positive with probability 1. The subscript in $\chi^2_k$ records how many independent standard normal squares are summed; it is not the value of the statistic.

Recognizing the Distribution and Its Support

To identify a chi-square distribution from a formula, work from the variables being squared. Each term must be standard normal, and the terms must be mutually independent. If those conditions hold for $k$ terms, their sum of squares has a chi-square distribution with $k$ degrees of freedom.

For example, if $Z_1,Z_2,Z_3$ are independent standard normal variables, then $Z_1^2+Z_2^2+Z_3^2\sim\chi^2_3$. The sum cannot be negative because each square is nonnegative. It can equal zero only when all three variables equal zero; since a continuous normal variable has probability zero of taking any one specified value, the sum equals zero with probability zero.

The value observed in a particular sample is a realization of the statistic, not the distribution itself. If an observed set of standardized values gives a sum of squares equal to 5, the observed statistic is 5; whether the statistic has a chi-square distribution, and its degrees of freedom, are determined by the model and construction.

Worked Example: Counting Independent Standard Normal Squares

Let $Z_1,Z_2,Z_3,Z_4$ be independent standard normal variables. In one realization, their observed values are $0.5,-1,1.5,0$. Find the distribution of $Q=Z_1^2+Z_2^2+Z_3^2+Z_4^2$, its support, and the observed value of $Q$.

Method and assumptions. Count the independent standard normal variables in the definition. There are four, so $Q\sim\chi^2_4$. This conclusion depends on both standard normality and independence.

Compute the observed statistic. Square each observed value and add: $0.5^2=0.25$, $(-1)^2=1$, $1.5^2=2.25$, and $0^2=0$. Thus $Q=0.25+1+2.25+0=3.5$.

Support and check. The statistic is nonnegative, so its support is $[0,\infty)$. The observed value, 3.5, is nonnegative and is one possible realization of a $\chi^2_4$ variable. The common mistake to avoid is treating 3.5 as the degrees of freedom; the degrees of freedom are 4, the number of independent squared terms.

From Normal Observations to Chi-Square Statistics

Suppose $X_1,\ldots,X_n$ are independent observations from a normal population with known mean $\mu$ and known standard deviation $\sigma>0$. Standardize each observation using $Z_i=(X_i-\mu)/\sigma$. Each $Z_i$ is standard normal, and independence is preserved because each observation is transformed separately. Consequently, $\sum_{i=1}^{n}((X_i-\mu)/\sigma)^2\sim\chi^2_n$. This is an exact result under the stated normal model; it is not an approximation based on a large sample.

Theorem: If $X_1,\ldots,X_n$ are independent and each has distribution $N(\mu,\sigma^2)$, with $\mu$ and $\sigma>0$ known, then $$ \sum_{i=1}^{n}\left(\frac{X_i-\mu}{\sigma}\right)^2\sim\chi^2_n. $$ The degrees of freedom are $n$, because the sum contains $n$ independent standard normal squares.

The reason this formula holds follows directly from the definition. Under the model, each standardized variable $Z_i=(X_i-\mu)/\sigma$ has distribution $N(0,1)$. Since the observations are independent, $Z_1,\ldots,Z_n$ are independent. The displayed statistic is exactly $\sum_{i=1}^{n}Z_i^2$, which is the defining construction of a $\chi^2_n$ variable.

Worked Example: Standardizing Observations With a Known Mean

A hypothetical instrument produces independent normal readings with mean $\mu=10$ units and standard deviation $\sigma=2$ units. Four readings are $12,8,10,14$. Find the observed value of the standardized sum of squares and state the distribution of that statistic under the model.

Method and assumptions. The readings are independent and normally distributed with the stated mean and standard deviation, both treated as known. Standardize each observation by subtracting 10 and dividing by 2.

For the readings $12,8,10,14$, the standardized values are $(12-10)/2=1$, $(8-10)/2=-1$, $(10-10)/2=0$, and $(14-10)/2=2$.

Square and add: $1^2+(-1)^2+0^2+2^2=1+1+0+4=6$. Thus the observed statistic is 6. Under the specified model, the statistic has distribution $\chi^2_4$, because it is the sum of four independent standard normal squares.

Check and common mistake. The squared terms are nonnegative and sum to 6, as required. The readings are four units apart in some cases, but the standardization denominator is the standard deviation 2, not the variance 4. Dividing by 4 would not produce standard normal variables and would not give the stated chi-square construction.

When the Mean Is Estimated From the Sample

A different statistic arises when the population mean is not known and is estimated by the sample mean. For a sample $X_1,\ldots,X_n$, define $\bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i$ and $S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$. If the observations are independent and identically distributed as $N(\mu,\sigma^2)$, with $n\geq2$ and $\sigma>0$, then $(n-1)S^2/\sigma^2\sim\chi^2_{n-1}$.

Theorem: For independent observations $X_1,\ldots,X_n\sim N(\mu,\sigma^2)$, the sample variance defined with denominator $n-1$ satisfies $$ \frac{(n-1)S^2}{\sigma^2} = \frac{\sum_{i=1}^{n}(X_i-\bar{X})^2}{\sigma^2} \sim\chi^2_{n-1}. $$ This exact result requires a normal population and independent observations. Estimating the mean from the sample leaves $n-1$ degrees of freedom.

The algebraic equality follows from the definition of $S^2$: multiplying both sides of $S^2=\sum_{i=1}^{n}(X_i-\bar{X})^2/(n-1)$ by $(n-1)/\sigma^2$ gives the displayed ratio. The degrees of freedom are $n-1$, rather than $n$, because the residuals $X_i-\bar{X}$ sum to zero. They cannot all vary independently: once $n-1$ residuals are specified, the last is fixed by their sum. Under the normal model, the corresponding standardized residual sum of squares has the stated chi-square distribution.

This distinction matters. When the population mean is known, the deviations are taken from that fixed mean and the statistic has $n$ degrees of freedom. When the mean is estimated from the same observations, the deviations are taken from $\bar{X}$ and the sample-variance statistic has $n-1$ degrees of freedom.

Worked Example: A Sample Variance From a Normal Population

A hypothetical sample of four independent observations comes from $N(\mu,\sigma^2)$, where the mean is unknown and the population variance is $\sigma^2=4$. The observed values are $2,4,6,8$. Calculate the sample variance and the observed value of $(n-1)S^2/\sigma^2$, then identify its distribution under the model.

Method and assumptions. Use the sample-variance chi-square result for an independent normal sample. Here $n=4$, the denominator in $S^2$ is $n-1=3$, and the population variance is 4.

First calculate the sample mean: $\bar{X}=(2+4+6+8)/4=20/4=5$. The deviations from the sample mean are $-3,-1,1,3$, so their squared values are $9,1,1,9$. Their sum is $9+1+1+9=20$.

Therefore $S^2=20/3$. The observed chi-square statistic is $(n-1)S^2/\sigma^2=3(20/3)/4=20/4=5$. Under the stated model, $(n-1)S^2/\sigma^2\sim\chi^2_3$, because the degrees of freedom are $n-1=3$.

Check and common mistake. The deviations sum to $-3-1+1+3=0$, as they must when measured from the sample mean. The statistic is $5S^2/4$, not $S^2/4$: omitting the factor $n-1=3$ gives the wrong statistic. The observed value is 5, while the distribution has 3 degrees of freedom.

Assumptions and Exam Traps

A sum of squares is not automatically chi-square. For the defining result, the squared terms must be independent standard normal variables. In normal-sample applications, standardization and the sample-variance theorem supply those conditions, but only when their model assumptions hold.

Do not count observations without checking the construction. The known-mean statistic has $n$ degrees of freedom; the sample-variance statistic with an estimated mean has $n-1$.
Do not confuse variance and standard deviation. Standardization divides by $\sigma$, while the sample-variance ratio divides by $\sigma^2$.
Do not assume independence from normality alone. The defining sum requires independent standard normal terms. Correlated normal variables do not automatically produce the same chi-square distribution by simply counting their squares.
Do not apply the exact sample-variance result to every population. For a nonnormal population, the stated chi-square law for $(n-1)S^2/\sigma^2$ is generally not exact.
Do not mistake a statistic's observed value for its degrees of freedom. The observed value comes from the data; the degrees of freedom come from the number of independent standard normal components in the construction.

The chi-square distribution is important because it translates squared standardized normal variation into a named sampling distribution. In normal-sample calculations, recognizing whether the mean is fixed or estimated identifies the right degrees of freedom and prevents a common source of incorrect conclusions. These results are aligned with the SOA VEE Mathematical Statistics syllabus.

Check Your Understanding

For each problem, identify the relevant construction before calculating. Answers are included for checking.

Three independent standard normal variables have observed values $0,2,-1$. What are the degrees of freedom and observed sum of squares? (Answer: $\chi^2_3$; observed value $5$.)
Independent normal readings have known mean 40 and standard deviation 5. Four readings are $45,35,40,50$. Calculate the standardized sum of squares and its degrees of freedom. (Answer: observed value $6$; 4 degrees of freedom, so the statistic has distribution $\chi^2_4$.)
A sample of four independent observations from a normal population is $2,4,6,8$, and the population variance is 4. Find $S^2$ and the observed value of $(n-1)S^2/\sigma^2$. (Answer: $S^2=20/3$; observed value $5$, with distribution $\chi^2_3$.)
A statistic is the sum of squares of five independent standard normal variables. State its support and degrees of freedom. (Answer: support $[0,\infty)$; 5 degrees of freedom, so it has distribution $\chi^2_5$.)
Six independent observations come from $N(\mu,\sigma^2)$, with unknown $\mu$. Write the chi-square statistic formed from the sample variance and state its degrees of freedom. (Answer: $5S^2/\sigma^2\sim\chi^2_5$.)
Four independent observations come from a uniform population rather than a normal population. Is $(n-1)S^2/\sigma^2$ guaranteed to have a chi-square distribution? (Answer: No; the exact sample-variance chi-square result requires a normal population.)

What You'll Learn