Chi-Square Variates From Normal Sums

From Standard Normals to Chi-Square Variates

A chi-square variate can be generated without directly inverting a chi-square cumulative distribution function. Instead, generate independent standard normal variates, square them, and add the squares. This construction is useful in simulation whenever a chi-square random quantity is needed and a standard normal generator is available.

The essential condition is independence: the number of independent standard normal draws used determines the degrees of freedom. A list of fixed numbers can illustrate the arithmetic, but it is not itself a random sample from a chi-square distribution. The distributional result concerns the procedure when its inputs are random and satisfy the required assumptions.

Definition: If $Z_1,\ldots,Z_k$ are independent random variables, each with the standard normal distribution $N(0,1)$, then $$ Q=\sum_{j=1}^{k}Z_j^2 $$ has a chi-square distribution with $k$ degrees of freedom, written $Q\sim\chi^2_k$. The number of squared independent standard normal variables, $k$, is the degrees of freedom.

For a positive integer $k$, this gives a direct generation method: obtain $k$ independent standard normal random variables, square each one, and sum. The resulting random variable is nonnegative. Its expected value is $k$, and its variance is $2k$. Those theoretical quantities are useful checks on repeated simulations, although a single simulated value need not be close to either one.

Why the Sum Has a Chi-Square Distribution

The construction can be justified using moment-generating functions. For a standard normal random variable $Z$, integration of its density gives the moment-generating function of its square:

M_{Z^2}(t)=E[e^{tZ^2}]=(1-2t)^{-1/2}, \qquad t<\frac12.

Because $Z_1,\ldots,Z_k$ are independent, the moment-generating function of their sum of squares is the product of the individual moment-generating functions:

M_Q(t)=\prod_{j=1}^{k}M_{Z_j^2}(t) =\left((1-2t)^{-1/2}\right)^k =(1-2t)^{-k/2},\qquad t<\frac12.

This is the moment-generating function of the chi-square distribution with $k$ degrees of freedom, establishing the result. Independence is what permits the product. If the normal variables are dependent, the same conclusion does not generally follow, even if each variable separately has a standard normal distribution.

The mean and variance follow from the same construction. A standard normal has $E[Z_j^2]=1$ and $\operatorname{Var}(Z_j^2)=2$. By linearity of expectation and, under independence, additivity of variance,

E[Q]=\sum_{j=1}^{k}E[Z_j^2]=k, \qquad \operatorname{Var}(Q)=\sum_{j=1}^{k}\operatorname{Var}(Z_j^2)=2k.

Generating the Required Normal Draws

If a normal generator is already available, use it to produce the required number of independent standard normal draws. For example, the Box–Muller transformation takes two independent random variables $U_1,U_2$, each uniformly distributed on $(0,1)$, and forms

Z_1=\sqrt{-2\ln U_1}\cos(2\pi U_2), \qquad Z_2=\sqrt{-2\ln U_1}\sin(2\pi U_2).

When the inputs are independent uniform random variables on $(0,1)$, the two outputs are independent standard normal random variables. Independent pairs of uniform inputs produce independent pairs of normal outputs. Thus, two outputs provide two standard normal draws, and their squared sum has a chi-square distribution with 2 degrees of freedom. In contrast, plugging fixed numerical inputs into these equations produces fixed outputs; it does not, by itself, establish a probability distribution for those outputs.

For an even number of degrees of freedom, generate exactly $k/2$ Box–Muller pairs. For an odd number, generate $(k+1)/2$ pairs and use only $k$ of the resulting normal variates. Discarding the unused output does not change the joint distribution of the $k$ retained independent standard normal draws.

Set the degrees of freedom.
For a target $\chi^2_k$ variate, plan to use exactly $k$ independent standard normal random variables.

Generate the normal draws.
Use a standard normal generator or generate Box–Muller pairs from independent uniform random inputs. If $k$ is odd, retain only $k$ of the generated outputs.

Square and add.
Compute $Q=Z_1^2+\cdots+Z_k^2$. Do not take a square root or divide by the number of draws.

Check the interpretation.
The random procedure has distribution $\chi^2_k$. A particular run gives one nonnegative realized value, not a distribution.

Worked Example: Four Independent Standard Normal Draws

A simulation produces four independent standard normal random variables. For one run, their observed values are $-1.2,\ 0.5,\ 0,\ -2$. Calculate the chi-square variate for that run and state its generating distribution. Also give the theoretical mean and variance. The values shown are a fixed realization for the arithmetic; the generating model assumes independent $N(0,1)$ random variables.

Method. Square each of the four normal draws and add the results. The number of independent standard normal draws is four, so the random variable produced by the procedure has four degrees of freedom.

Calculation.

$$ Q=(-1.2)^2+(0.5)^2+0^2+(-2)^2 =1.44+0.25+0+4 =5.69. $$

The observed outcome is $5.69$. Under the stated random-generation assumptions, $Q\sim\chi^2_4$. Its theoretical mean is $4$, and its theoretical variance is $2(4)=8$. The value $5.69$ is one outcome of that distribution; it is not itself a distribution.

Check. The sum is nonnegative, and four independent standard normal draws were squared, so the degrees of freedom are 4. A common mistake is to label the numerical outcome $5.69$ as “the chi-square distribution.” The distribution is $\chi^2_4$; $5.69$ is one realized value.

Worked Example: Box–Muller Arithmetic and the Randomness Condition

For a transparent arithmetic illustration, substitute the following fixed values into two Box–Muller pairs: $(U_1,U_2)=(e^{-1/2},0)$ and $(U_1,U_2)=(e^{-2},1/4)$. Find the four outputs and their squared sum. Then state separately what distribution results when the uniform inputs are instead independent random variables uniformly distributed on $(0,1)$.

First pair. Since $-2\ln(e^{-1/2})=1$, the radius is $\sqrt{1}=1$. The angle is $2\pi(0)=0$. Therefore the outputs are

Z_1=1\cos(0)=1,\qquad Z_2=1\sin(0)=0.

Second pair. Since $-2\ln(e^{-2})=4$, the radius is $\sqrt{4}=2$. The angle is $2\pi(1/4)=\pi/2$. Therefore the outputs are

Z_3=2\cos(\pi/2)=0,\qquad Z_4=2\sin(\pi/2)=2.

Squared sum. For these fixed inputs, the outputs are $1,0,0,2$, so the arithmetic result is

$$ 1^2+0^2+0^2+2^2=5. $$

This calculation is deterministic: it demonstrates how the transformation and squaring work for the stated constants. It does not make the fixed outputs independent standard normal random variables, and the value 5 from these constants is not a random draw from $\chi^2_4$. Separately, if the four inputs are generated as two independent pairs of independent uniform random variables on $(0,1)$, the Box–Muller outputs are four independent standard normal random variables, and their squared sum has distribution $\chi^2_4$. The distinction is between illustrating a transformation with fixed inputs and applying it to random inputs under its assumptions.

Worked Example: An Odd Number of Degrees of Freedom

A simulation requires one $\chi^2_5$ variate and generates three independent Box–Muller pairs from independent uniform random inputs. For one run, suppose the six resulting standard normal values are $0.4,-1.1,0.7,0,-0.5,1.3$. Use the first five values to calculate the outcome, explain what happens to the sixth, and identify the distribution of the random procedure.

Method. Five degrees of freedom require five independent standard normal draws. Three Box–Muller pairs provide six outputs, so use five and discard the remaining output. The assumed values are one realization from the random generation procedure.

Calculation.

$$ Q=(0.4)^2+(-1.1)^2+(0.7)^2+0^2+(-0.5)^2 =0.16+1.21+0.49+0+0.25 =2.11. $$

The sixth output, $1.3$, is not included. The first five outputs are independent standard normal random variables under the stated generation assumptions, so the resulting random variable has distribution $\chi^2_5$, with mean $5$ and variance $10$. The observed value for this run is $2.11$.

Check. There are five squared terms, so the degrees of freedom are 5, not 6. Including the sixth output would give $2.11+1.3^2=2.11+1.69=3.80$, a sum with six degrees of freedom rather than the requested five. Discarding an unused output is the correct way to handle an odd target count.

Standardize Before Squaring

The construction requires standard normal variables, not merely normal variables. If $X_j\sim N(\mu,\sigma^2)$, where $\sigma>0$, then $Z_j=(X_j-\mu)/\sigma$ has distribution $N(0,1)$. If $X_1,\ldots,X_k$ are independent with the same specified normal distribution, their standardized values remain independent, and the sum of their standardized squares has distribution $\chi^2_k$.

Worked Example: Converting Nonstandard Normal Draws

Three independent random variables have distribution $N(2,9)$, so their mean is 2 and their standard deviation is 3. In one run their observed values are $5,-1,2$. Calculate the chi-square outcome obtained by standardizing each draw first, and explain why directly squaring the original values would be incorrect.

Standardize. For each observed value $x$, compute $z=(x-2)/3$:

z_1=\frac{5-2}{3}=1,\qquad z_2=\frac{-1-2}{3}=-1,\qquad z_3=\frac{2-2}{3}=0.

Square and sum.

$$ Q=1^2+(-1)^2+0^2=2. $$

Since the original variables are independent and each has distribution $N(2,9)$, their standardized versions are independent standard normal variables. Thus the random quantity constructed by standardizing and summing three squares has distribution $\chi^2_3$; its observed value here is 2. Directly squaring the original observations gives $5^2+(-1)^2+2^2=30$, which does not follow the stated chi-square construction because those observations are not standard normal variates.

Common Mistakes and Practical Checks

The method is short, but several small interpretation errors change the result. Keep the number of terms, the distribution of each term, and the status of the inputs clear.

Using the wrong degrees of freedom. Count the independent standard normal variables actually squared. The count is not the number of Box–Muller pairs if an output is discarded.
Squaring unstandardized normal values. A normal variable with nonzero mean or variance other than 1 is not a standard normal. Standardize using its specified mean and standard deviation first.
Assuming independence without a model basis. The chi-square result requires independent standard normal inputs. Marginally standard normal variables alone are not sufficient.
Confusing fixed arithmetic with a random draw. Fixed uniform constants give fixed Box–Muller outputs. The distributional statement requires independent random uniform inputs on $(0,1)$.
Taking an unnecessary square root or average. The chi-square variate is the sum of squares, not the square root of that sum and not the average of the squares.
Expecting every outcome to equal the mean. The mean $k$ describes the distribution over repeated independent runs. A single outcome can be smaller or larger than $k$.

Key takeaway: To generate $\chi^2_k$, produce exactly $k$ independent standard normal random variables and sum their squares. The resulting random variable has $k$ degrees of freedom; the numerical result of one run is only one realized outcome.

Check Your Understanding

For each problem, distinguish the random generation assumptions from any numerical outcome provided.

Four independent standard normal draws in one run are $-0.5,1,2,0$. Calculate the sum of squares and state the generating distribution, mean, and variance. (Answer: outcome $5.25$; generating distribution $\chi^2_4$, mean $4$, variance $8$.)
A program needs a $\chi^2_7$ variate and uses Box–Muller pairs. How many pairs must it generate, how many normal outputs does it retain, and what happens to any unused output? (Answer: generate 4 pairs, retain 7 outputs, discard 1 output; the retained sum of squares has distribution $\chi^2_7$ when the uniform inputs are independent $U(0,1)$ variables.)
Three independent draws from $N(1,4)$ are observed as $3,-1,5$. Standardize them and calculate the sum of standardized squares. State its generating distribution. (Answer: standardized values $1,-1,2$; outcome $6$; generating distribution $\chi^2_3$.)
Five independent standard normal draws produce an observed squared sum of $8.4$. Identify the degrees of freedom and the theoretical mean and variance. Does the outcome have to equal the mean? (Answer: 5 degrees of freedom; mean $5$, variance $10$; no, $8.4$ is one possible outcome.)
A developer squares six independent normal variables, each with distribution $N(0,4)$, without standardizing. What transformation gives a chi-square variate, and what are its degrees of freedom? (Answer: divide each normal draw by its standard deviation $2$, then sum the six standardized squares; the result has distribution $\chi^2_6$.)

What You'll Learn