How Non-Normality Affects the Classical F Test

Why Normality Matters for the Classical F Test

The classical F test for comparing two population variances is exact under a specific sampling model: independent random samples from normal populations. Independence between the samples is not enough on its own. If the population shapes are non-normal, the sample variances generally do not have the chi-square distributions that produce the exact F distribution. Using the usual F critical values can then give a test whose actual significance level differs from its stated level.

The issue is calibration. A test with nominal significance level 0.05 is intended to reject a true null hypothesis about 5% of the time. If the test statistic has a different distribution from the one used to find its critical value, that rejection rate need not be 5%. The direction and size of the difference depend on the population shapes and sample sizes.

Definition: For an independent random sample $X_1,\ldots,X_n$ with finite variance, the sample variance is $$ S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2, \qquad \bar{X}=\frac{1}{n}\sum_{i=1}^{n}X_i. $$ The classical two-sample variance-ratio statistic is $S_1^2/S_2^2$. Under independent normal samples and the null hypothesis $\sigma_1^2=\sigma_2^2$, it has an exact $F$ distribution with degrees of freedom $n_1-1$ and $n_2-1$.

The Exact Result and Its Normality Condition

Suppose the first sample consists of independent observations from $N(\mu_1,\sigma_1^2)$, and the second consists of independent observations from $N(\mu_2,\sigma_2^2)$. Assume that the two samples are independent of each other. For each sample, normality gives the exact result

\frac{(n_j-1)S_j^2}{\sigma_j^2}\sim\chi^2_{n_j-1}, \qquad j=1,2.

Under the null hypothesis $H_0:\sigma_1^2=\sigma_2^2$, the ratio of these two independent chi-square variables, each divided by its degrees of freedom, gives

\frac{S_1^2}{S_2^2}\sim F_{n_1-1,n_2-1}.

Normality is what gives each scaled sample variance its chi-square law. One way to see why is to standardize a normal sample and apply an orthogonal change of coordinates: one coordinate represents the sample mean, and the remaining $n-1$ coordinates represent deviations from that mean. Those residual coordinates are independent standard normal variables, and their squared sum is chi-square with $n-1$ degrees of freedom. This argument relies on the joint normal distribution; it does not apply as stated to a general non-normal sample.

Worked Example: Identifying When the Exact F Calibration Applies

Two independent samples are taken from normal populations. The sample sizes are $n_1=8$ and $n_2=10$, and the observed sample variances are $S_1^2=12$ and $S_2^2=6$. For testing equality of population variances, identify the statistic and its null distribution.

Method. Use the exact normal-sample result. The model assumptions are independent observations within each sample, independent samples, and normal populations. Under the null, the two population variances are equal.

The observed ratio is $$ \frac{S_1^2}{S_2^2}=\frac{12}{6}=2. $$

The numerator degrees of freedom are $n_1-1=8-1=7$, and the denominator degrees of freedom are $n_2-1=10-1=9$. Thus, under the null, the statistic has an $F_{7,9}$ distribution, and the observed statistic is 2.

Check. The ratio is dimensionless, and the degrees of freedom are one fewer than the sample sizes. The exact $F$ calibration is justified here because both populations are normal. If the populations were non-normal, the computed ratio would still be 2, but its exact null distribution would not generally be $F_{7,9}$. This avoids the mistake of treating the formula for the statistic as proof that its usual reference distribution always applies.

How Population Shape Changes Sample-Variance Variability

A useful way to assess the effect of non-normality is to examine the population’s fourth central moment. Let $X_1,\ldots,X_n$ be independent and identically distributed observations with mean $\mu$, variance $\sigma^2>0$, and finite fourth central moment $\mu_4=E[(X-\mu)^4]$. Define the standardized fourth moment, or kurtosis, by $\kappa=\mu_4/\sigma^4$. For $n\ge 2$, the variance of the unbiased sample variance is

\operatorname{Var}(S^2) =\sigma^4\left(\frac{2}{n-1}+\frac{\kappa-3}{n}\right).

This formula shows that, even when two populations have the same variance, their sample variances can have different sampling variability if their kurtoses differ. For a normal population, $\kappa=3$, so the formula reduces to $2\sigma^4/(n-1)$, consistent with the chi-square result. For non-normal populations, the fourth-moment term changes the variability and the sample variance’s distribution is generally not the normal-theory chi-square distribution.

Deriving the Sample-Variance Formula

The fourth-moment formula can be justified by centering the observations. Write $Y_i=X_i-\mu$, so that $E[Y_i]=0$, $E[Y_i^2]=\sigma^2$, and $E[Y_i^4]=\mu_4$. Then

(n-1)S^2=\sum_{i=1}^{n}Y_i^2-\frac{1}{n}\left(\sum_{i=1}^{n}Y_i\right)^2.

For independent centered observations, expansion of the sums gives

E\left[\left(\sum_iY_i^2\right)^2\right] =n\mu_4+n(n-1)\sigma^4,

E\left[\left(\sum_iY_i^2\right)\left(\sum_iY_i\right)^2\right] =n\mu_4+n(n-1)\sigma^4,

and

E\left[\left(\sum_iY_i\right)^4\right] =n\mu_4+3n(n-1)\sigma^4.

Substituting these expectations into the square of the expression for $(n-1)S^2$, and using $E[S^2]=\sigma^2$, yields

\operatorname{Var}(S^2) =\frac{1}{n}\left(\mu_4-\frac{n-3}{n-1}\sigma^4\right) =\sigma^4\left(\frac{2}{n-1}+\frac{\kappa-3}{n}\right).

The algebra requires a finite fourth moment. If that moment does not exist, this variance formula does not apply; in particular, it cannot be used to infer that the sample variance has well-behaved variability.

Worked Example: Comparing Sample-Variance Variability

A population is uniform on $[-\sqrt{3},\sqrt{3}]$, and a random sample of size $n=5$ is drawn. Compare the variance of $S^2$ with the variance it would have for a normal population with the same population variance. Assume independent observations.

Method. Calculate the population variance and kurtosis, then use the finite-fourth-moment formula. For a uniform distribution on $[-a,a]$, the variance is $a^2/3$ and the fourth central moment is $a^4/5$. Here $a=\sqrt{3}$, so $$ \sigma^2=\frac{(\sqrt{3})^2}{3}=1, \qquad \mu_4=\frac{(\sqrt{3})^4}{5}=\frac{9}{5}=1.8. $$ Thus $\kappa=\mu_4/\sigma^4=1.8$.

For the uniform population, $$ \operatorname{Var}(S^2) =1^2\left(\frac{2}{5-1}+\frac{1.8-3}{5}\right) =0.5-0.24=0.26. $$

For a normal population with the same variance $\sigma^2=1$, $\kappa=3$, giving $$ \operatorname{Var}(S^2)=1^2\frac{2}{5-1}=0.5. $$

Check. Both sample variances have expectation 1, but the uniform population’s sample variance has lower variance: $0.26$ rather than $0.5$. The calculation avoids assuming that equal population variances imply equal sampling distributions for the sample variances. No interest-rate or mortality assumptions are involved in this statistical model.

Consequences for Significance Levels

A test’s actual significance level is the probability that it rejects the null hypothesis when the null is true. If normal-theory $F$ critical values are used with non-normal populations, that probability need not equal the nominal level. A useful large-sample guide comes from the sample-variance formula: when $n$ is large,

\operatorname{Var}(S^2)\ \approx\ \frac{(\kappa-1)\sigma^4}{n}.

For independent samples with equal population variances under the null, a first-order approximation for the variance of the log ratio is

\operatorname{Var}\left(\log\frac{S_1^2}{S_2^2}\right) \ \approx\ \frac{\kappa_1-1}{n_1}+\frac{\kappa_2-1}{n_2}.

Under normality, both kurtoses are 3, and this becomes $2/n_1+2/n_2$. If the true log-ratio variability is larger than its normal-theory value, normal-theory critical values may be crossed too often, making the test liberal. If the true variability is smaller, the test may be conservative. This is an approximation for large samples, not a replacement exact $F$ law.

Worked Example: Approximate Significance-Level Distortion

Consider a two-sided comparison of two independent samples of size $n_1=n_2=100$, with equal population variances under the null. Suppose a large-sample normal-theory calibration uses the nominal 5% critical limits for the log variance ratio. Compare its approximate actual rejection probability in two cases: both populations are uniform with kurtosis 1.8; or both have kurtosis 6. Assume the samples have finite fourth moments.

Method. The nominal two-sided 5% limits are approximately $$ \pm 1.96\sqrt{\frac{2}{100}+\frac{2}{100}} =\pm1.96(0.2)=\pm0.392. $$ Use the approximate log-ratio variance under each pair of population shapes.

Uniform populations. The approximate variance is $$ \frac{1.8-1}{100}+\frac{1.8-1}{100} =\frac{1.6}{100}=0.016, $$ so the approximate standard deviation is $\sqrt{0.016}=0.1265$. The standardized distance to either limit is $0.392/0.1265\approx3.10$. Using a standard normal tail probability, the two-sided rejection probability is approximately $2P(Z>3.10)=0.0019$, where $Z\sim N(0,1)$. This is below the nominal 0.05.

Populations with kurtosis 6. The approximate variance is $$ \frac{6-1}{100}+\frac{6-1}{100} =\frac{10}{100}=0.10, $$ and the approximate standard deviation is $\sqrt{0.10}=0.3162$. The standardized distance is $0.392/0.3162\approx1.24$, giving an approximate two-sided rejection probability of $2P(Z>1.24)\approx0.215$. This is well above 0.05.

Check. These are large-sample approximations, not exact probabilities for finite samples. They illustrate that the same normal-theory limits can be conservative for one shape and liberal for another. A distribution with kurtosis 6 and variance 1 is possible; for example, a mixture that is zero with probability $1/2$ and otherwise $N(0,2)$ has fourth central moment 6. The method avoids presenting an approximate calculation as an exact $F$-test result.

What to Conclude in an Exam Setting

The exact classical F test is a normal-theory procedure. Non-normality does not change how to compute the observed ratio $S_1^2/S_2^2$, but it removes the general justification for comparing that ratio with the exact $F$ distribution. In particular, independence of the two samples does not restore the missing chi-square distributions.

State the model. The exact reference distribution requires independent random samples from normal populations.
Distinguish exact from approximate. A large-sample approximation may be informative, but it is not the exact finite-sample F result.
Do not predict the direction from non-normality alone. The fourth moment helps describe sample-variance variability, but the effect on a test depends on the population shapes and sample sizes.
Do not infer normality from equal variances. Equal population variances do not imply that the ratio of sample variances has the normal-theory F distribution.

For actuarial and other applied analyses, this distinction matters whenever a conclusion depends on a variance comparison. The F calculation may be mechanically correct while the stated significance level is not well calibrated under the actual sampling model. Recognizing that limitation is aligned with the SOA VEE Mathematical Statistics syllabus.

Key takeaway: The exact F law for a ratio of sample variances follows from normal-sample chi-square distributions. Non-normality can change sample-variance variability and distort the test’s actual significance level, so normal-theory F critical values are not automatically exact.

Check Your Understanding

For each problem, distinguish the exact normal-theory result from conclusions that require an approximation.

An independent sample of size 8 comes from a population with variance $\sigma^2$ and kurtosis $\kappa=1.5$. Using the finite-fourth-moment formula, find $\operatorname{Var}(S^2)/\sigma^4$. (Answer: $2/7-1.5/8=0.0982$, to four decimal places.)
A population has variance 4, kurtosis 4.5, and a sample of size 10 is drawn independently. Find $\operatorname{Var}(S^2)$. (Answer: $16[2/9+1.5/10]=16(0.3722)=5.9556$, approximately.)
For a large sample from a population with kurtosis 8, compare the approximate variance of $S^2$ with the normal-theory approximation, holding $n$ and $\sigma^2$ fixed. (Answer: the ratio is $(8-1)/(3-1)=3.5$; the non-normal approximation is 3.5 times the normal-theory value.)
Two independent samples each have size 80. Under the null of equal variances, suppose their kurtoses are 1.8 and 4. Compute the approximate variance of the log sample-variance ratio and compare it with the normal-theory value. (Answer: $(0.8+3)/80=0.0475$, compared with $4/80=0.0500$.)
Two independent samples have sizes 12 and 15 from populations with equal variances, but the populations are not known to be normal. Is their sample-variance ratio exactly $F_{11,14}$ under the null? Explain the missing condition. (Answer: No. The exact $F_{11,14}$ result requires normal populations as well as independent samples; non-normality generally prevents the scaled sample variances from having the required chi-square distributions.)

What You'll Learn