Comparing Two Normal Means With A Likelihood Ratio

Why Compare the Means Through a Likelihood Ratio?

Suppose two independent groups have normal measurements, and the question is whether their population means are equal. If both groups have the same unknown variance, the likelihood ratio test compares the best fit subject to equal means with the best fit when the means are allowed to differ. The comparison is useful because it makes the effect of the estimated mean difference explicit while accounting for the common variance.

In this model, the likelihood ratio test and the usual pooled two-sample t test lead to the same rejection decisions for a two-sided test. The derivation shows why: the likelihood ratio is a decreasing function of the square of the pooled t statistic. The equivalence depends on the common-variance normal model; it is not a general identity for arbitrary samples or unequal variances.

Model and hypotheses: Let \(X_1,\ldots,X_{n_1}\) and \(Y_1,\ldots,Y_{n_2}\) be independent samples, with \(X_i\sim N(\mu_1,\sigma^2)\) and \(Y_j\sim N(\mu_2,\sigma^2)\). The variance \(\sigma^2\) is unknown but common to both populations. The test is \(H_0:\mu_1=\mu_2\) against \(H_1:\mu_1\ne\mu_2\). Assume independent observations, normal populations, and sample sizes for which the pooled residual sum of squares is positive.

Separate the Within-Group and Between-Group Variation

Write \(\bar X\) and \(\bar Y\) for the two sample means. Define the within-group sum of squares \(W\), the weighted squared difference \(B\), and the total sample size \(n\) as follows:

W=\sum_{i=1}^{n_1}(X_i-\bar X)^2+\sum_{j=1}^{n_2}(Y_j-\bar Y)^2,\qquad B=\frac{n_1n_2}{n_1+n_2}(\bar X-\bar Y)^2,\qquad n=n_1+n_2.

The unrestricted model estimates one mean for each sample. Its fitted means are \(\bar X\) and \(\bar Y\), and its residual sum of squares is \(W\). Under the null hypothesis, the two means must be the same. The fitted common mean is the overall sample mean, and the resulting residual sum of squares is \(W+B\). Thus \(B\) is the extra squared error incurred by requiring equal means.

Likelihood ratio convention: In this tutorial, \(\lambda\) is the maximized likelihood under \(H_0\) divided by the maximized likelihood under the unrestricted model. Smaller values of \(\lambda\) provide stronger evidence against \(H_0\), so a likelihood ratio test rejects for sufficiently small \(\lambda\).

Deriving the Likelihood Ratio

For a normal sample with residual sum of squares \(Q\), the likelihood as a function of the common variance, after maximizing over the fitted mean or means, is \((2\pi\sigma^2)^{-n/2}\exp[-Q/(2\sigma^2)]\). Maximizing this expression over \(\sigma^2>0\) gives the maximum likelihood estimate \(Q/n\). Substitution gives the maximized likelihood in terms of \(Q\).

\widehat{\sigma}^2=\frac{Q}{n},\qquad \sup_{\sigma^2>0}L=(2\pi)^{-n/2}\left(\frac{Q}{n}\right)^{-n/2}e^{-n/2}.

Under the unrestricted model \(Q=W\), while under the null \(Q=W+B\). The common factors cancel in their ratio. Therefore, provided \(W>0\), the likelihood ratio is:

\lambda =\frac{\sup_{H_0}L}{\sup_{\mu_1,\mu_2,\sigma^2}L} =\left(\frac{W}{W+B}\right)^{n/2} =\left(1+\frac{B}{W}\right)^{-n/2}.

This also justifies the rejection direction: \(B\geq 0\), and a larger difference between the sample means increases \(B/W\), which decreases \(\lambda\). The likelihood ratio therefore ranks data in the same order as the squared standardized difference.

Connection to the Pooled Two-Sample t Statistic

The pooled variance estimate uses the \(n-2\) residual degrees of freedom left after estimating two separate means. It is not the unrestricted maximum likelihood estimate \(W/n\). Define the pooled sample variance and pooled t statistic by:

s_p^2=\frac{W}{n-2},\qquad T=\frac{\bar X-\bar Y}{s_p\sqrt{1/n_1+1/n_2}}.

The identity \(1/n_1+1/n_2=n/(n_1n_2)\) implies \(B=(\bar X-\bar Y)^2/(1/n_1+1/n_2)\). Substituting this and \(s_p^2=W/(n-2)\) into \(T^2\) gives the key relationship:

T^2 =\frac{(\bar X-\bar Y)^2}{s_p^2(1/n_1+1/n_2)} =\frac{(n-2)B}{W}, \qquad \lambda=\left(1+\frac{T^2}{n-2}\right)^{-n/2}.

For fixed sample sizes, \(\lambda\) decreases as \(T^2\) increases. Thus a two-sided likelihood ratio test rejects for large \(|T|\). Under \(H_0\), with independent normal samples and the same variance, \(T\) has a \(t\) distribution with \(n-2\) degrees of freedom. An exact level-\(\alpha\) test rejects when \(|T|>t_{1-\alpha/2,n-2}\), where \(t_{1-\alpha/2,n-2}\) is the corresponding upper quantile. Equivalently, it rejects when \(\lambda\) is below the value obtained by substituting that critical value for \(|T|\) in the displayed relationship.

Fit the unrestricted model.
Use the separate sample means and calculate \(W\).

Fit the null model.
Use the pooled mean; its residual sum of squares is \(W+B\).

Calculate the ratio or t statistic.
Use \(\lambda=(W/(W+B))^{n/2}\), or calculate \(T\) and use its exact \(t_{n-2}\) calibration.

Make the two-sided decision.
At level \(\alpha\), reject if \(|T|>t_{1-\alpha/2,n-2}\); the equivalent likelihood ratio rule rejects for sufficiently small \(\lambda\).

Worked Applications

Worked Example: Testing Two Group Means at Five Percent

In a hypothetical study, two independent normal populations are assumed to have a common unknown variance. The sample sizes are \(n_1=4\) and \(n_2=6\), the within-group sum of squares is \(W=30\), and the difference of sample means is \(\bar X-\bar Y=2\). Calculate the likelihood ratio and pooled t statistic, then test equality of means at the 5% level. Use the two-sided critical value \(2.306\) with 8 degrees of freedom.

Method and assumptions. Use the likelihood ratio derived for independent normal samples with equal variances, and calibrate the test with the exact pooled \(t\) distribution. Here \(n=4+6=10\), so the degrees of freedom are \(10-2=8\).

Calculate \(B\). \(B=(4\cdot6/10)(2)^2=(24/10)(4)=9.6\). Thus the null residual sum of squares is \(W+B=30+9.6=39.6\).

Calculate the likelihood ratio. \(\lambda=(30/39.6)^{10/2}=(25/33)^5\). Numerically, this is approximately \(0.2495\).

Calculate the pooled t statistic. \(T^2=(10-2)(9.6)/30=76.8/30=2.56\), so \(|T|=\sqrt{2.56}=1.6000\).

Decision and check. Since \(1.6000<2.306\), do not reject equality of means at 5%. The likelihood ratio is not small, consistent with that decision. The common mistake avoided here is using \(W/n\) as the pooled variance in the t statistic; the pooled variance is \(W/(n-2)=30/8=3.75\), giving \(T=2/\sqrt{3.75(1/4+1/6)}=1.6000\). The displayed relationship also checks it: \(T^2=8B/W=2.56\).

Worked Example: A Large Mean Difference With Few Observations

Two independent normal samples have a common unknown variance, with \(n_1=3\), \(n_2=4\), \(W=12\), and \(\bar X-\bar Y=4\). Find \(B\), \(\lambda\), and \(|T|\), then test equality of means at 5%. Use the two-sided critical value \(2.571\) with 5 degrees of freedom.

Method and assumptions. Apply the same likelihood ratio and pooled t formulas. The total sample size is \(n=3+4=7\), so \(n-2=5\).

Calculate the between-group component. \(B=(3\cdot4/7)(4)^2=(12/7)(16)=192/7\), or approximately \(27.4286\). The null residual sum of squares is \(12+192/7=276/7\).

Calculate the likelihood ratio. \(\lambda=(12/(276/7))^{7/2}=(7/23)^{7/2}\). Since \(7/23\approx0.30435\), this is approximately \(0.0155\).

Calculate the pooled t statistic. \(T^2=5(192/7)/12=80/7\approx11.4286\), and \(|T|=\sqrt{80/7}\approx3.3806\).

Decision and check. Because \(3.3806>2.571\), reject equality of means at 5%. The small likelihood ratio agrees with this result. A common arithmetic trap is to round \(7/23\) too early when raising it to the power \(7/2\); retain the exact fraction until the final decimal. As a check, \(T^2=5B/W=80/7\).

Worked Example: A Difference That Does Not Reach the Critical Value

In a hypothetical comparison, two independent normal samples are assumed to have a common unknown variance. Let \(n_1=n_2=5\), \(W=40\), and \(\bar X-\bar Y=3\). Calculate the likelihood ratio and \(|T|\), and decide whether to reject equal means at 5%, using the two-sided critical value \(2.306\) with 8 degrees of freedom.

Method and assumptions. Use the pooled t calibration, with \(n=5+5=10\) and \(n-2=8\).

Calculate \(B\). \(B=(5\cdot5/10)(3)^2=(25/10)(9)=22.5\). Therefore \(W+B=40+22.5=62.5\).

Calculate the likelihood ratio. \(\lambda=(40/62.5)^5=(16/25)^5=0.64^5=0.1073741824\), or \(0.1074\) to four decimals.

Calculate the t statistic. \(T^2=8(22.5)/40=180/40=4.5\), so \(|T|=\sqrt{4.5}\approx2.1213\).

Decision and check. Since \(2.1213<2.306\), do not reject at 5%. Here the sample means differ, but the observed difference is not large enough relative to the estimated common variability to cross the specified threshold. The common mistake avoided is treating any nonzero \(B\) as grounds for rejection: a test compares the statistic with a critical value. The relation \(T^2=8B/W=4.5\) confirms the calculation.

Interpretation and Common Traps

The null model is less flexible: it must explain both samples with one mean. Its loss of fit is measured by \(B\), while \(W\) measures the within-group variation against which that loss is judged. A given difference in sample means can therefore yield different evidence depending on the sample sizes and the within-group variability.

Reversing the ratio. Here \(\lambda\) is null maximum likelihood divided by unrestricted maximum likelihood, so it is at most 1. A small value, not a large one, supports rejection.
Using the wrong variance denominator. The unrestricted variance maximum likelihood estimate is \(W/n\); the pooled variance used in \(T\) is \(W/(n-2)\). They serve different purposes.
Using a one-sided cutoff. The stated alternative is \(\mu_1\ne\mu_2\), so the exact test is two-sided and uses \(t_{1-\alpha/2,n-2}\).
Ignoring the model. The exact t calibration and the likelihood ratio relationship require independent normal samples with a common variance. If variances differ, this pooled test is not justified by the stated model.

Key takeaway: For two independent normal samples with a common unknown variance, the likelihood ratio is \(\lambda=(1+T^2/(n-2))^{-n/2}\). It is a decreasing function of \(T^2\), so the exact two-sided likelihood ratio test has the same rejection decisions as the pooled two-sample t test.

Check Your Understanding

For each problem, use the independent normal, common unknown variance model unless the question asks you to identify a model limitation. Give probabilities or likelihood ratios to four decimals where a decimal is requested.

Two samples have \(n_1=4\), \(n_2=5\), \(W=18\), and \(\bar X-\bar Y=3\). Find \(B\), \(|T|\), and \(\lambda\). (Answer: \(B=20\), \(|T|=\sqrt{70/9}\approx2.7889\), \(\lambda=(9/19)^{9/2}\approx0.0120\).)
For \(n_1=6\), \(n_2=4\), \(W=45\), and \(\bar X-\bar Y=2\), find \(B\), \(|T|\), and \(\lambda\). (Answer: \(B=48/5=9.6\), \(|T|=\sqrt{64/45}\approx1.1926\), \(\lambda=(75/91)^5\approx0.3867\).)
Two samples have \(n_1=3\), \(n_2=5\), \(W=20\), and \(\bar X-\bar Y=5\). Calculate \(|T|\) and decide at 5%, using the two-sided critical value \(2.447\) with 6 degrees of freedom. (Answer: \(B=375/8=46.875\), \(|T|=3.7500\); reject equality of means.)
For \(n_1=8\), \(n_2=7\), \(W=52\), and \(\bar X-\bar Y=2.5\), calculate \(B\) and \(|T|\), then decide at 5% using the two-sided critical value \(2.160\) with 13 degrees of freedom. (Answer: \(B=70/3\approx23.3333\), \(|T|=\sqrt{455/78}\approx2.4152\); reject equality of means.)
A study has independent groups, but the population variances are not assumed equal. Explain whether the likelihood ratio-to-pooled-t relationship derived here can be used as an exact test without changing the model. (Answer: No. The derivation assumes a common variance; with unequal variances, this pooled likelihood ratio and its exact pooled t calibration do not apply as stated.)

What You'll Learn