The Likelihood-Ratio G-Test for Categorical Fit

Why Use a Likelihood-Ratio Goodness-of-Fit Test?

A one-way goodness-of-fit test asks whether observed counts across categories are consistent with a specified probability distribution. Pearson’s chi-square statistic measures discrepancies by squaring the differences between observed and expected counts. The likelihood-ratio G-test measures the same general mismatch in a different way: it compares how well the proposed probability model fits the data with how well a model tailored to the observed counts fits.

The G-test is an alternative to Pearson’s test, not a different research question. With adequate expected counts, both statistics are commonly compared with a chi-square distribution. Their values, and therefore their approximate p-values, can differ slightly. A G-test result should still be checked against the study design and the expected-count conditions for the approximation.

Definition: Suppose a sample of size $n$ produces counts $O_1,\ldots,O_k$ in $k$ mutually exclusive and exhaustive categories. Under a null model with category probabilities $p_1,\ldots,p_k$, the expected counts are $E_i=np_i$. The likelihood-ratio statistic is $$ G^2=2\sum_{i=1}^{k} O_i\ln\left(\frac{O_i}{E_i}\right), $$ where a term with $O_i=0$ is defined to be zero. The statistic is compared with a chi-square distribution under the null model when the chi-square approximation is appropriate.

Here $G^2$ is the conventional name for the statistic; it is not a direction to square a separately calculated value of $G$. The logarithm is the natural logarithm. Each category’s contribution can be positive or negative, but the total $G^2$ is nonnegative. Categories with observed counts above expectation contribute positively; categories below expectation contribute negatively. Their contributions combine to give a measure of overall discrepancy.

Conditions and Degrees of Freedom

The sampling model is multinomial: there are $n$ independent observations, each observation falls into exactly one of the stated categories, and the null model assigns the stated probabilities to those categories. The total $n$ is fixed, and the category counts sum to $n$. A category with probability zero under the null cannot produce an observation under that model; it is a structural zero, not an ordinary category to which the formula should be applied.

For a fixed null probability vector, the usual degrees of freedom are $k-1$. The counts have one total constraint, $\sum O_i=n$, so only $k-1$ counts can vary freely. If the null model estimates $r$ independent parameters from the same data, the usual degrees of freedom are $k-1-r$, provided the model is regular and the parameters are identifiable. The expected counts must be calculated using the fitted probabilities in that case.

The chi-square approximation is most dependable when expected counts are not sparse. A common guideline is that expected counts should generally be at least 5, though suitability depends on the setting and the approximation being used. When counts are too sparse, an exact or simulation-based approach may be preferable. The large-sample chi-square reference distribution does not make a sparse-data problem disappear.

Worked Example: A Three-Category Distribution

Suppose 120 hypothetical, independent selections are classified into three categories. A proposed model assigns probabilities 0.50, 0.30, and 0.20; the observed counts are 66, 30, and 24. Test the fit using $G^2$, and compare it with Pearson’s statistic.

The expected counts under the proposed model are $120(0.50)=60$, $120(0.30)=36$, and $120(0.20)=24$. They sum to 120, as do the observed counts. All are at least 5, and the setup specifies independent selections into three exhaustive categories.

\begin{aligned} G^2 &=2\left[66\ln\left(\frac{66}{60}\right) +30\ln\left(\frac{30}{36}\right) +24\ln\left(\frac{24}{24}\right)\right]\\ &=2\left[6.29047-5.46965+0\right]\\ &\approx 1.64165. \end{aligned}

The null probabilities are fixed, so $df=3-1=2$. The chi-square right-tail p-value is approximately 0.4401. At significance level $\alpha=0.05$, the result is not statistically significant: these counts do not provide sufficient evidence that the category probabilities differ from the proposed vector.

For comparison, Pearson’s statistic is $$ X^2=\frac{(66-60)^2}{60}+\frac{(30-36)^2}{36}+\frac{(24-24)^2}{24} =0.600+1.000+0=1.600. $$ The two statistics are close here, but they are calculated differently. Neither result establishes that the proposed probabilities are exactly true.

Why the Statistic Has This Form

The likelihood-ratio formula follows from comparing the null model with the most flexible categorical model. The null model uses its specified category probabilities. The saturated model allows each category probability to match its observed sample proportion, $O_i/n$. Since both models describe the same multinomial sample, their likelihoods contain the same multinomial coefficient, which cancels in their ratio.

For positive observed counts, the null likelihood divided by the saturated likelihood is $$ \frac{L_0}{L_{\text{sat}}} =\prod_{i=1}^{k}\left(\frac{p_i}{O_i/n}\right)^{O_i} =\prod_{i=1}^{k}\left(\frac{E_i}{O_i}\right)^{O_i}, $$ because $E_i=np_i$. Taking the natural logarithm and multiplying by $-2$ gives $$ -2\ln\left(\frac{L_0}{L_{\text{sat}}}\right) =-2\sum_{i=1}^{k}O_i\ln\left(\frac{E_i}{O_i}\right) =2\sum_{i=1}^{k}O_i\ln\left(\frac{O_i}{E_i}\right)=G^2. $$

If $O_i=0$, the corresponding likelihood factor has exponent zero and contributes nothing to the log-likelihood. Equivalently, the limiting value of $O_i\ln(O_i/E_i)$ as $O_i$ approaches zero is zero for $E_i>0$. This justifies the zero-count convention in the definition.

Under the null model and suitable large-sample conditions, this likelihood-ratio statistic has an approximate chi-square distribution. For a fixed probability vector, the saturated model has $k-1$ free probabilities, while the null model has none; the difference gives $k-1$ degrees of freedom. When parameters are estimated under the null, the fitted null model has additional free parameters, reducing the difference in dimensions.

Worked Calculation and Comparison

Worked Example: Four Equally Likely Categories

In a hypothetical process, 100 independent items are classified into one of four categories. The proposed distribution makes the categories equally likely. The observed counts are 40, 30, 20, and 10. Calculate $G^2$ and compare its conclusion with Pearson’s test.

Under the null model each expected count is $100(0.25)=25$. The categories are exhaustive, the counts total 100, and each expected count is at least 5. The null probabilities are fixed, so $df=4-1=3$.

\begin{aligned} G^2 &=2\left[ 40\ln\left(\frac{40}{25}\right) +30\ln\left(\frac{30}{25}\right) +20\ln\left(\frac{20}{25}\right) +10\ln\left(\frac{10}{25}\right)\right]\\ &=2[18.80015+5.46965-4.46287-9.16291]\\ &\approx 21.2880. \end{aligned}

The chi-square right-tail p-value with 3 degrees of freedom is approximately 0.0000917. At $\alpha=0.05$, this is strong evidence against equal category probabilities. It does not show that any particular alternative distribution is correct.

Pearson’s statistic for the same counts is $$ X^2=\frac{15^2}{25}+\frac{5^2}{25}+\frac{(-5)^2}{25}+\frac{(-15)^2}{25} =9+1+1+9=20. $$ Its chi-square p-value with 3 degrees of freedom is about 0.00017. The two approximate p-values are not identical, but both lead to rejection of the equal-probability model at the 0.05 level.

Pearson’s statistic weights squared count differences by their expected counts, $X^2=\sum (O_i-E_i)^2/E_i$. The G-test instead uses logarithms of observed-to-expected ratios. Near a good fit, write $O_i=E_i(1+\delta_i)$ and expand the logarithm to second order, $\ln(1+\delta)\approx\delta-\delta^2/2$; because $\sum E_i\delta_i=0$, the linear terms cancel and $G^2\approx\sum E_i\delta_i^2=X^2$, so the leading terms of the two statistics agree. For this reason they often give similar results in large samples, while differences can be more noticeable for larger discrepancies or sparse counts. They are not interchangeable formulas.

Reading One-Way Software Output

Software may label the result “G-test,” “likelihood-ratio chi-square,” or “deviance.” Check that the output is for a one-way categorical fit and identify the observed counts, expected counts, degrees of freedom, and p-value. Some programs also report Pearson’s statistic. The reported chi-square p-value for $G^2$ is an approximation based on the stated degrees of freedom, not an exact probability calculated from the likelihood ratio itself.

Worked Example: Interpreting a Software Result

A hypothetical quality-control sample contains 200 independent items in three mutually exclusive categories. The model probabilities are 0.40, 0.35, and 0.25; observed counts are 90, 70, and 40. Suppose software reports “Likelihood-ratio chi-square = 3.3495, df = 2, p = 0.1874.” Verify the statistic and interpret the output.

The expected counts are $200(0.40)=80$, $200(0.35)=70$, and $200(0.25)=50$. All exceed 5, the counts total 200, and the stated sampling setup provides independent categorical observations. The null probability vector is fixed, giving $df=3-1=2$.

\begin{aligned} G^2 &=2\left[90\ln\left(\frac{90}{80}\right) +70\ln\left(\frac{70}{70}\right) +40\ln\left(\frac{40}{50}\right)\right]\\ &=2[10.60047+0-8.92574]\\ &\approx 3.3495. \end{aligned}

The calculation agrees with the reported statistic. Since $p=0.1874$ is greater than 0.05, the sample does not provide sufficient evidence at the 5% significance level against the proposed probability vector. This is not proof that the model is true; it means the observed discrepancy is not unusual enough under the chi-square approximation to reject the model at that threshold.

Pearson’s statistic is $100/80+0+100/50=3.2500$, close to $G^2$ for these data. Its closeness is informative as a comparison, but the software’s likelihood-ratio p-value must be interpreted using its own statistic and degrees of freedom.

Common Mistakes and Practical Checks

Using the wrong ratio. The logarithm is $\ln(O_i/E_i)$, with the observed count in the numerator. Reversing the ratio changes the sign of the log-likelihood expression.
Giving a zero-count term a numerical value. If $O_i=0$ and $E_i>0$, its contribution is defined as zero, not as a calculation involving $\ln(0)$.
Using Pearson’s formula for the G-test. The formulas are different. Calculate the statistic requested, even when both tests have similar conclusions.
Using $k$ degrees of freedom. For a fixed one-way probability vector, use $k-1$. Account for parameters estimated from the data when the null model includes them.
Treating a large p-value as confirmation. A large p-value is not the probability that the null model is true and does not prove exact agreement. It indicates that the data do not provide strong evidence against the null model.

A sound report names the categories or distribution being tested, gives $G^2$, the degrees of freedom, and the approximate p-value, and states the conclusion in context. Before relying on that conclusion, verify the multinomial sampling setup, the expected counts, and whether the probabilities were fixed or estimated.

Check Your Understanding

Answer each question using the definitions and reasoning in this tutorial.

What observed-to-expected ratio appears inside the logarithm in the G-test statistic?
A one-way test has five categories and a fixed probability vector. What are its usual degrees of freedom?
If a category has observed count zero and positive expected count, what is its contribution to $G^2$, and why?
Why might the G-test and Pearson’s test produce different numerical statistics for the same counts?
A software output reports $G^2=2.8$, $df=2$, and $p=0.247$. What can and cannot be concluded at $\alpha=0.05$?