Likelihood Ratio Goodness Of Fit For Multinomial Data

Why Use a Likelihood Ratio Goodness-of-Fit Test?

A multinomial goodness-of-fit test asks whether observed counts across a set of mutually exclusive categories are consistent with a specified probability model. The likelihood ratio approach compares the best fit allowed by the null model with the unrestricted fit that assigns a separate probability to each category. The resulting statistic measures how much fit is lost by imposing the null model.

This test is useful when a model specifies category probabilities directly or through a small number of parameters. Its degrees of freedom depend on how many parameters are estimated from the same data. That adjustment is essential: using the number of categories minus one in every case can give the wrong reference distribution.

Definition: Suppose \(n\) independent observations each fall into exactly one of \(k\) categories, with probabilities \(p_1,\ldots,p_k\), where \(p_i>0\) and \(\sum_{i=1}^k p_i=1\). The count vector \((N_1,\ldots,N_k)\) has a multinomial distribution with sample size \(n\) and probabilities \((p_1,\ldots,p_k)\). The observed count in category \(i\) is \(O_i\), so \(\sum_{i=1}^k O_i=n\).

The null hypothesis specifies a probability vector, or a model for that vector. The expected count in category \(i\) under the null is \(E_i=n p_i\) when the null probabilities are fixed. If the null contains unknown parameters estimated from the data, use the fitted null probabilities instead: \(E_i=n\widehat p_i\). In either case, the expected counts must be based on the null fit, not on the unrestricted fit.

Deriving the Multinomial Likelihood Ratio Statistic

For a multinomial count vector, the likelihood at probabilities \(p_1,\ldots,p_k\) is

L(p_1,\ldots,p_k) =\frac{n!}{O_1!\cdots O_k!}\prod_{i=1}^k p_i^{O_i}.

The saturated model allows any probability vector and therefore fits each observed proportion exactly. Its maximum likelihood estimates are \(\widehat p_i=O_i/n\). Under a fixed null probability vector \((p_1^0,\ldots,p_k^0)\), the likelihood ratio is the null likelihood divided by the saturated likelihood. The common multinomial coefficient cancels.

\begin{aligned} \Lambda &=\frac{L(p_1^0,\ldots,p_k^0)} {L(\widehat p_1,\ldots,\widehat p_k)}\\ &=\prod_{i=1}^k \left(\frac{p_i^0}{O_i/n}\right)^{O_i}. \end{aligned}

By convention, a category with \(O_i=0\) contributes a factor of one to the likelihood product, since \(p_i^0{}^{\,0}=1\). Taking negative twice the logarithm gives the likelihood ratio goodness-of-fit statistic, also called the multinomial deviance:

G^2=-2\log\Lambda =2\sum_{i=1}^k O_i\log\left(\frac{O_i}{n p_i^0}\right) =2\sum_{i=1}^k O_i\log\left(\frac{O_i}{E_i}\right).

Here and throughout, \(\log\) is the natural logarithm. A zero observed count contributes zero to the sum, interpreted as the limit \(O_i\log(O_i/E_i)\to 0\) as \(O_i\) decreases to zero. Expected counts in the formula are positive under the regular null model.

Derivation for a composite null: If the null probabilities depend on \(r\) unknown parameters, maximize the likelihood over those parameters subject to the null model. Let the resulting fitted probabilities be \(\widehat p_i^{\,0}\). The likelihood ratio is the maximized null likelihood divided by the saturated likelihood, so cancellation and taking \(-2\log\) give \(G^2=2\sum_i O_i\log\{O_i/(n\widehat p_i^{\,0})\}\). Thus the same observed-to-expected form applies, with expected counts calculated from the constrained null maximum likelihood fit.

Degrees of Freedom and the Chi-Square Approximation

For a fixed null probability vector with all \(k\) probabilities specified, the asymptotic reference distribution of \(G^2\) is chi-square with \(k-1\) degrees of freedom. There are \(k\) category proportions, but they must sum to one, leaving \(k-1\) independent proportions to compare.

For a regular composite null, suppose the null model has \(r\) freely estimated, identifiable parameters, estimated from these same multinomial observations, and its fitted probability vector lies in the interior of the probability simplex. The asymptotic degrees of freedom are the difference between the saturated model's dimension and the null model's dimension:

\text{degrees of freedom}=(k-1)-r.

The saturated model has dimension \(k-1\) because its \(k\) probabilities sum to one. Subtracting the \(r\) parameters allowed by the null leaves the number of independent restrictions tested. This dimension difference is the reason the chi-square degrees of freedom change when parameters are estimated from the counts.

This approximation is an asymptotic result: it is used for sufficiently large samples under regularity conditions, including positive null category probabilities and an identifiable model whose fitted parameters are not on a boundary. It can be unreliable when expected counts are very small. If a null probability is zero, a positive observed count in that category is incompatible with that null; the ordinary chi-square approximation does not apply in that situation.

Write the null model.
State whether its category probabilities are fixed or depend on parameters estimated from the sample.

Find expected counts.
Use \(E_i=np_i^0\) for fixed probabilities, or \(E_i=n\widehat p_i^{\,0}\) for probabilities fitted under the null.

Calculate the likelihood ratio statistic.
Evaluate \(2\sum_i O_i\log(O_i/E_i)\), treating a zero observed count as a zero contribution.

Choose the reference degrees of freedom.
Use \(k-1\) for fixed probabilities and \(k-1-r\) when \(r\) regular parameters are estimated from the same data.

Compare and conclude.
For a right-tailed test, reject the null at significance level \(\alpha\) when \(G^2\) exceeds the upper-tail chi-square critical value with the chosen degrees of freedom.

Worked Applications

Worked Example: A Fixed Four-Category Model

A hypothetical laboratory classifies 100 independent specimens into four types. Under the model being assessed, the probabilities are \((0.40,0.30,0.20,0.10)\). The observed counts are \((46,26,18,10)\). Calculate the likelihood ratio statistic and decide whether to reject the model at the 5% level. State the multinomial model and degrees of freedom.

Method and model. Under the null, \((O_1,O_2,O_3,O_4)\sim\operatorname{Multinomial}(100;0.40,0.30,0.20,0.10)\). The observations are independent, each specimen belongs to exactly one type, and the null probabilities are fixed.

Expected counts. Multiplying each null probability by 100 gives \((E_1,E_2,E_3,E_4)=(40,30,20,10)\). The observed and expected counts both sum to 100.

Statistic. Substituting into the likelihood ratio formula and retaining the full logarithmic expressions until the final rounding gives

\begin{aligned} G^2 &=2\left[ 46\log\left(\frac{46}{40}\right) +26\log\left(\frac{26}{30}\right) +18\log\left(\frac{18}{20}\right) +10\log\left(\frac{10}{10}\right)\right]\\ &=2\left[ 46\log(1.15)+26\log(13/15)+18\log(0.9)+10\log(1)\right]\\ &\approx 1.6239. \end{aligned}

The null has four fixed probabilities, so the degrees of freedom are \(4-1=3\). The 5% upper-tail critical value is approximately 7.815. Since \(1.6239<7.815\), do not reject the specified model at the 5% level. This does not prove the probabilities are correct; the data do not provide sufficient evidence against them by this test.

Common mistake avoided: The fourth category has an observed count equal to its expectation, so its term is \(10\log(1)=0\). It is still a category in the degrees-of-freedom calculation.

Worked Example: Comparing Counts With Fixed Probabilities

A hypothetical claims review team independently assigns 120 files to one of three outcomes. Its prespecified null model has probabilities \((0.50,1/3,1/6)\). The observed counts are \((72,36,12)\). Test the model at the 5% level, giving the statistic and its degrees of freedom.

Method and model. Under the null, the count vector has a multinomial distribution with \(n=120\) and the stated three fixed probabilities. The categories are mutually exclusive and exhaustive.

Expected counts. The expected counts are \(120(0.50)=60\), \(120(1/3)=40\), and \(120(1/6)=20\). They sum to 120.

Statistic.

\begin{aligned} G^2 &=2\left[ 72\log\left(\frac{72}{60}\right) +36\log\left(\frac{36}{40}\right) +12\log\left(\frac{12}{20}\right)\right]\\ &=2\left[72\log(1.2)+36\log(0.9)+12\log(0.6)\right]\\ &\approx 6.4085. \end{aligned}

With three fixed category probabilities, the degrees of freedom are \(3-1=2\). The 5% critical value is approximately 5.991. Since \(6.4085>5.991\), reject the null model at the 5% level.

Common mistake avoided: The probabilities are not three independent parameters, because they sum to one. The degrees of freedom are two, not three.

Worked Example: Estimating a Parameter Under the Null

A hypothetical device records one of three outcomes on each of 100 independent trials. The null model says the probabilities are \((\theta/2,\theta/2,1-\theta)\), where \(0<\theta<1\) is unknown. The observed counts are \((36,24,40)\). Fit the null model, calculate \(G^2\), and determine whether there is evidence against the null at the 5% level.

Method and model. The count vector is multinomial with \(n=100\) and the stated parameterized probabilities. The two first categories have equal probabilities under the null. The likelihood for \(\theta\), apart from factors not involving it, is proportional to \(\theta^{O_1+O_2}(1-\theta)^{O_3}\). Maximizing this likelihood gives \(\widehat\theta=(O_1+O_2)/100=60/100=0.60\).

Fitted expected counts. The fitted probabilities are \((0.30,0.30,0.40)\), so the expected counts under the null are \((30,30,40)\). These differ from the unrestricted fitted counts \((36,24,40)\).

Statistic.

\begin{aligned} G^2 &=2\left[ 36\log\left(\frac{36}{30}\right) +24\log\left(\frac{24}{30}\right) +40\log\left(\frac{40}{40}\right)\right]\\ &=2\left[36\log(1.2)+24\log(0.8)+40\log(1)\right]\\ &\approx 2.4163. \end{aligned}

There are \(k=3\) categories and \(r=1\) parameter estimated from the sample. Thus the degrees of freedom are \((3-1)-1=1\). The 5% critical value is approximately 3.841. Since \(2.4163<3.841\), do not reject the null at the 5% level.

Common mistake avoided: Using \(k-1=2\) degrees of freedom would ignore that \(\theta\) was estimated from these observations. The null fit uses one parameter, leaving one degree of freedom for assessing fit.

Interpretation and Common Exam Traps

The value \(G^2\) compares the observed counts with the counts predicted by the fitted null model. A larger value indicates a greater loss of fit under the null relative to the saturated model. The test is right-tailed because large values provide evidence that the null restrictions fit poorly.

A frequent calculation error is to round each logarithmic contribution before adding. Small rounding errors can accumulate and may change the final reported digits. Keep the exact count ratios in the expression, calculate the sum at full calculator precision, and round only the final statistic. Report the chosen degrees of freedom separately from the statistic.

Do not use observed proportions as the null expectations. That would make the unrestricted model appear to be the null model and erase the comparison being tested.
Do not forget estimated parameters. For \(r\) regular parameters estimated from the sample, use \(k-1-r\), not \(k-1\).
Do not take absolute values of log terms. A category with \(O_i<E_i\) contributes a negative term; the total statistic is nonnegative because it is based on the likelihood ratio.
Do not discard a category with a zero observed count. Keep it in the model and degrees-of-freedom count; its contribution to \(G^2\) is zero when its expected count is positive.

Key takeaway: For multinomial goodness of fit, compare the saturated fit with the null fit using \(G^2=2\sum_i O_i\log(O_i/E_i)\). Use \(k-1\) degrees of freedom for fixed probabilities and \(k-1-r\) when \(r\) regular parameters are estimated from the data.

Check Your Understanding

For each problem, identify the appropriate null expected counts and degrees of freedom before evaluating the likelihood ratio statistic.

A hypothetical inspection process has 80 independent items and fixed null probabilities \((0.50,0.25,0.25)\). The observed counts are \((48,16,16)\). Find \(G^2\) and its degrees of freedom. (Answer: \(G^2\approx3.2217\), 2 degrees of freedom.)
In a hypothetical three-category survey, \(n=90\), the null probabilities are all \(1/3\), and the observed counts are \((40,30,20)\). Calculate the statistic and degrees of freedom. (Answer: \(G^2\approx6.7960\), 2 degrees of freedom.)
A hypothetical four-category model has probabilities \((\theta/3,\theta/3,\theta/3,1-\theta)\), with unknown \(0<\theta<1\). For \(n=100\), the observed counts are \((18,22,20,40)\). Find the null maximum likelihood estimate of \(\theta\), the statistic, and the degrees of freedom. (Answer: \(\widehat\theta=0.60\), \(G^2\approx0.4007\), 2 degrees of freedom.)
A null model assigns fixed probabilities \((0.20,0.30,0.50)\) to three categories. In \(n=50\) independent trials, the observed counts are \((0,20,30)\). Calculate \(G^2\), using the zero-count convention, and give the degrees of freedom. (Answer: \(G^2\approx22.4466\), 2 degrees of freedom.)
A regular null model for five categories has two identifiable parameters estimated from the same sample, and all fitted probabilities are positive. State the asymptotic degrees of freedom for its likelihood ratio goodness-of-fit statistic. (Answer: \(5-1-2=2\) degrees of freedom.)

What You'll Learn