Pearson Chi-Square Goodness-of-Fit Tests for Multinomial Counts

Why Test a Multinomial Model?

A specified probability model makes a precise prediction about how observations should be distributed across categories. For example, a hypothetical insurer might classify incoming claims into four mutually exclusive claim types and, for planning, specify a probability for each type. A goodness-of-fit test asks whether the observed counts are reasonably consistent with those probabilities, or whether the discrepancies are too large to attribute plausibly to sampling variation alone.

The test concerns the overall pattern of category counts. It does not test a separate hypothesis for each category, and it does not identify which category caused a rejection by itself. The method here applies when the categories are mutually exclusive and exhaustive and their probabilities are specified before examining the observed counts.

Definition: Suppose \(n\) independent observations each fall into exactly one of \(k\) categories. Under the null hypothesis, the probability of category \(j\) is a specified value \(p_j\), where \(p_j>0\) and \(\sum_{j=1}^{k}p_j=1\). The observed counts \(O_1,\ldots,O_k\) then have a multinomial distribution with sample size \(n\) and probabilities \(p_1,\ldots,p_k\). The goodness-of-fit hypotheses are \(H_0:\) the category probabilities are \(p_1,\ldots,p_k\), against \(H_a:\) the probability vector differs from this specified vector.

Expected Counts and Pearson’s Statistic

Under the null hypothesis, the expected count in category \(j\) is the sample size multiplied by the specified probability for that category. Denote this expected count by \(E_j\). Pearson’s statistic adds the squared discrepancies between observed and expected counts, scaled by the expected counts.

E_j=np_j,\qquad X^2=\sum_{j=1}^{k}\frac{(O_j-E_j)^2}{E_j}.

Large values of \(X^2\) indicate that the observed counts are far from their null expectations. The squaring means that a category count above expectation and one below expectation both contribute positively. For a test at significance level \(\alpha\), reject \(H_0\) when the upper-tail p-value is at most \(\alpha\), or equivalently when the statistic exceeds the appropriate upper-tail critical value.

Theorem: When the null probabilities are specified in advance, the usual large-sample reference distribution for Pearson’s goodness-of-fit statistic is approximately chi-square with \(k-1\) degrees of freedom. Thus, the approximate p-value is the probability, under a chi-square distribution with \(k-1\) degrees of freedom, of obtaining a value at least as large as the observed \(X^2\). The approximation requires an adequate sample size; it can be unreliable when expected counts are too small.

The degrees of freedom are \(k-1\), not \(k\), because the observed counts must add to \(n\). Once \(k-1\) counts are known, the final count is determined. This formula is for probabilities specified independently of the observed data. When probabilities have been estimated from the same data, the degrees-of-freedom calculation changes; that case is distinct from the fixed-probability test covered here.

Why the Statistic Has This Form

The denominator \(E_j\) scales a discrepancy by the size of the count expected in that category. A difference of 5 is more notable when 10 observations are expected than when 100 are expected. More specifically, under the null model, the variance of a single category count \(O_j\) is \(np_j(1-p_j)\), while Pearson’s statistic uses \(np_j=E_j\) as a convenient scaling that leads to the standard chi-square approximation for the complete multinomial count vector.

The direct algebraic motivation is especially clear by writing each contribution as a squared scaled discrepancy:

\left(\frac{O_j-E_j}{\sqrt{E_j}}\right)^2 =\frac{(O_j-E_j)^2}{E_j}, \qquad X^2=\sum_{j=1}^{k}\left(\frac{O_j-E_j}{\sqrt{E_j}}\right)^2.

Thus, Pearson’s statistic is the sum of squared observed-versus-expected discrepancies after scaling each by the square root of its expected count. Also, because \(\sum_j O_j=n\) and \(\sum_j E_j=n\), the deviations obey \(\sum_j(O_j-E_j)=0\). There are only \(k-1\) freely varying count deviations, which explains the degrees-of-freedom count. The chi-square reference distribution is an approximation, rather than an exact finite-sample distribution for every multinomial problem.

Worked Tests

Worked Example: A Four-Category Claim Classification

In a hypothetical portfolio review, a model specifies probabilities \(0.40, 0.30, 0.20,\) and \(0.10\) for four claim categories. A sample of \(n=100\) independent claims produces counts \(48,24,18,\) and \(10\). At significance level \(\alpha=0.05\), test whether the specified model fits.

Method. Apply Pearson’s goodness-of-fit test for fixed multinomial probabilities. Under \(H_0\), the probability vector is \((0.40,0.30,0.20,0.10)\); under \(H_a\), it differs from that vector.

Step 1: Find expected counts. Multiply each probability by 100:

$$ (E_1,E_2,E_3,E_4) =(100(0.40),100(0.30),100(0.20),100(0.10)) =(40,30,20,10). $$

Step 2: Calculate each contribution.

\begin{aligned} \frac{(48-40)^2}{40}&=\frac{64}{40}=1.6,\\ \frac{(24-30)^2}{30}&=\frac{36}{30}=1.2,\\ \frac{(18-20)^2}{20}&=\frac{4}{20}=0.2,\\ \frac{(10-10)^2}{10}&=0. \end{aligned}

Step 3: Add the contributions and set the degrees of freedom. There are \(k=4\) categories with probabilities specified in advance, so \(df=4-1=3\). The statistic is \(X^2=1.6+1.2+0.2+0=3.0000\). The upper-tail p-value from a chi-square distribution with 3 degrees of freedom is approximately \(0.3916\), to four decimals.

Decision and check. Since \(0.3916>0.05\), do not reject \(H_0\). The observed counts are not unusually far from the specified expectations at the 5% level. As a check, the observed counts sum to \(48+24+18+10=100\), and the expected counts also sum to 100.

Common mistake avoided. The test is based on the sum of all four contributions, not on the largest single discrepancy. Here the largest contribution is 1.6, but the statistic is 3.0.

Worked Example: Five Categories and a Smallest-Category Trap

A hypothetical service team specifies five probabilities \((0.10,0.20,0.30,0.25,0.15)\) for the types of requests it expects to receive. From \(n=200\) independent requests, the observed counts are \((12,48,66,44,30)\). Test the specified model at \(\alpha=0.05\).

Method. Use a fixed-probability Pearson goodness-of-fit test. The null hypothesis is that the five category probabilities equal the stated values; the alternative is that at least one probability differs.

Step 1: Calculate the expected counts.

$$ (E_1,E_2,E_3,E_4,E_5) =(200(0.10),200(0.20),200(0.30),200(0.25),200(0.15)) =(20,40,60,50,30). $$

Step 2: Calculate and add the contributions.

\begin{aligned} X^2 &=\frac{(12-20)^2}{20} +\frac{(48-40)^2}{40} +\frac{(66-60)^2}{60} +\frac{(44-50)^2}{50} +\frac{(30-30)^2}{30}\\ &=\frac{64}{20}+\frac{64}{40}+\frac{36}{60}+\frac{36}{50}+0\\ &=3.2+1.6+0.6+0.72=6.12. \end{aligned}

Step 3: Find degrees of freedom and decide. Since \(k=5\) and the probabilities are fixed, \(df=5-1=4\). The upper-tail p-value is approximately \(0.1904\), to four decimals. Because \(0.1904>0.05\), do not reject the specified model.

Check. The observed counts sum to \(12+48+66+44+30=200\), matching the sample size. The expected counts also sum to \(20+40+60+50+30=200\).

Common mistake avoided. Do not omit the category with observed count equal to its expectation when checking the setup: it contributes zero to \(X^2\), but it remains one of the five categories when determining \(df\).

Worked Example: A Borderline Statistic With Four Fixed Probabilities

A hypothetical quality-control plan specifies probabilities \((0.50,0.25,0.15,0.10)\) for four mutually exclusive inspection outcomes. In a sample of \(n=120\) independent items, the observed counts are \((72,24,15,9)\). Use a 5% test.

Method. Calculate the Pearson statistic for the specified multinomial model and compare it with the upper-tail chi-square critical value.

Step 1: Calculate expected counts.

$$ (E_1,E_2,E_3,E_4) =(120(0.50),120(0.25),120(0.15),120(0.10)) =(60,30,18,12). $$

Step 2: Calculate contributions.

\begin{aligned} \frac{(72-60)^2}{60}&=\frac{144}{60}=2.4,\\ \frac{(24-30)^2}{30}&=\frac{36}{30}=1.2,\\ \frac{(15-18)^2}{18}&=\frac{9}{18}=0.5,\\ \frac{(9-12)^2}{12}&=\frac{9}{12}=0.75. \end{aligned}

Step 3: Add and compare. The statistic is \(X^2=2.4+1.2+0.5+0.75=4.85\). With four fixed-probability categories, \(df=3\). The 5% upper-tail critical value is approximately \(7.815\). Since \(4.85<7.815\), do not reject \(H_0\) at the 5% level.

Check. The observed counts total \(72+24+15+9=120\), and the expected counts total \(60+30+18+12=120\). Each contribution is nonnegative, as required by the squared-discrepancy formula.

Common mistake avoided. This is an upper-tail test: small values of \(X^2\) indicate close agreement, not evidence against the model. Rejecting because the statistic is merely nonzero would use the test incorrectly.

Assumptions, Interpretation, and Common Traps

Before using the reference distribution, check the design and the model. Each observation must belong to exactly one category, categories must cover all possible outcomes, and the observations must be independent under the sampling model. The null probabilities must be valid, sum to one, and be specified in advance for the \(k-1\) degrees-of-freedom rule used here.

The chi-square approximation is intended for settings with expected counts large enough for the approximation to be credible. If expected counts are very small, the nominal p-value may be inaccurate; do not treat the test result as automatically trustworthy just because a statistic can be computed. The adequacy of expected counts is a substantive condition, not an arithmetic detail.

Using observed counts as expectations. Expected counts come from \(E_j=np_j\), not from redistributing the observed counts to make the model look closer.
Using the wrong degrees of freedom. With \(k\) fixed categories and all probabilities specified in advance, use \(k-1\), because the counts sum to \(n\).
Forgetting to square deviations. Positive and negative deviations do not cancel; each contribution is \((O_j-E_j)^2/E_j\).
Using a two-sided tail. Pearson’s statistic becomes more extreme as it increases, so the p-value is an upper-tail probability.
Overstating a non-rejection. Not rejecting \(H_0\) does not prove that the specified model is true; it means the data do not provide sufficient evidence against it at the chosen significance level.

Key takeaway: For a multinomial goodness-of-fit test with fixed probabilities, compute \(E_j=np_j\), add \((O_j-E_j)^2/E_j\) across all categories, and compare the result with an upper-tail chi-square distribution having \(k-1\) degrees of freedom. Check the assumptions and expected-count adequacy before interpreting the decision.

Check Your Understanding

For each problem, calculate the expected counts and Pearson statistic before making the stated test decision. Probabilities are specified in advance in every question.

A hypothetical transit desk classifies \(50\) independent inquiries into three categories with specified probabilities \((0.50,0.30,0.20)\). The observed counts are \((30,12,8)\). Find \(X^2\), the degrees of freedom, and the decision at \(\alpha=0.05\). (Answer: Expected counts \((25,15,10)\); \(X^2=2.0000\), \(df=2\), \(p\approx0.3679\); do not reject.)
A four-outcome model assigns probability \(0.25\) to each outcome. For \(n=80\), the counts are \((20,20,20,20)\). Test at \(\alpha=0.05\). (Answer: Expected counts \((20,20,20,20)\); \(X^2=0.0000\), \(df=3\), \(p=1.0000\); do not reject.)
A hypothetical claims model specifies probabilities \((0.40,0.30,0.20,0.10)\). In \(100\) independent claims the counts are \((30,35,20,15)\). Give the statistic, degrees of freedom, and decision at the 5% level. (Answer: Expected counts \((40,30,20,10)\); \(X^2=5.8333\), \(df=3\); do not reject at 5%.)
A five-category model specifies probabilities \((0.10,0.20,0.25,0.25,0.20)\). A sample of \(200\) independent records gives counts \((30,35,50,55,40)\). Test at \(\alpha=0.05\). (Answer: Expected counts \((20,40,50,50,40)\); \(X^2=6.1250\), \(df=4\); do not reject at 5%.)
A hypothetical inspection model specifies probabilities \((0.15,0.25,0.35,0.25)\). In \(160\) independent inspections, the observed counts are \((18,52,60,30)\). Find the statistic and decide at \(\alpha=0.05\). (Answer: Expected counts \((24,40,56,40)\); \(X^2\approx7.8857\), \(df=3\); reject at 5%.)

What You'll Learn