Pooling Categories Without Changing the Question

Why Pool Categories?

A goodness-of-fit test compares observed category counts with the counts expected under a specified null distribution. If some expected counts are small, the chi-square approximation to the test statistic’s reference distribution may be unreliable. Pooling can sometimes help: it replaces several fine-grained categories with a broader category whose expected count is their sum.

Pooling is not a way to repair every difficult analysis. It is defensible only when the combined categories make sense for the question being asked. For ordered categories, this often means combining adjacent ranges. For unordered categories, it may mean combining outcomes that have a meaningful shared interpretation. Pooling also reduces detail: a test on grouped categories cannot detect every mismatch among the original categories.

Definition: Suppose a goodness-of-fit test has \(k\) mutually exclusive and exhaustive categories, observed counts \(O_1,\ldots,O_k\), and specified null probabilities \(p_1,\ldots,p_k\), where \(\sum_{i=1}^k p_i=1\). A pooling is a partition of these categories into \(g\) groups. For group \(j\), its observed count is \(O'_j=\sum_{i\in G_j}O_i\), its null probability is \(p'_j=\sum_{i\in G_j}p_i\), and its expected count is \(E'_j=np'_j\).

The grouped test asks whether the probabilities of the pooled outcomes match their grouped null probabilities. For example, if several waiting-time bands are pooled into “30 minutes or longer,” the test can assess the probability of that broad outcome, but no longer tests whether the counts are correctly split among the original long-wait bands.

How the Grouped Test Is Calculated

Assume the observations are independent, each observation belongs to exactly one category, and the null probabilities are specified for the population or process being studied. The grouped categories must remain mutually exclusive and exhaustive, and their null probabilities must add to 1. Under these conditions, calculate the Pearson statistic using the grouped counts and probabilities:

X^2_{\text{grouped}} = \sum_{j=1}^{g}\frac{(O'_j-E'_j)^2}{E'_j}, \qquad E'_j=np'_j.

For a fixed probability model with \(g\) positive-probability groups, the reference chi-square distribution has \(g-1\) degrees of freedom. Pooling categories reduces the number of groups, and therefore reduces the degrees of freedom. If probabilities are estimated from the data rather than fixed in advance, the degrees of freedom must also account for the parameters estimated in the grouped model; the fixed-model formula \(g-1\) does not apply unchanged.

The usual chi-square approximation also requires adequate expected counts. A common conservative guideline is that every expected count is at least 5, though approximation conditions are guidelines rather than a universal law. Pooling may improve the expected-count pattern, but it does not resolve dependence between observations, incorrect null probabilities, or categories that fail to represent the scientific question.

Worked Example: Pooling Sparse Adjacent Wait-Time Bands

In a hypothetical sample of 30 independent service visits, waiting time is classified into five ordered bands. Under a fixed null model, the probabilities are 0.10, 0.20, 0.40, 0.20, and 0.10. The observed counts are 4, 7, 10, 5, and 4. The question concerns whether the broad distribution of waiting time follows the null model. The original expected counts are \(30(0.10)=3\), \(30(0.20)=6\), \(30(0.40)=12\), \(30(0.20)=6\), and \(30(0.10)=3\). The two outer bands have expected counts below 5.

Suppose the bands were combined in advance into “15 minutes or less” (the first two bands), “the middle band” (the third band), and “30 minutes or longer” (the last two bands). These are adjacent combinations that preserve the broad time-range question.

The grouped observed counts are \(4+7=11\), \(10\), and \(5+4=9\). The grouped null probabilities are \(0.10+0.20=0.30\), \(0.40\), and \(0.20+0.10=0.30\). The expected counts are therefore \(30(0.30)=9\), \(30(0.40)=12\), and \(30(0.30)=9\), each at least 5.

The grouped statistic is

X^2_{\text{grouped}} = \frac{(11-9)^2}{9} + \frac{(10-12)^2}{12} + \frac{(9-9)^2}{9} = \frac{4}{9}+\frac{4}{12}+0 = 0.7778.

There are \(g=3\) grouped categories, so \(df=3-1=2\). Using the chi-square approximation, the upper-tail p-value is about 0.678. At a 0.05 significance level, this result does not provide evidence against the grouped null distribution. The test is about the three broad ranges, not whether the five original bands separately match their probabilities.

The observations are stipulated to be independent and assigned to exactly one waiting-time band, and the grouped categories cover all five bands. The grouped expected counts meet the stated guideline. Pooling thus addresses the small expected counts for this grouped question, although it sacrifices information about differences within each combined range.

Why Pooling Reduces the Statistic and the Degrees of Freedom

A useful way to understand pooling is to compare the contribution from several original categories with the contribution from their combined group. For the categories in one group, let \(E_i=np_i\) be their original expected counts and let \(d_i=O_i-E_i\) be their deviations from expectation. The pooled deviation is \(\sum_i d_i\), and the pooled expected count is \(\sum_i E_i\).

By the Cauchy–Schwarz inequality,

\left(\sum_i d_i\right)^2 = \left(\sum_i \sqrt{E_i}\frac{d_i}{\sqrt{E_i}}\right)^2 \leq \left(\sum_i E_i\right)\left(\sum_i \frac{d_i^2}{E_i}\right).

Since the pooled expected count \(\sum_i E_i\) is positive, dividing both sides by it gives

\frac{(\sum_i d_i)^2}{\sum_i E_i} \leq \sum_i \frac{d_i^2}{E_i}.

The left side is the pooled group’s contribution to \(X^2\); the right side is the sum of the original categories’ contributions. Thus pooling cannot increase the statistic for a fixed set of observed counts and a fixed null model. The inequality can be strict because deviations in different categories may offset one another. Equality holds when the ratios \(d_i/E_i\) are the same across the categories being combined.

If the original model has \(k\) positive-probability categories and the grouped model has \(g\), the fixed-model degrees of freedom fall from \(k-1\) to \(g-1\). Each independent merge reduces the number of groups, and hence the degrees of freedom, by one. A smaller statistic does not guarantee a larger p-value: the reference distribution changes too when the degrees of freedom change.

Worked Example: Pooling Can Hide a Difference Within a Group

A hypothetical study records 100 independent observations in six mutually exclusive categories. The fixed null probabilities are 0.10, 0.10, 0.20, 0.20, 0.20, and 0.20, giving expected counts 10, 10, 20, 20, 20, and 20. The observed counts are 5, 15, 20, 20, 20, and 20. All original expected counts meet the common guideline, so sparse expectations do not require pooling.

The original six-category statistic is

X^2 = \frac{(5-10)^2}{10} + \frac{(15-10)^2}{10} + 4\left(\frac{(20-20)^2}{20}\right) = 2.5+2.5+0 = 5.000.

With six fixed-probability categories, \(df=6-1=5\). Now suppose the first two categories had been combined in advance because the intended question concerns their shared broad outcome, rather than the distinction between them. Their pooled observed count is \(5+15=20\), and their pooled null probability is \(0.10+0.10=0.20\), so the pooled expected count is \(100(0.20)=20\). The other four categories are unchanged.

The five-group statistic is therefore

X^2_{\text{grouped}} = \frac{(20-20)^2}{20} + 4\left(\frac{(20-20)^2}{20}\right) = 0, \qquad df=5-1=4.

The grouped counts match the grouped null probabilities exactly, but the original split is far from its expected counts: one category has 5 rather than 10 observations, while the other has 15 rather than 10. The grouped test cannot detect this redistribution within the combined outcome. Pool only when the grouped question is scientifically appropriate, not simply because doing so produces a more reassuring result.

Prespecification and Interpretation

Whenever possible, decide which categories to pool before examining the observed counts. The grouping should follow the meaning of the outcomes or a design decision, not a search for a particular test statistic or p-value. If a grouping is chosen after seeing the data, the usual chi-square p-value does not automatically account for that selection. In that case, report the data-dependent choice transparently; do not present the unadjusted p-value as though the grouping had been fixed in advance.

Pooling can preserve a broad question while changing its resolution. If the question is whether a specified proportion of observations fall into “any event” rather than “no event,” combining meaningful event subtypes may be suitable. If the question asks whether each subtype has its own specified probability, combining those subtypes removes distinctions central to the question and is not an adequate substitute.

Worked Example: A Prespecified “Any Event” Category

In a hypothetical process, each of 100 independent units is classified into exactly one of five mutually exclusive outcomes: no event, or one of four event subtypes. A rule specified before data collection calls for testing the broad distinction “any event” versus “no event.” Under the fixed null model, the probabilities are 0.90 for no event and 0.04, 0.03, 0.02, and 0.01 for the four subtypes. The observed counts are 90, 7, 2, 1, and 0.

The four event probabilities sum to \(0.04+0.03+0.02+0.01=0.10\), and their observed counts sum to \(7+2+1+0=10\). The two grouped probabilities are 0.90 and 0.10, so their expected counts are \(100(0.90)=90\) and \(100(0.10)=10\). Both expected counts meet the common guideline.

The grouped statistic is

X^2_{\text{grouped}} = \frac{(90-90)^2}{90} + \frac{(10-10)^2}{10} = 0.

There are two groups and no probabilities estimated from the data, so \(df=2-1=1\). The chi-square upper-tail p-value is 1.000. The grouped counts exactly match the null expectation for “any event” versus “no event.” This result does not show that the four subtype probabilities are individually correct; their observed counts have been combined, and that finer question was not tested.

The observations are independent, the classification places each unit in exactly one outcome, and the five outcomes cover all possibilities. The grouping was specified in advance and directly matches the stated broad question. Those features make the pooled analysis interpretable for its intended purpose.

Common Mistakes

Adding observed counts but not null probabilities. The expected count for a pooled group must be \(n\) times the sum of the group’s null probabilities. Summing counts while keeping only one original probability gives the wrong expectation.
Assuming pooling leaves the question unchanged in every sense. It may preserve a broad question, but it removes the ability to assess differences among the categories inside each group. State what the grouped test can and cannot detect.
Keeping the original degrees of freedom. For a fixed probability model, use \(g-1\), where \(g\) is the number of pooled groups, not the number of original categories.
Pooling until the p-value is attractive. A choice driven by the observed results can invalidate the usual p-value interpretation. Plan defensible groups in advance and report any later grouping choice clearly.
Assuming a smaller statistic means a larger p-value. Pooling changes both the statistic and the degrees of freedom, so the p-value need not move in the same direction as the statistic.

Key takeaway: Pool by adding both observed counts and null probabilities, and then recalculate the statistic and degrees of freedom using the grouped categories. A defensible grouping fits the scientific question, supports the approximation conditions, and does not conceal distinctions that the analysis is meant to test.

Check Your Understanding

Try these questions without looking back.

A fixed null model has four categories with probabilities 0.10, 0.20, 0.30, and 0.40. If the first two are pooled, what is the pooled probability, and how many degrees of freedom does the three-group test have?
Why must the expected count of a pooled group be calculated from the sum of its null probabilities?
Explain why pooling two categories can lower the Pearson statistic even when their observed counts differ substantially from expectation in opposite directions.
A researcher pools categories only after seeing which combination gives the largest p-value. Why is the usual p-value potentially misleading?
What distinction is lost when a test combines four event subtypes into one “any event” category?