Comparing Categorical Distributions Across Populations
Suppose an insurer is considering whether customers in several regions choose the same type of hypothetical service plan. Each region is a separate population, and each customer contributes one categorical response. The question is whether the category proportions are the same across the populations—not whether the number of customers sampled from each region is the same.
A chi-square test of homogeneity addresses this question using a contingency table of observed counts. The test pools information across populations to calculate the counts expected if their categorical distributions were identical, then measures how far the observed counts depart from those expectations.
Model, Hypotheses, and Expected Counts
Arrange the data in an \(r \times c\) table. Rows represent populations, columns represent categories, and each entry \(O_{ij}\) is the observed count from population \(i\) in category \(j\). Let \(n_i\) be the total in row \(i\), \(n_{\cdot j}\) the total in column \(j\), and \(N\) the grand total. Thus \(N=\sum_i n_i=\sum_j n_{\cdot j}\).
Under the null hypothesis, each population has the same category probabilities, say \(p_1,\ldots,p_c\), where the probabilities sum to 1. The common probability for category \(j\) is estimated by its overall proportion, \(n_{\cdot j}/N\). The expected count in row \(i\), column \(j\), is therefore:
The Pearson chi-square statistic compares every observed count with its corresponding expected count:
When the null hypothesis is true and the expected-count approximation is adequate, this statistic has approximately a chi-square distribution with
degrees of freedom. The p-value is the probability, under that null distribution, of obtaining a statistic at least as large as the observed \(X^2\). Large values provide evidence against the null hypothesis.
Why the Pooled Expected Count Is the Right Comparison
The expected count is not obtained by assigning the same number to every cell. It reflects both the size of each population sample and the overall frequency of each category. If one sample is twice as large as another, the null model expects it to contribute twice as many observations to each category.
To see why the pooled category proportion is used, suppose the null model assigns probability \(p_j\) to category \(j\) in every population. Across all samples, the total count in that category is \(n_{\cdot j}\), out of \(N\) observations. Its pooled estimate is therefore \(\widehat p_j=n_{\cdot j}/N\). The expected count for population \(i\), which contributes \(n_i\) observations, is \(n_i\widehat p_j\). This gives the expected-count formula above. The deviations \(O_{ij}-E_{ij}\) are then scaled by the expected counts in the Pearson statistic, so a given absolute difference counts more when it is large relative to its expected count.
The degrees of freedom also reflect the shared-distribution restriction. With \(r\) separate distributions, the alternative has \(r(c-1)\) freely varying category probabilities. Under the null, the populations share one distribution with \(c-1\) free probabilities. The difference is \(r(c-1)-(c-1)=(r-1)(c-1)\), giving the test’s degrees of freedom.
Worked Tests
Worked Example: Three Service Regions With Similar Counts
A hypothetical survey records one preferred service category for each of 60 customers sampled independently in each of two regions. The counts are shown below. At the 5% significance level, test whether the category distributions are the same.
| Region | Basic | Plus | Flex | Total |
|---|---|---|---|---|
| North | 30 | 20 | 10 | 60 |
| South | 20 | 30 | 10 | 60 |
| Total | 50 | 50 | 20 | 120 |
Method and hypotheses. These are independent samples with the same three categories, so use the Pearson chi-square test of homogeneity. The null hypothesis is that both regions have the same category distribution; the alternative is that their distributions differ.
Expected counts. Each row total is 60. The expected counts in either row are \(60(50/120)=25\), \(60(50/120)=25\), and \(60(20/120)=10\). Thus both rows have expected counts 25, 25, and 10.
Statistic. In the North row, the contributions are \((30-25)^2/25=1\), \((20-25)^2/25=1\), and \((10-10)^2/10=0\). In the South row, they are \((20-25)^2/25=1\), \((30-25)^2/25=1\), and \((10-10)^2/10=0\). Adding the exact contributions gives \(X^2=4\).
Degrees of freedom and decision. Here \(r=2\) and \(c=3\), so \(df=(2-1)(3-1)=2\). For a chi-square distribution with 2 degrees of freedom, the upper-tail probability at 4 is \(e^{-4/2}=e^{-2}\approx0.1353\). Since \(0.1353>0.05\), do not reject the null hypothesis. The sample does not provide sufficient evidence at the 5% level that the regions’ distributions differ.
Check and common mistake avoided. The expected counts sum to 60 in each row and to 50, 50, and 20 down the columns, matching the observed margins. A common mistake is to use 20 as the expected count in every cell simply because the grand total is 120 across six cells. The correct expectations must preserve both the row and column totals.
Worked Example: Comparing Three Categories Across Two Populations
In a hypothetical study, 50 policyholders are sampled independently from each of two populations and asked which of three optional coverage features they would select. The observed counts are \((32,8,10)\) in Population A and \((18,22,10)\) in Population B. Test whether the distributions differ at the 5% level.
Method and hypotheses. Use the chi-square test of homogeneity. The null hypothesis is that the two populations share the same three category probabilities; the alternative is that at least one probability differs.
Expected counts. The column totals are 50, 30, and 20, and the grand total is 100. Each row total is 50. Therefore, the expected counts in each row are \(50(50/100)=25\), \(50(30/100)=15\), and \(50(20/100)=10\).
Statistic. For Population A, the three contributions are \((32-25)^2/25=49/25=1.96\), \((8-15)^2/15=49/15\), and \((10-10)^2/10=0\). For Population B, the contributions are \((18-25)^2/25=49/25\), \((22-15)^2/15=49/15\), and 0. Using exact fractions before rounding:
Degrees of freedom and decision. With two populations and three categories, \(df=(2-1)(3-1)=2\). The upper-tail probability is \(e^{-X^2/2}=e^{-392/75}\approx0.0054\). Since \(0.0054<0.05\), reject the null hypothesis. The data provide evidence that the category distributions differ.
Check and common mistake avoided. Expected counts total 50 in each row, and their column totals match 50, 30, and 20. Do not round the component \(49/15\) before adding all contributions: carry the fractions through the sum and round the final statistic to four decimal places.
Worked Example: A Two-by-Two Homogeneity Test
A hypothetical insurer independently samples 100 customers from each of two regions. In Region A, 70 prefer a proposed digital option and 30 do not; in Region B, 40 prefer it and 60 do not. Test equality of the preference proportions at the 5% level using the uncorrected Pearson statistic.
Method and hypotheses. This is a chi-square test of homogeneity for two populations and two categories. The null hypothesis is that the preference proportions are equal; the alternative is that they are different.
Expected counts. The column totals are 110 and 90, with grand total 200. Each row total is 100, so the expected counts in each row are \(100(110/200)=55\) and \(100(90/200)=45\).
Statistic. The four observed-minus-expected differences are \(15,-15,-15,15\). Hence:
Degrees of freedom and decision. Here \(df=(2-1)(2-1)=1\). The upper-tail probability for \(X^2=18.1818\) with 1 degree of freedom is less than 0.0001. Reject the null hypothesis at the 5% level; the sample provides strong evidence that the preference proportions differ.
Check and common mistake avoided. Each expected row sums to 100 and each expected column sums to its observed total. Do not compare the two observed proportions as if each sample had its own unrelated category baseline: the null hypothesis requires one common proportion, estimated from the pooled column totals.
Interpreting the Test and Avoiding Traps
A homogeneity test and a test of independence can use the same Pearson statistic and degrees-of-freedom formula. Their distinction is in the sampling design and question. A homogeneity test starts with separate samples from different populations and asks whether their categorical distributions are equal. A test of independence starts with one sample and asks whether two categorical variables are associated. State the design and hypotheses in context rather than choosing a test from the table’s shape alone.
Rejecting the null hypothesis indicates evidence that not all population distributions are the same. It does not show which specific category or population accounts for the difference, nor does it establish that the difference is practically important. The test statistic is a single overall measure of discrepancy. Conversely, failing to reject does not prove the distributions are identical; it means the data do not provide sufficient evidence of a difference at the chosen significance level.
- Do not assume equal sample sizes. Expected counts use each row total and the pooled category proportions. Unequal row totals are handled by the formula.
- Do not use each population’s separate proportions to calculate expected counts. Those proportions describe the observed data; under the null, expected counts use the common pooled distribution.
- Do not round each contribution and then add the rounded values. Keep fractions or adequate precision through the sum, then round the final statistic.
- Do not ignore the sampling design or sparse expected counts. Independence and a suitable chi-square approximation are conditions for the usual p-value.
Check Your Understanding
For each problem, use the ordinary Pearson chi-square approximation when the expected counts are adequate. Round test statistics and p-values to four decimal places unless the answer is stated as a bound.
- Two independent groups of 40 customers each choose between two hypothetical billing formats. Their counts are \((24,16)\) and \((16,24)\). Find the expected counts, \(X^2\), degrees of freedom, and p-value; state the 5% decision. (Answer: Expected counts are 20 in each cell; \(X^2=3.2000\), \(df=1\), \(p\approx0.0736\); do not reject.)
- Three independent samples of 30 residents each report one of two hypothetical transit preferences. The counts are \((18,12)\), \((15,15)\), and \((12,18)\). Calculate the expected counts, statistic, degrees of freedom, and 5% conclusion. (Answer: Expected counts are 15 in every cell; \(X^2=7.2000\), \(df=2\), \(p\approx0.0273\); reject homogeneity.)
- Two populations each contribute 40 independently sampled observations to a three-category survey. Their counts are \((20,12,8)\) and \((12,16,12)\). Find the expected counts, Pearson statistic, degrees of freedom, and p-value; state the 5% decision. (Answer: Expected counts in each row are \((16,14,10)\); \(X^2=118/35\approx3.3714\), \(df=2\), \(p\approx0.1853\); do not reject.)
- Three independent populations each contribute 50 observations to a two-category hypothetical enrollment survey. Their counts are \((30,20)\), \((25,25)\), and \((15,35)\). Calculate the expected counts, test statistic, degrees of freedom, and p-value, then state the conclusion at 5%. (Answer: Expected counts in each row are \((70/3,80/3)\); \(X^2=3.1250\), \(df=2\), \(p\approx0.2096\); do not reject.)