Why Compare Outcome Distributions Across Groups?
Suppose a health system wants to know whether patients at several clinics choose the same way to receive appointment reminders. It takes a separate sample of patients from each clinic and records one categorical outcome for each patient: text message, phone call, or email. The question is not whether two variables measured on one sample are associated. It is whether the distribution of one categorical outcome is the same across the groups sampled.
A chi-square test of homogeneity addresses this question. Its calculation uses a two-way table and the Pearson chi-square statistic, but the study design gives the test its meaning: separate groups are sampled, and their outcome distributions are compared. This tutorial works through that analysis from hypotheses to conclusion.
Hypotheses, Expected Counts, and the Test Statistic
Suppose there are \(r\) groups and \(c\) outcome categories. The null hypothesis states that the category proportions are the same for every group. The alternative is that at least one group has a different distribution. Equivalently, the alternative says that not all corresponding category proportions are equal; it does not specify which groups or categories differ.
Write \(O_{ij}\) for the observed count in row \(i\), column \(j\), \(R_i\) for row \(i\)'s total, \(C_j\) for column \(j\)'s total, and \(N\) for the grand total. Under the null hypothesis, the pooled proportion in category \(j\) estimates the common population proportion for that category. The expected count in group \(i\), category \(j\), is therefore the group size multiplied by that pooled proportion.
The Pearson chi-square statistic adds the squared difference between each observed and expected count, scaled by the expected count. Larger values indicate greater overall discrepancy between the observed table and the table expected under a common distribution.
When the null hypothesis is true and the chi-square conditions are met, the statistic is approximately distributed as chi-square with \((r-1)(c-1)\) degrees of freedom. The test is right-tailed: unusually large values of \(X^2\) count against the null hypothesis.
Worked Test Across Three Groups
Worked Example: Comparing Reminder Preferences at Three Clinics
Hypothetical data. Independent random samples of patients are taken from each of three clinics. Each sampled patient reports one preferred appointment reminder method. The counts are shown below.
| Clinic | Text | Phone | Total | |
|---|---|---|---|---|
| A | 30 | 20 | 10 | 60 |
| B | 20 | 25 | 15 | 60 |
| C | 10 | 20 | 30 | 60 |
| Total | 60 | 65 | 55 | 180 |
Step 1: State the hypotheses. \(H_0\): The distribution of reminder preferences is the same at all three clinics. \(H_a\): At least one clinic has a different distribution of preferences.
Step 2: Check the conditions. The hypothetical design uses independent random samples from the three clinics, and each patient contributes to exactly one cell. Each patient selects one of the three listed categories, so the categories are mutually exclusive and exhaustive. The expected counts, calculated below, are all at least 18.3, so the usual expected-count condition is met.
Step 3: Calculate the expected counts. For Clinic A and Text, for example, the expected count is \(60(60/180)=20\). Because all three row totals are 60, the expected counts are the same in each row:
Step 4: Calculate the statistic. Substitute all observed and expected counts into the Pearson formula:
Step 5: Find the degrees of freedom and p-value. There are \(r=3\) groups and \(c=3\) categories, so \(df=(3-1)(3-1)=4\). The right-tail probability is \(P(\chi^2_4\geq22.587)\approx0.000153\).
Step 6: Conclude in context. For a significance level of \(\alpha=0.05\), the p-value is smaller than \(\alpha\), so reject \(H_0\). The data provide strong evidence that appointment-reminder preferences are not distributed identically across these three clinic populations. This test alone does not identify which clinics or categories account for the difference, and it does not show that clinic location causes a preference.
Why the Formula and Degrees of Freedom Fit the Question
The expected-count formula follows from the null hypothesis of a common category distribution. If the pooled fraction of observations in category \(j\) is \(C_j/N\), then a group with \(R_i\) observations would be expected to have \(R_i(C_j/N)\) observations in that category if the same proportions applied to every group. The expected counts are therefore determined by the row totals and the pooled category proportions rather than by treating each group’s observed proportions as separate estimates.
The degrees of freedom reflect the number of independent ways the table can depart from the null pattern once its margins are accounted for. A table with \(r\) rows and \(c\) columns has \(rc\) cells, but fixing its row and column totals imposes \(r+c-1\) independent constraints; the last constraint is redundant because both sets of totals sum to \(N\). Thus the remaining dimension is:
This gives the degrees of freedom used for the Pearson chi-square reference distribution. It also clarifies why one does not use \(rc-1\) degrees of freedom: the margins constrain the possible table patterns.
A Nonsignificant Result Is Not Proof of Equality
Worked Example: A Difference That Is Not Statistically Significant
Hypothetical data. Independent random samples of 30 patients are taken from each of three clinics. Each patient is classified according to whether they prefer a morning or afternoon appointment.
| Clinic | Morning | Afternoon | Total |
|---|---|---|---|
| A | 18 | 12 | 30 |
| B | 15 | 15 | 30 |
| C | 9 | 21 | 30 |
| Total | 42 | 48 | 90 |
The hypotheses are \(H_0\): the morning-versus-afternoon preference distribution is the same in all three clinic populations; \(H_a\): at least one population has a different distribution. The samples are independent, each patient is counted once in one category, and the categories are exhaustive. Expected counts are 14 for morning and 16 for afternoon in each clinic, so the expected-count condition is satisfied.
For each clinic, the expected counts are \(30(42/90)=14\) for morning and \(30(48/90)=16\) for afternoon. Therefore:
Here \(df=(3-1)(2-1)=2\), and the right-tail p-value is \(P(\chi^2_2\geq5.625)\approx0.0601\). At \(\alpha=0.05\), do not reject \(H_0\). These data do not provide sufficient evidence at the 5% significance level that the preference distributions differ across the clinics. They do not establish that the distributions are exactly equal; a real difference might be too small for these samples to detect.
The Study Design Determines the Name of the Test
A test of homogeneity and a test of independence can use the same table layout, expected-count formula, Pearson statistic, and degrees of freedom. The distinction is the sampling plan and the question. In a homogeneity study, separate groups or populations are sampled and their outcome distributions are compared. In an independence study, one sample is classified according to two categorical variables, and the question is whether those variables are associated in the population.
For homogeneity, group sample sizes are often set by the design, while the numbers in the outcome categories are observed. For example, a researcher might decide in advance to sample 40 patients from one clinic and 60 from another, then record each patient’s preference. The test compares the category proportions across the sampled populations; the different planned group sizes do not themselves indicate a difference in distribution.
Worked Example: Unequal Group Sizes
Hypothetical data. Separate random samples from two clinics report a preferred appointment channel.
| Clinic | Text | Phone | Total | |
|---|---|---|---|---|
| X | 20 | 12 | 8 | 40 |
| Y | 18 | 27 | 15 | 60 |
| Total | 38 | 39 | 23 | 100 |
The null hypothesis is that the channel-preference distribution is the same in both clinic populations; the alternative is that it differs. The random samples are independent, each person contributes once, and the three categories are mutually exclusive and exhaustive. The expected counts are all above 5, as shown below.
For Clinic X, the expected counts are \(40(38/100)=15.2\), \(40(39/100)=15.6\), and \(40(23/100)=9.2\). For Clinic Y, they are \(60(38/100)=22.8\), \(60(39/100)=23.4\), and \(60(23/100)=13.8\). The statistic is:
The degrees of freedom are \((2-1)(3-1)=2\), giving a right-tail p-value of approximately \(0.124\). At the 5% level, do not reject the null hypothesis. The sample does not provide sufficient evidence that the two clinic populations have different preference distributions. Unequal sample sizes are handled through the expected-count formula; they do not change the hypotheses or make the group distributions directly comparable by raw counts alone.
Common Mistakes and a Reliable Workflow
A common mistake is to compare the observed counts directly and conclude that the group with more observations in a category prefers it more. That reasoning ignores the number sampled from each group. In the unequal-size example, Clinic Y has more phone responses than Clinic X, but it also has 20 more sampled patients. The test instead compares observed counts with expected counts that account for both group size and the pooled category totals.
Another mistake is to interpret a small p-value as the probability that the null hypothesis is true. The p-value is calculated under the assumption that the null hypothesis is true: it is the probability of obtaining a chi-square statistic at least as large as the observed statistic. It is not a probability assigned to the null hypothesis. Similarly, failing to reject the null does not prove that the distributions match.
A disciplined analysis keeps each decision visible: identify the sampling design, state the common-distribution hypotheses, verify independence and expected counts, calculate the Pearson statistic, determine degrees of freedom, and interpret the right-tail p-value in context. The conclusion should answer the distribution-comparison question without claiming more than the test supports.
Check Your Understanding
Try these questions before moving on.
- A researcher samples patients separately from four clinics and records each patient’s preferred language for reminders. State the null and alternative hypotheses in context.
- A row total is 50, a column total is 36, and the grand total is 180. Calculate the expected count in that cell under the null hypothesis.
- A table has three groups and four outcome categories. Calculate the degrees of freedom for the Pearson chi-square test.
- Explain why a test of homogeneity uses a right-tail p-value.
- A test gives \(p=0.18\) at \(\alpha=0.05\). State an appropriate conclusion and one conclusion that would be too strong.