Why Conditions Matter
A chi-square statistic can always be calculated from a table of observed and expected counts. But a calculated statistic does not automatically have the chi-square distribution used to obtain an approximate p-value. That approximation is dependable only when the data and study design meet important conditions. Checking them is part of the analysis, not an optional step after the calculation.
For tests of independence and homogeneity, the central checks concern the observational units, the way categories are recorded, and the expected cell counts under the null hypothesis. A failure in any of these areas can make the usual chi-square p-value unreliable, even if the arithmetic is correct.
Check the Observational Units
Independence is about the units contributing observations. In a test of independence, each sampled unit contributes a value for each of two categorical variables. In a test of homogeneity, units in the different groups or samples must be independent in the way assumed by the study design. In either case, a person, animal, or other unit should not be counted as though it were several unrelated units when its observations are linked.
A random sample of distinct people from a large population often supports the independence condition. If sampling without replacement from a finite population, a common safeguard is that the sample be no more than about 10% of that population. For a designed study, the allocation and measurement plan also matter: repeated measurements on the same subject, matched pairs, or clusters of related subjects can create dependence that an ordinary chi-square test does not account for.
Independence of observational units does not mean that the cell counts in the table are independent random variables. For example, once the row and column totals are fixed, changing one cell count constrains others. The requirement concerns the underlying observational units and study design, not the arithmetic relationships among table cells.
Worked Example: Checking Independence in a Survey
Suppose a hypothetical survey randomly selects 240 adults from a large city. Each adult is asked whether they live in an urban or rural area and whether they use a particular public service. Each person answers once, and the survey includes no more than 10% of the city’s adult population. Is the independence condition plausible?
Each of the 240 adults is one observational unit and contributes to exactly one row and one column of the table. The sample is random, the units are distinct, and each person is measured once. The sample is also less than 10% of the finite population, so sampling without replacement is not expected to create substantial dependence.
On these stated facts, independence is plausible. The word “plausible” matters: the check depends on how the sample was actually selected and recorded. If several people came from the same household and shared relevant circumstances, that clustering could also matter even though no person was counted twice.
Check That the Table Contains Valid Counts
A two-way table for an ordinary chi-square test should summarize counts of observational units in categories. Each unit must be assigned to one category of each variable, and the categories should be mutually exclusive and collectively cover the units being analyzed. Thus, a person who selects exactly one response from a complete list can contribute one count to one cell.
Percentages, averages, and repeated mentions are not themselves counts of independent units. A table of percentages can conceal different sample sizes across groups; treating its percentages as if they were counts can change the analysis. If the original sample sizes and percentages are known, the underlying counts may sometimes be recovered, subject to rounding. The test should then be based on the actual counts, not on the displayed percentages as though they were frequencies.
Also check how missing answers, overlapping categories, and structural impossibilities were handled. If an individual could be counted in multiple cells, the entries are not the ordinary one-unit-per-cell counts required here. A category that is impossible by design is not simply an observed zero; the table and its analysis must reflect that design feature rather than treating the impossible cell as an ordinary outcome.
Worked Example: Use Counts, Not Percentages as Frequencies
In a hypothetical comparison, 50 people in Group A and 150 people in Group B each choose exactly one of two service options. Group A has 30 choosing Option 1 and 20 choosing Option 2. Group B has 60 choosing Option 1 and 90 choosing Option 2. The corresponding percentages are 60% and 40% in Group A, and 40% and 60% in Group B.
The valid table entries are the counts: 30, 20, 60, and 90. The table total is \(n=200\); the row totals are 50 and 150, and the column totals are 90 and 110. Under the null hypothesis that the option distribution is the same in both groups, the expected counts are:
The expected counts are \(50(90)/200=22.5\), \(50(110)/200=27.5\), \(150(90)/200=67.5\), and \(150(110)/200=82.5\). Each is at least 5. Since each participant selected exactly one option and contributes once, the table also has valid categorical counts, assuming the participants were sampled independently within the groups.
Entering the four percentages 60, 40, 40, and 60 as though they were counts would discard the unequal group sizes and analyze a different table. The reported percentages describe the groups; they do not replace the underlying frequencies.
Check Expected Cell Counts
The expected count \(E_{ij}\) is the count predicted for cell \(i,j\) under the null hypothesis, using the table margins. For a test of independence or homogeneity, it is calculated as the row total multiplied by the column total and divided by the grand total. It is not the count actually observed in that cell.
The expected-count formula follows from the null model. Under independence, the chance of belonging to row \(i\) and column \(j\) is the product of the chance of row \(i\) and the chance of column \(j\). Estimating those marginal chances by their observed proportions gives:
This calculation gives a benchmark for the observed counts if the null model holds. When expected counts are very small, the possible observed counts are highly discrete and can be far from the smooth chi-square approximation. The resulting approximate p-value may then be inaccurate. Adequate expected counts help justify the approximation; they do not establish that the null hypothesis is true.
There is no single expected-count cutoff that is a mathematical guarantee for every table and every purpose. A conservative and commonly taught check is that every expected count be at least 5. Another widely used guideline allows at most 20% of expected counts to be below 5, provided none is below 1. These are practical guidelines, not universal boundaries at which an approximation suddenly becomes valid or invalid. For a small table, especially a \(2\times2\) table, requiring all expected counts to be at least 5 is a clear conservative choice.
Worked Example: A Table With Too Many Small Expected Counts
A hypothetical survey records one response from each of 100 people. The observed counts and margins are:
| Option A | Option B | Option C | Row total | |
|---|---|---|---|---|
| Group 1 | 0 | 1 | 9 | 10 |
| Group 2 | 1 | 2 | 17 | 20 |
| Group 3 | 4 | 12 | 54 | 70 |
| Column total | 5 | 15 | 80 | 100 |
For Group 1 and Option A, the expected count under the null hypothesis is \(10(5)/100=0.5\). The nine expected counts, computed by the same formula, are:
Five of the nine expected counts are below 5, so about 56% are below 5; one is also below 1. This table fails both the all-at-least-5 guideline and the guideline allowing no more than 20% below 5 with none below 1. The ordinary chi-square approximation should not be treated as reliable on the basis of those guidelines. This judgment concerns the approximation, not whether the observed pattern is interesting.
A Large Expected Count Cannot Fix Dependence
The checks are separate. Large expected counts do not repair a study design with dependent observations, and independent observations do not guarantee that the expected counts are adequate. Each condition must be assessed on its own.
Worked Example: Repeated Measurements Despite Adequate Counts
In a hypothetical study, 40 patients each have a binary outcome recorded before and after a program. Before the program, 16 outcomes are Yes and 24 are No; after the program, 24 are Yes and 16 are No. The table has two rows, one for each time, and two columns for the outcome.
If the 80 measurements were incorrectly treated as independent records, the row totals would be 40 and 40, the column totals 40 and 40, and \(n=80\). Each expected count would be \(40(40)/80=20\), which comfortably meets common expected-count guidelines.
However, each patient contributes to both rows. The two measurements from a patient are paired, so the 80 records are not 80 independent observational units. The usual chi-square test of independence would ignore this link and would not be justified simply because every expected count is 20. The analysis must use a method suited to paired categorical data rather than treating the repeated measurements as unrelated.
What to Do When a Condition Fails
A condition check is a decision about whether the proposed chi-square approximation is appropriate, not an invitation to adjust data until it passes. If observations are dependent, first describe the dependence and use an analysis that matches the design. If the categories do not produce valid counts, clarify what each unit contributes and how categories were defined. If expected counts are sparse, the usual approximate p-value may not be trustworthy; an appropriate exact or simulation-based approach may be considered, or the study may need more data.
Do not combine categories solely to increase expected counts if doing so would erase a meaningful distinction or change the research question. Any category combination needs a substantive justification and should be consistent with the purpose of the analysis. Nor does meeting all the checks prove the sampling was unbiased, the variables were measured well, or the conclusion is causal. The conditions support a particular approximation; they do not validate every aspect of a study.
Common Mistakes
- Checking observed counts instead of expected counts. A large observed count does not show that the null-model approximation is adequate. Calculate expected counts from the margins.
- Assuming adequate expected counts guarantee validity. A table can have every expected count above 5 and still contain repeated or clustered observations. Check the design separately.
- Calling the cell counts independent. The condition concerns the underlying observational units. Cell counts are constrained by the table totals even when the units are independent.
- Treating percentages as counts. Percentages alone do not preserve different sample sizes. Use the actual number of units in each cell.
- Treating a guideline as an absolute theorem. Expected-count cutoffs are practical checks on an approximation. Report the guideline used and be cautious when a table is sparse.
Check Your Understanding
For each question, explain which condition is relevant and what the stated facts allow you to conclude.
- A random sample of distinct adults is surveyed once, and every adult selects exactly one of three response categories. What facts support independent observations and valid counts?
- In a \(2\times3\) table, one expected count is 0.8 and the other five exceed 5. What does this suggest about using the ordinary chi-square approximation?
- A table contains 75% and 25% for each group, but the group sample sizes differ. Why should these percentages not simply be entered as counts?
- Each participant contributes one response before and one after an intervention. Which condition needs special attention, even if every expected count is large?
- Explain why independent observational units do not require the cell counts in a two-way table to be independent of one another.