Why Expected Counts Matter

A Pearson chi-square goodness-of-fit test compares observed category counts with the counts predicted by a proposed model. The calculation of the statistic is straightforward; the more consequential question is whether its reference distribution is a reasonable approximation. That depends in part on the expected counts, not just on the total sample size or the observed pattern.

If a model assigns very small probabilities to some categories, even a moderate sample may contain only a few observations in those categories on average. Their counts are highly discrete and can vary sharply relative to their means. In that setting, the usual continuous chi-square reference distribution may not give reliable p-values.

Definition: Suppose $n$ independent observations are classified into $k$ mutually exclusive and exhaustive categories. Under a specified null model, let $p_i$ be the probability of category $i$, where $\sum_{i=1}^k p_i=1$. The expected count in category $i$ is $e_i=np_i$. If $O_i$ is its observed count, the Pearson statistic is $$ X^2=\sum_{i=1}^k\frac{(O_i-e_i)^2}{e_i}. $$ The denominator is the expected count. The statistic is meaningful only for categories with positive model probability.

The expected counts describe what the null model predicts before seeing the sample. They are not the observed counts and should not be replaced by them in the formula or in an assessment of the approximation.

When the Chi-Square Reference Distribution Is Used

Under the null model, the vector of counts $(O_1,\ldots,O_k)$ has a multinomial distribution with sample size $n$ and category probabilities $p_1,\ldots,p_k$. When the expected counts are sufficiently large, the centered count vector is approximately normal. The Pearson statistic then has an approximate chi-square distribution. For a fully specified model with $k$ positive-probability categories, its usual degrees of freedom are $k-1$; estimating parameters from the data changes the degrees of freedom and may require additional care.

Set the null probabilities.
Use the model probabilities $p_i$ for the same categories used to calculate the test statistic.

Calculate expected counts.
For every category, compute $e_i=np_i$, then check that the expected counts sum to $n$.

Assess the approximation.
Compare the expected counts with common guidelines. If counts are sparse, do not assume the usual chi-square p-value is reliable merely because the statistic can be calculated.

Interpret with the right calibration.
When the large-sample approximation is doubtful, consider whether the analysis needs a justified alternative calibration, such as a suitable exact or simulation-based method.

A widely used rule of thumb is that no expected count should be below 1 and no more than 20% of expected counts should be below 5. A more conservative check is that every expected count is at least 5. These are guidelines, not universal mathematical guarantees. The reliability of an approximation can depend on the full probability pattern, the statistic, and how the model was specified or fitted.

Why this matters: The chi-square reference distribution is an approximation justified by large-sample behavior. Sparse expected counts can leave the actual count distribution too discrete or asymmetric for that approximation to represent its tail probabilities well. A computed statistic is not, by itself, evidence that the corresponding asymptotic p-value is accurate.

Why Sparse Categories Weaken the Approximation

The reason can be seen from the multinomial count vector. Its components add to $n$, so only $k-1$ count deviations can vary independently. With sufficiently large expected counts, a normal approximation to those deviations leads to the Pearson quadratic form and its chi-square reference distribution.

For a category with a small $e_i$, however, its count can take only a few small integer values near its mean. It cannot vary continuously or symmetrically in the way the normal approximation assumes. A rare category might have expected count $0.8$, for example: observing zero or one occurrence is unsurprising, while observing several occurrences is relatively unusual. A continuous chi-square curve may not reproduce the resulting discrete tail probabilities well.

E[O_i]=np_i=e_i,\qquad \operatorname{Var}(O_i)=np_i(1-p_i).

These formulas follow because $O_i$ is the sum of $n$ indicator variables, one for each observation being in category $i$. Each indicator has mean $p_i$ and variance $p_i(1-p_i)$; independence across observations makes the variances add. When $np_i$ is small, the count is concentrated on a few integers, which is precisely where a continuous large-sample approximation can be weak.

The issue is not that a small expected count automatically invalidates every test. Rather, it warns that the usual reference distribution may not calibrate the test's tail probabilities adequately. A p-value obtained from that approximation can then be misleading, even if the Pearson statistic itself has been calculated correctly.

Worked Example: Checking a Well-Populated Table

A hypothetical insurer classifies 160 reported service claims into four severity bands. Under a specified model, the probabilities are $0.50,\ 0.25,\ 0.15,\ 0.10$. The observed counts are $75,\ 43,\ 26,\ 16$. Assess the expected-count condition and calculate the Pearson statistic. The model probabilities are treated as fully specified.

Method. Calculate $e_i=np_i$ for each band, check the counts, and then evaluate the Pearson sum. This avoids judging the approximation from the observed counts alone.

The expected counts are:

(160)(0.50)=80,\quad (160)(0.25)=40,\quad (160)(0.15)=24,\quad (160)(0.10)=16.

They sum to $80+40+24+16=160$, as required. All four are at least 5, so this table meets the conservative expected-count guideline.

The observed counts also sum to $75+43+26+16=160$. The four Pearson contributions are:

\frac{(75-80)^2}{80}=\frac{25}{80}=0.3125,\quad \frac{(43-40)^2}{40}=\frac{9}{40}=0.225, $$ $$ \frac{(26-24)^2}{24}=\frac{4}{24}=0.1667,\quad \frac{(16-16)^2}{16}=0.

Thus $X^2=0.3125+0.225+0.1667+0=0.7042$, rounded to four decimal places. For a fully specified model with four categories, the usual reference degrees of freedom are $4-1=3$. The expected counts support using the conventional chi-square approximation; that check does not, on its own, establish that the null model is true.

Check. The four expected counts total 160, and the four observed counts also total 160. The Pearson contributions are nonnegative, and their sum is $0.7042$. A common mistake avoided here is treating a satisfactory count condition as proof of model fit: it only supports the approximation used to assess fit.

Rules of Thumb in Practice

For a quick diagnostic, count how many categories have expected counts below 5, and check whether any are below 1. Under the common 20% guideline, a table with $k$ categories should have no more than $0.20k$ expected counts below 5, as well as none below 1. Because the number of categories is an integer, apply that limit carefully; for example, with four categories, one count below 5 represents 25%, not 20%.

Always base this check on $np_i$. An observed count of zero does not mean its expected count is zero. Conversely, an observed count of 8 does not make a category suitable if the model expected only 1. Expected-count adequacy is a property of the null model and sample size, not a rating of whether the observed data look plausible.

Worked Example: A Rare Category in a Small Sample

A hypothetical equipment-protection plan classifies $n=50$ incidents into four mutually exclusive types. A specified model assigns probabilities $0.72,\ 0.18,\ 0.08,\ 0.02$. The observed counts are $35,\ 10,\ 4,\ 1$. Determine whether the common guideline is met and calculate $X^2$.

Method. First calculate all expected counts; do not infer them from the observed table. Then compare the counts with the rule that none be below 1 and no more than 20% be below 5.

The expected counts are:

50(0.72)=36,\quad 50(0.18)=9,\quad 50(0.08)=4,\quad 50(0.02)=1.

They sum to 50. Two of the four expected counts are below 5, so 50% are below 5. None is below 1, but 50% exceeds the 20% guideline. The table therefore does not meet that common condition.

The Pearson contributions are:

\frac{(35-36)^2}{36}=\frac{1}{36}=0.0278,\quad \frac{(10-9)^2}{9}=\frac{1}{9}=0.1111, $$ $$ \frac{(4-4)^2}{4}=0,\quad \frac{(1-1)^2}{1}=0.

Therefore $X^2=0.0278+0.1111=0.1389$, rounded to four decimal places. The calculation is valid, but a p-value taken automatically from the asymptotic chi-square reference distribution may not be well calibrated because half the expected counts are sparse.

Check. Observed counts total $35+10+4+1=50$, and expected counts total 50. The small statistic does not repair the failed count guideline. The common mistake avoided is to see a calculated statistic and proceed directly to the usual p-value without first checking whether its approximation is supported.

Sample Size and Expected Counts

Expected counts increase in direct proportion to $n$ when the category probabilities stay fixed. This gives a useful planning calculation: to make the expected count in a category with probability $p_i$ at least $c$, the sample size must satisfy $n\geq c/p_i$. Meeting such a threshold is a diagnostic, not a guarantee that every feature of an approximation is satisfactory.

Worked Example: Finding a Sample Size for a Rare Band

A hypothetical warranty study uses four loss bands with model probabilities $0.80,\ 0.10,\ 0.06,\ 0.04$. For the study as planned, $n=100$ observations produce counts $80,\ 10,\ 10,\ 0$. Assess the count condition, calculate the statistic, and find the smallest sample size for which every expected count would be at least 5.

Method. Compute expected counts at $n=100$, assess the guideline, and use the smallest category probability to determine the required sample size.

At $n=100$, the expected counts are $80,\ 10,\ 6,\ 4$. One of four is below 5, so 25% are below 5; none is below 1. The table misses the common 20% guideline.

Using the stated observed counts, the Pearson contributions are:

\frac{(80-80)^2}{80}=0,\quad \frac{(10-10)^2}{10}=0,\quad \frac{(10-6)^2}{6}=\frac{16}{6}=2.6667,\quad \frac{(0-4)^2}{4}=\frac{16}{4}=4.

Thus $X^2=2.6667+4=6.6667$, rounded to four decimal places. The zero observed count has a contribution of 4 because the expected count is 4; it does not make the denominator zero.

The smallest category probability is $0.04$. To have its expected count at least 5, require $0.04n\geq5$, so $n\geq125$. At $n=125$, the expected counts are $100,\ 12.5,\ 7.5,\ 5$; all are at least 5.

Check. The observed counts total 100 and the expected counts at the planned size also total 100. At $n=125$, the expected counts total 125. A common mistake avoided is dividing by the observed zero in the last category: the Pearson denominator is its expected count, 4.

Common Mistakes and Interpretation

Checking observed rather than expected counts. Use $e_i=np_i$ to assess the approximation. Observed counts are used in the numerator of the Pearson statistic.
Assuming a large total sample resolves every sparse category. A large $n$ can still leave a small expected count when some $p_i$ is very small.
Treating a rule of thumb as an absolute theorem. The familiar count guidelines are practical diagnostics. Passing them supports, but does not prove, accurate calibration.
Confusing a computable statistic with a reliable p-value. The Pearson sum can be calculated even when the chi-square reference approximation is questionable.
Using a zero observed count as a zero denominator. The denominator remains the positive expected count. If the model probability itself is zero, the model and category structure require attention before applying the usual statistic.

When expected counts are sparse, the appropriate response is not to conceal the issue or report an asymptotic p-value without qualification. The analysis may require a justified exact or simulation-based calibration, or a revised category design. Whatever approach is used, its assumptions should be stated. The key diagnostic remains the same: evaluate the model's expected counts before relying on the usual chi-square reference distribution.

Key takeaway: Expected counts determine whether the large-sample chi-square approximation is plausible. Calculate $e_i=np_i$, inspect the whole set of category counts, and treat sparse-count p-values cautiously rather than assuming that a valid Pearson sum guarantees a valid reference distribution.

Check Your Understanding

For each problem, use the expected counts—not the observed counts—to assess the approximation.

A hypothetical model has three categories with probabilities $0.50,\ 0.30,\ 0.20$, and the sample size is 120. Find the expected counts and state whether all are at least 5. (Answer: $60,\ 36,\ 24$; yes.)
A specified model has probabilities $0.85,\ 0.10,\ 0.04,\ 0.01$, with $n=80$. Find the expected counts and assess the common guideline. (Answer: $68,\ 8,\ 3.2,\ 0.8$; it fails because one expected count is below 1 and two of four are below 5.)
A category has null probability $0.025$. What is the smallest integer sample size that gives this category an expected count of at least 5? (Answer: $n=200$.)
For $n=20$, a three-category model has probabilities $0.50,\ 0.40,\ 0.10$, and observed counts $10,\ 10,\ 0$. Calculate the expected counts and Pearson statistic, then comment on the approximation. (Answer: expected counts $10,\ 8,\ 2$; $X^2=2.5$; one of three expected counts is below 5, so the common 20% guideline is not met.)
A hypothetical model has five category probabilities $0.70,\ 0.15,\ 0.08,\ 0.05,\ 0.02$, with $n=300$. Do the expected counts satisfy both parts of the common guideline? (Answer: $210,\ 45,\ 24,\ 15,\ 6$; yes, all are at least 5 and none is below 1.)

Why Expected Counts Matter

What You'll Learn