Testing Whether Categories Are Equally Likely

When Is an Equal-Probability Model Appropriate?

Some categorical processes have a natural claim of equal likelihood: a fair die has six faces, a spinner may have four equal sectors, or a survey may offer response options that are intended to be selected equally often under a particular model. Data rarely fall exactly evenly across categories, even when the model is correct. The question is whether the differences between observed counts and equal expected counts are large enough to provide evidence against that model.

A chi-square goodness-of-fit test addresses this question. It compares observed category counts with the counts expected if every category had the same probability. The test evaluates the overall mismatch; it does not assume in advance which category, if any, will occur more often.

Definition: For \(k\) mutually exclusive and exhaustive categories, a uniform probability model assigns probability \(1/k\) to each category. If \(n\) independent observations are made under this model, the expected count in each category is \(n/k\).

Hypotheses and Expected Counts

Let \(p_i\) denote the probability of category \(i\), for \(i=1,\ldots,k\). For a test of whether the categories are equally likely, the hypotheses are:

H_0: p_1=p_2=\cdots=p_k=\frac{1}{k}

H_a: \text{at least one } p_i \ne \frac{1}{k}

The alternative is two-sided in the sense that a category can occur either more or less often than expected. It does not specify a particular category or direction. If \(O_i\) is the observed count for category \(i\), the expected count under the null model is \(E_i=n/k\). The observed counts must sum to \(n\), and the expected counts also sum to \(n\).

For a fixed uniform model with \(k\) categories, the Pearson goodness-of-fit statistic is

\chi^2=\sum_{i=1}^{k}\frac{(O_i-E_i)^2}{E_i}

Large values indicate that the observed counts are far from the counts predicted by the null model. The reference distribution is chi-square with \(df=k-1\). The test p-value is the probability, under \(H_0\), of obtaining a statistic at least as large as the observed statistic. It is calculated from the right tail of that chi-square distribution.

Why the Uniform Model Gives These Counts

Under \(H_0\), each observation has probability \(1/k\) of falling in any particular category. Across \(n\) observations, the expected count for that category is therefore \(n(1/k)=n/k\). This expected value need not be a whole number: it is a model-based average count, not a prediction that each actual category count must be an integer equal to it.

For a uniform model, the Pearson statistic can also be written in a compact form. Substituting \(E_i=n/k\) into the definition gives

\chi^2=\sum_{i=1}^{k}\frac{(O_i-n/k)^2}{n/k} =\frac{k}{n}\sum_{i=1}^{k}(O_i-n/k)^2

Thus, the statistic grows when observed counts are farther from the common expected count. The squared deviations ensure that a category above expectation does not cancel a category below expectation. The sum of the deviations themselves is always zero, because both the observed and expected counts add to \(n\); squaring is what makes the statistic measure overall discrepancy.

The degrees of freedom are \(k-1\). Once \(k-1\) category counts are known, the final count is determined by the fixed total \(n\). The chi-square reference distribution and this degrees-of-freedom rule are used here for a fixed probability model; the probabilities are not estimated from the observed category counts.

Worked Examples

Worked Example: Is a Die Consistent With Equal Face Probabilities?

In a hypothetical test, a die is rolled 120 times. The face counts are 14, 18, 22, 17, 25, and 24. Test whether the data provide evidence that the six faces are not equally likely. Use \(\alpha=0.05\).

Model and conditions. The six faces are mutually exclusive and exhaustive outcomes. Assume the rolls are independent and the die’s probabilities are fixed across rolls. The null model assigns probability \(1/6\) to each face. Each expected count is \(E_i=120/6=20\), which is at least 5, so the usual chi-square approximation condition is met.

Hypotheses. \(H_0\) states that each face has probability \(1/6\); \(H_a\) states that at least one face has a different probability. The observed counts sum to 120, as required.

Statistic. Substituting the observed and expected counts into Pearson’s formula,

\chi^2= \frac{(14-20)^2}{20}+ \frac{(18-20)^2}{20}+ \frac{(22-20)^2}{20}+ \frac{(17-20)^2}{20}+ \frac{(25-20)^2}{20}+ \frac{(24-20)^2}{20}

\chi^2=\frac{36+4+4+9+25+16}{20} =\frac{94}{20}=4.70

There are \(df=6-1=5\) degrees of freedom. The right-tail p-value for \(\chi^2=4.70\) with 5 degrees of freedom is approximately 0.453. Since \(0.453>0.05\), fail to reject \(H_0\). These 120 rolls do not provide statistically significant evidence that the die’s face probabilities differ from \(1/6\). This result does not prove the die is fair; it means the observed discrepancies are not unusually large under the equal-probability model.

Worked Example: Testing a Four-Sector Spinner

A hypothetical spinner has four labeled sectors, each designed to occupy one quarter of the circle. It is spun 80 times, producing counts of 28, 17, 19, and 16. Test whether the outcomes are consistent with equal probabilities at \(\alpha=0.05\).

Model and conditions. The four sectors form the possible outcomes, and each spin is assumed independent under a stable probability model. The null probability for each sector is \(1/4\). The expected count is \(E_i=80/4=20\) in every sector, meeting the usual expected-count condition. The observed counts sum to 80.

Hypotheses. \(H_0\) assigns probability \(1/4\) to each sector. \(H_a\) says that at least one sector’s probability is not \(1/4\).

Statistic and reference distribution.

\chi^2= \frac{(28-20)^2+(17-20)^2+(19-20)^2+(16-20)^2}{20} =\frac{64+9+1+16}{20} =\frac{90}{20}=4.50

Here \(df=4-1=3\). The right-tail p-value is approximately 0.212. Because \(0.212>0.05\), fail to reject \(H_0\). The results do not give sufficient evidence at the 5% significance level that the spinner’s four outcomes have unequal probabilities. In particular, the count of 28 for the first sector is not considered alone: the test assesses the complete pattern of four counts.

Worked Example: Equal Likelihood of Survey Response Options

Suppose a hypothetical survey asks 200 randomly selected participants to choose one of five response options. A specified null model treats all five options as equally likely. The observed counts are 31, 38, 44, 47, and 40. Test this model using \(\alpha=0.05\).

Model and conditions. Each participant contributes one response, the five options are mutually exclusive and cover all possible answers, and responses are assumed independent. The test evaluates the stated equal-probability model; it does not assume that real preferences must be equal. Under \(H_0\), each expected count is \(200/5=40\), which is at least 5. The counts sum to 200.

Hypotheses and calculation. \(H_0\) states that each option has probability \(1/5\). \(H_a\) states that at least one option has a different probability. The statistic is

\chi^2= \frac{(31-40)^2+(38-40)^2+(44-40)^2+(47-40)^2+(40-40)^2}{40}

\chi^2=\frac{81+4+16+49+0}{40} =\frac{150}{40}=3.75

The degrees of freedom are \(df=5-1=4\), and the right-tail p-value is approximately 0.441. Since \(0.441>0.05\), fail to reject \(H_0\). Under the stated assumptions, the data do not provide statistically significant evidence that response options have unequal probabilities. This conclusion concerns compatibility with the specified model; it does not establish that respondents truly have equal preferences or explain why the counts differ.

Conditions and Interpretation

A numerical chi-square result is useful only when the design and model make its reference distribution appropriate. Before interpreting the p-value, check the following:

One categorical variable: each observational unit contributes a count to exactly one of the \(k\) categories.
Complete categories: the categories are mutually exclusive and exhaustive for the outcomes being analyzed.
Independent observations: one observation’s outcome does not determine or strongly constrain another’s. For example, repeated outcomes from a process should reasonably be treated as independent under the model.
Fixed null probabilities: the test compares the counts with equal probabilities specified in advance, rather than probabilities estimated from these same counts.
Adequate expected counts: the usual chi-square approximation is used when expected counts are sufficiently large; a common introductory condition is that every expected count is at least 5.

When these conditions are plausible, select a significance level \(\alpha\), compute \(\chi^2\), and compare the right-tail p-value with \(\alpha\). A p-value at or below \(\alpha\) is evidence against the equal-probability model. A p-value above \(\alpha\) means the data do not give enough evidence to reject it; it is not evidence that the probabilities have been proven equal.

Common Mistake: Treating “Not Significant” as “Equal”

A common mistake is to say, “The p-value is above 0.05, so all categories are equally likely.” That conclusion is too strong. The test can fail to detect differences because the probabilities are in fact equal, but it can also fail to detect real differences when the sample provides limited information. The appropriate conclusion is that the data do not provide sufficient evidence against the uniform model at the chosen significance level.

Another mistake is to calculate \(n/k\) for only one category and then use a different expected count for the others. Under the uniform null, every category has the same expected count. For example, with 80 spins and four categories, all four expected counts are \(80/4=20\), regardless of the observed counts.

Key takeaway: To test whether \(k\) categories are equally likely, compare each observed count with \(n/k\), calculate \(\chi^2=\sum (O_i-E_i)^2/E_i\), and use a chi-square distribution with \(k-1\) degrees of freedom. A small right-tail p-value signals evidence that the uniform model does not fit well.

Check Your Understanding

Answer each question using the equal-probability goodness-of-fit framework.

A hypothetical experiment has 60 independent outcomes divided among three categories. Under a uniform null model, what is the expected count in each category?
For five response categories and a fixed equal-probability model, state \(H_0\), \(H_a\), and the degrees of freedom.
Why does a goodness-of-fit test use squared differences between observed and expected counts rather than simply adding the differences?
A test produces a p-value of 0.12 at \(\alpha=0.05\). State a suitable conclusion without claiming that the category probabilities are proven equal.
Name two conditions that should be checked before using the chi-square approximation for a uniform-model test.