Tutorials › Introduction to Statistics › End-to-End Goodness-of-Fit Problem Workshop

Home  /  Tutorials  /  Chi-Square Goodness of Fit

Chi-Square Goodness of Fit · Tutorial 822 of 996

End-to-End Goodness-of-Fit Problem Workshop

Learn a reliable end-to-end approach for choosing, calculating, and interpreting a chi-square goodness-of-fit test.

Intermediate 12 min read

What You'll Learn

  • Set up null and alternative hypotheses for categorical goodness-of-fit problems.
  • Calculate expected counts and the Pearson chi-square statistic for uniform and nonuniform models.
  • Choose degrees of freedom when model parameters are estimated from the grouped counts.
  • Check when the chi-square approximation is suitable and identify alternatives when it is not.
  • Interpret a p-value in context and use residuals to locate categories contributing to a mismatch.

From a Probability Model to a Conclusion

A goodness-of-fit problem asks whether observed counts across one categorical variable are compatible with a specified probability model. The model might say that all categories are equally likely, assign different probabilities in advance, or estimate some probabilities from the data. A sound solution makes those distinctions explicit: they determine the expected counts and, when parameters are fitted, the degrees of freedom.

The usual Pearson chi-square method compares each observed count with the count expected under the null model. It summarizes the overall discrepancy, but it does not by itself say which categories account for that discrepancy. After the test, residuals can help locate the largest differences for investigation.

Definition: For \(k\) mutually exclusive and exhaustive categories, let \(O_i\) be the observed count and \(E_i\) the expected count in category \(i\) under the null model. The Pearson goodness-of-fit statistic is $$ \chi^2_{\text{GOF}}=\sum_{i=1}^{k}\frac{(O_i-E_i)^2}{E_i}. $$ When the null model specifies category probabilities \(p_1,\ldots,p_k\), the expected counts are \(E_i=np_i\), where \(n=\sum_{i=1}^{k}O_i\).

The null hypothesis, \(H_0\), states that the population category probabilities follow the proposed model. The alternative, \(H_a\), states that the probability distribution differs from that model. The test is an overall test: a small p-value is evidence against the probability vector as a whole, not a separate test for each category.

A Consistent Workflow

Before calculating, establish what probability model is being tested and how it was obtained. For a uniform model, \(p_i=1/k\) for every category. For a prespecified nonuniform model, the probabilities are supplied independently of the observed counts. For a fitted model, one or more parameters are estimated from the data; the estimation method and the data used for fitting matter for the reference distribution.

1
Define the categories and hypotheses.
Check that each observation belongs to exactly one category and that the categories cover all outcomes being analyzed. State the proposed probabilities under \(H_0\).
2
Check the sampling and approximation conditions.
Observations should be independent, the total \(n\) should be fixed for the multinomial model, and expected counts should be adequate for a chi-square approximation. A common guideline is that every \(E_i\) is at least 5, though the quality of the approximation depends on the full pattern of counts.
3
Compute expected counts and the statistic.
Use \(E_i=np_i\), then add the category contributions \((O_i-E_i)^2/E_i\).
4
Choose degrees of freedom and find the p-value.
For a fully specified probability vector, \(df=k-1\). If \(m\) independent parameters are estimated from the same grouped counts by an appropriate likelihood fit, the usual reference has \(df=k-1-m\), subject to regularity conditions.
5
Conclude in context and inspect the pattern.
Compare the p-value with the chosen significance level \(\alpha\). If the overall test is significant, examine residuals and contributions to see which categories merit further investigation.

The fixed-total condition means that the analysis concerns counts from \(n\) independent observations classified into categories, rather than a set of unrelated counts with no common sampling total. If observations are clustered or dependent, the ordinary multinomial model may not describe the sampling process, even if the arithmetic can be carried out.

Worked Example: Testing Equal Probabilities

Worked Example: Testing Equal Probabilities

Suppose a hypothetical researcher records which of four equally available routes 120 independently selected participants choose. The observed counts are 35, 28, 31, and 26. Test the claim that the four routes are equally likely using \(\alpha=0.05\).

Hypotheses and conditions. Under \(H_0\), each route has probability \(1/4\); \(H_a\) says the four probabilities are not all \(1/4\). The scenario specifies independent participants and a fixed total of 120. The categories are exhaustive and mutually exclusive. Each expected count is 30, so the usual expected-count guideline is satisfied.

Expected counts and statistic. For each route, \(E_i=120(1/4)=30\). Thus

$$ \chi^2_{\text{GOF}} =\frac{(35-30)^2}{30} +\frac{(28-30)^2}{30} +\frac{(31-30)^2}{30} +\frac{(26-30)^2}{30} =\frac{25+4+1+16}{30} =1.533. $$

No probabilities are estimated from the counts, so \(df=k-1=4-1=3\). The right-tail p-value is approximately 0.67. Since this exceeds 0.05, fail to reject \(H_0\). These hypothetical data do not provide strong evidence that route choices differ from equal probabilities. This is not proof that the routes are exactly equally likely.

Worked Example: Testing a Prespecified Nonuniform Model

Worked Example: Testing Prespecified Probabilities

A hypothetical system assigns outcomes to four categories with probabilities specified in advance as 0.50, 0.25, 0.15, and 0.10. In a run of 200 independent cases, the observed counts are 125, 35, 25, and 15. Test the stated model at \(\alpha=0.05\).

Hypotheses and conditions. Under \(H_0\), the four category probabilities are the prespecified values; \(H_a\) says the probability vector differs. The categories partition the outcomes, observations are assumed independent, and the total is fixed at 200. Because the model is specified in advance, its probabilities are not estimated from these observed counts.

Expected counts. Multiplying each probability by 200 gives \(E=(100,50,30,20)\). All expected counts are at least 5. The Pearson statistic is

$$ \chi^2_{\text{GOF}} =\frac{(125-100)^2}{100} +\frac{(35-50)^2}{50} +\frac{(25-30)^2}{30} +\frac{(15-20)^2}{20} =6.250+4.500+0.833+1.250 =12.833. $$

There are \(k=4\) categories and no fitted parameters, so \(df=4-1=3\). The right-tail p-value is approximately 0.0050. Since \(0.0050<0.05\), reject \(H_0\). The observed category distribution is inconsistent with the prespecified probability model at the 5% significance level; the test does not identify a cause for the discrepancy.

For a follow-up description, the signed Pearson residuals \((O_i-E_i)/\sqrt{E_i}\) are approximately 2.50, \(-2.12\), \(-0.91\), and \(-1.12\). The first two categories have the largest absolute residuals: the first count is above expectation and the second below. These are diagnostics that locate discrepancies, not independent post-hoc tests. Any formal category-by-category claims would need to account for the fact that several categories were inspected.

When Probabilities Are Fitted

A fitted model is different from a fully specified one because the observed data help determine its probabilities. For the standard \(k-1-m\) degrees-of-freedom reference, the \(m\) parameters should be estimated by fitting the model to the same grouped category counts used in the Pearson statistic, with suitable regularity conditions. An estimate based on finer, ungrouped observations can use information that the grouped counts do not contain; the usual degrees-of-freedom adjustment may then fail to calibrate the test.

The degrees-of-freedom reasoning is that the \(k\) observed proportions sum to one, leaving \(k-1\) independent directions in which they can differ from the model. Fitting \(m\) independent parameters uses \(m\) of those directions to bring the model closer to the observed proportions. Under the null and regularity conditions, the remaining discrepancy has an approximate chi-square distribution with \(k-1-m\) degrees of freedom. This dimension argument explains the adjustment; it is not a substitute for checking that the fitting method and model satisfy the conditions.

Degrees-of-freedom rule: For a fully specified model, use \(df=k-1\). For a regular model with \(m\) parameters estimated from the same grouped counts by maximum likelihood, the usual Pearson reference uses \(df=k-1-m\). The rule is not automatic for boundary estimates, unidentified parameters, or estimates obtained using information outside the grouped counts.

Worked Example: A Fitted Binomial Model

Worked Example: Estimating a Binomial Probability

In a hypothetical setting, each of 100 independent units has two opportunities for an event, each with event probability \(p\). The number of events per unit is classified as 0, 1, or 2. The observed counts are 28, 48, and 24. Fit a binomial model and assess its fit at \(\alpha=0.05\).

Model and estimate. Under the model, the category probabilities are \((1-p)^2\), \(2p(1-p)\), and \(p^2\). Estimate \(p\) by maximizing the likelihood of the three grouped counts. Ignoring factors that do not depend on \(p\), the log-likelihood is

$$ \ell(p)=2n_0\log(1-p)+n_1\log\!\big(2p(1-p)\big)+2n_2\log(p). $$

Differentiating and setting the derivative to zero gives

$$ \frac{d\ell}{dp} =-\frac{2n_0}{1-p} +n_1\left(\frac{1}{p}-\frac{1}{1-p}\right) +\frac{2n_2}{p}=0, \qquad \hat p=\frac{n_1+2n_2}{2(n_0+n_1+n_2)}. $$

Substituting the grouped counts gives \(\hat p=(48+2(24))/(2(100))=96/200=0.48\). This is an interior estimate; the grouped model is fitted to exactly the three counts being tested.

Expected counts and conditions. The fitted probabilities are \((1-0.48)^2=0.2704\), \(2(0.48)(0.52)=0.4992\), and \((0.48)^2=0.2304\). The expected counts are therefore 27.04, 49.92, and 23.04, all above 5. The hypothetical units are independent, the total is fixed, and the categories are exhaustive.

Statistic and conclusion. One parameter was estimated, so \(df=3-1-1=1\). The Pearson statistic is

$$ \chi^2_{\text{GOF}} =\frac{(28-27.04)^2}{27.04} +\frac{(48-49.92)^2}{49.92} +\frac{(24-23.04)^2}{23.04} =0.0341+0.0738+0.0400 =0.148. $$

The right-tail p-value for 1 degree of freedom is approximately 0.70. Since it exceeds 0.05, fail to reject the fitted binomial model. These counts show no meaningful evidence of lack of fit by this test; they do not establish that the binomial model is true.

Conditions, Interpretation, and Common Pitfalls

A small p-value means that, if the null model and sampling assumptions were correct, a statistic at least as large as the observed one would be unusual. It is not the probability that \(H_0\) is true. A large p-value means the data do not provide strong evidence against the model; it is not evidence that the model must be correct.

If expected counts are too small for a reliable chi-square approximation, the reported chi-square p-value may not be trustworthy. Depending on the problem, an exact multinomial test or a simulation-based calibration may be appropriate. Pooling categories is not a purely computational fix: it changes the question and should be done only when the combined categories make substantive sense.

  • Using observed counts as expected counts. Expected counts come from the null probabilities, \(E_i=np_i\), not from the observed pattern.
  • Forgetting the fitted-parameter adjustment. If parameters were estimated from the grouped counts, account for them in the degrees of freedom under the regular fitted-model conditions.
  • Applying the adjustment mechanically. Estimating a parameter from finer data and then testing coarser counts does not automatically justify \(k-1-m\); the estimation and test must be calibrated together.
  • Reading a significant result as a causal explanation. The test detects a mismatch with the model; it does not explain why the mismatch occurred.

For follow-up, report which categories had the largest residuals and whether they were above or below expectation. Then consider data quality, the substantive plausibility of the probability model, and whether the categories were defined appropriately before drawing a broader conclusion.

Key takeaway: A complete goodness-of-fit analysis identifies the model, checks the sampling and expected-count conditions, calculates expected counts and the Pearson statistic, uses degrees of freedom appropriate to any fitted parameters, and states a cautious conclusion in context.

Check Your Understanding

Try these questions using the definitions and workflow above.

  1. A model gives four category probabilities in advance, and 160 independent observations are classified into those categories. If the expected counts all meet the approximation guideline, what degrees of freedom are used?
  2. Why does estimating one parameter from the same grouped counts usually reduce the degrees of freedom by one?
  3. In a goodness-of-fit test, what does a small right-tail p-value indicate about the null model?
  4. Why might a parameter estimate based on ungrouped observations fail to justify the usual \(k-1-m\) reference distribution for grouped counts?
  5. After a significant overall test, what can signed Pearson residuals tell you, and what can they not establish by themselves?

Previous

Next

Back to Tutorials