Tutorials › Introduction to Statistics › Using Residuals to Locate Misfitting Categories

Home  /  Tutorials  /  Chi-Square Goodness of Fit

Chi-Square Goodness of Fit · Tutorial 807 of 996

Using Residuals to Locate Misfitting Categories

Use signed residuals to find categories driving a goodness-of-fit mismatch, and control false alarms when examining several categories.

Intermediate 10 min read

What You'll Learn

  • Calculate Pearson residuals and connect them to category contributions to the chi-square statistic.
  • Distinguish Pearson residuals from residuals standardized using each category’s marginal standard deviation.
  • Interpret the sign and size of a category residual relative to a specified probability model.
  • Use Bonferroni-adjusted comparisons to account for examining multiple categories.
  • Explain why category residuals are dependent and why an unusual residual is a diagnostic, not a causal explanation.

Why Examine Category Residuals?

A goodness-of-fit test can indicate that observed counts do not fit a specified probability model, but its overall statistic does not by itself identify which categories differ most from what the model predicts. Category residuals help answer that follow-up question: which categories have more or fewer observations than expected?

A residual is a signed difference between an observed count and its expected count, scaled to express the size of the difference relative to its variability. Its sign shows the direction of the discrepancy: positive means more observations than expected, and negative means fewer. Because examining several categories creates several opportunities for a chance finding, the comparisons also need to account for multiplicity.

Two Useful Forms of a Category Residual

Consider a one-way goodness-of-fit model with \(k\) mutually exclusive categories. Under the null hypothesis, category \(i\) has specified probability \(p_i\), with \(\sum_{i=1}^{k}p_i=1\). For a sample of size \(n\), let \(O_i\) be the observed count and \(E_i=np_i\) the expected count in category \(i\). The Pearson chi-square contribution from category \(i\) is \((O_i-E_i)^2/E_i\).

Definition: The Pearson residual for category \(i\) is \(r_i=(O_i-E_i)/\sqrt{E_i}\). It is signed, and its square is exactly that category’s contribution to the Pearson chi-square statistic. A positive value indicates an excess relative to expectation; a negative value indicates a deficit.

The Pearson residual is useful for locating categories that contribute substantially to the overall chi-square statistic. However, because the total sample size is fixed, category counts are not independent: an excess in one category must be balanced by deficits elsewhere. The marginal standard deviation of \(O_i\), considered one category at a time under the null, is \(\sqrt{np_i(1-p_i)}\), not \(\sqrt{E_i}\). Dividing by this marginal standard deviation gives a residual that is approximately on a standard-normal scale when the category and its complement have adequate expected counts.

Definition: The marginally standardized residual for category \(i\) is \(z_i=(O_i-E_i)/\sqrt{np_i(1-p_i)}=(O_i-E_i)/\sqrt{E_i(1-p_i)}\). Under the fixed-probability null model, it is approximately standard normal when the normal approximation for the category count is adequate. Its sign gives the direction of the difference, and its magnitude is measured in approximate marginal standard deviations.

These two residuals answer related but distinct questions. Pearson residuals square to the category contributions that add to the overall statistic. Marginally standardized residuals are more suitable for approximate category-by-category comparisons with standard-normal cutoffs. They should not be treated as independent tests: the category counts, and therefore their residuals, are linked by the fixed total.

Worked Example: Separating Excesses From Deficits

Worked Example: Separating Excesses From Deficits

Suppose 200 hypothetical observations are classified into three categories. A specified model assigns probabilities \(0.50\), \(0.30\), and \(0.20\); the observed counts are 116, 48, and 36. Calculate both residual forms and identify the largest discrepancies.

The expected counts are \(E_i=np_i\): \(200(0.50)=100\), \(200(0.30)=60\), and \(200(0.20)=40\). Each observed-minus-expected difference is therefore \(16\), \(-12\), and \(-4\).

For the Pearson residuals, divide each difference by the square root of its expected count:

$$ r_1=\frac{116-100}{\sqrt{100}}=1.600,\qquad r_2=\frac{48-60}{\sqrt{60}}=-1.549,\qquad r_3=\frac{36-40}{\sqrt{40}}=-0.632 $$

Category 1 has more observations than expected, while categories 2 and 3 have fewer. The corresponding chi-square contributions are the squared residuals: \(2.560\), \(2.400\), and \(0.400\). They sum to \(5.360\), the overall Pearson statistic for these counts. Categories 1 and 2 contribute most to that statistic.

For the marginally standardized residuals, use the null probabilities in the denominator:

$$ z_1=\frac{16}{\sqrt{200(0.50)(0.50)}}=2.263,\qquad z_2=\frac{-12}{\sqrt{200(0.30)(0.70)}}=-1.852,\qquad z_3=\frac{-4}{\sqrt{200(0.20)(0.80)}}=-0.707 $$

On this scale, category 1 has the largest standardized discrepancy. These calculations describe differences relative to the specified model; they do not, by themselves, establish that any category is statistically unusual after accounting for the fact that three categories were examined.

Accounting for Multiple Category Comparisons

If each category is examined separately at level \(\alpha\), the chance of at least one false positive across the set of comparisons can exceed \(\alpha\). A simple way to control this familywise error rate—the probability of one or more false rejections in the family—is the Bonferroni procedure. For \(m\) comparisons, compare each two-sided p-value with \(\alpha/m\), or equivalently require each absolute standardized residual to exceed the standard-normal cutoff with two-sided tail area \(\alpha/m\).

Procedure: For \(m\) planned category comparisons and familywise level \(\alpha\), use a per-comparison level \(\alpha/m\). For approximate two-sided normal comparisons, flag category \(i\) only if \(|z_i|>z_{1-\alpha/(2m)}\), where \(z_q\) is the \(q\)-quantile of the standard normal distribution. Equivalently, multiply each unadjusted two-sided p-value by \(m\), capping the adjusted value at 1, and compare it with \(\alpha\).

The reason Bonferroni controls the familywise error rate does not require the comparisons to be independent. The probability of at least one false rejection is no greater than the sum of the probabilities of the individual false rejections. If each of \(m\) comparisons has false-rejection probability at most \(\alpha/m\), that sum is at most \(m(\alpha/m)=\alpha\). This control can be conservative, especially when category counts are dependent, but it provides a straightforward safeguard.

The overall goodness-of-fit test and the category comparisons have different roles. The overall test asks whether the count distribution departs from the specified model at all. The adjusted comparisons help describe where the observed discrepancies lie while limiting false alarms across the chosen family of category checks. A small overall p-value does not imply that every category differs, and an unadjusted category p-value should not be presented as evidence from a family of comparisons without noting the multiplicity.

Worked Example: Applying a Bonferroni Adjustment

In a hypothetical sample of 400 observations, the model probabilities are \(0.50\), \(0.30\), and \(0.20\), and the observed counts are 240, 100, and 60. At familywise level \(\alpha=0.05\), assess all three categories using marginally standardized residuals and a Bonferroni adjustment.

The expected counts are 200, 120, and 80. The differences are 40, \(-20\), and \(-20\). The marginal standard deviations under the model are \(\sqrt{400(0.50)(0.50)}=10\), \(\sqrt{400(0.30)(0.70)}=\sqrt{84}\), and \(\sqrt{400(0.20)(0.80)}=8\). Thus,

$$ z_1=\frac{40}{10}=4.000,\qquad z_2=\frac{-20}{\sqrt{84}}=-2.182,\qquad z_3=\frac{-20}{8}=-2.500 $$

There are \(m=3\) category comparisons, so the Bonferroni per-comparison level is \(0.05/3=0.0167\). For two-sided standard-normal comparisons, the corresponding cutoff is approximately \(z_{1-0.05/(2\cdot3)}=2.394\). Categories 1 and 3 exceed this cutoff in absolute value; category 2 does not. In direction and context, category 1 has more observations than its model expectation, and category 3 has fewer. Category 2’s deficit is not sufficiently pronounced under this adjusted comparison rule.

For comparison, the overall Pearson statistic is

$$ \chi^2=\frac{40^2}{200}+\frac{(-20)^2}{120}+\frac{(-20)^2}{80} =8.000+3.333+5.000=16.33 $$

With \(df=2\), its chi-square p-value is approximately \(0.000284\), indicating an overall mismatch with the specified model. The category comparisons add detail about the direction and location of that mismatch; they are not replacements for the overall test.

Worked Example: When Only One Category Stands Out

Worked Example: When Only One Category Stands Out

A hypothetical sample of 400 observations is split among four equally likely categories under a specified model. The observed counts are 130, 90, 90, and 90. Examine all four categories with a familywise level of \(0.05\).

Each expected count is \(400(0.25)=100\), and each category’s marginal standard deviation is \(\sqrt{400(0.25)(0.75)}=\sqrt{75}=8.660\). The marginally standardized residuals are

$$ z_1=\frac{130-100}{\sqrt{75}}=3.464,\qquad z_2=z_3=z_4=\frac{90-100}{\sqrt{75}}=-1.155 $$

For \(m=4\), the per-comparison level is \(0.05/4=0.0125\), and the two-sided normal cutoff is approximately \(z_{1-0.05/(2\cdot4)}=2.498\). Only category 1 exceeds the cutoff. Its observed count is higher than expected; the other three categories each have fewer observations than expected, but none is individually flagged by this adjusted procedure.

The Pearson contributions are \(30^2/100=9\) for category 1 and \((-10)^2/100=1\) for each other category, giving \(\chi^2=12.00\) with \(df=3\). Thus category 1 accounts for most of the statistic. The example also shows why signs and magnitudes should be considered together: the three negative differences are real count differences, but they are smaller on the standardized scale than the excess in category 1.

Conditions and Interpretation

The category-level normal approximations rely on the same basic setting as the fixed-probability goodness-of-fit test: observations are independent, each observation belongs to exactly one of the specified categories, and the null probabilities are specified in advance. The expected counts must be adequate for chi-square calibration, and a normal approximation for a particular category count also depends on having enough expected observations both in that category and outside it. When these approximations are poor, a normal cutoff for residuals may be unreliable.

A flagged residual means that, under the stated model and approximation, that category’s count is unusually high or low relative to its null expectation after the chosen multiplicity adjustment. It does not identify why the count differs, prove that the model is wrong in a particular substantive way, or establish a causal explanation. Report the observed and expected counts, the direction, the adjustment used, and the comparison family so readers can interpret the finding accurately.

Common Mistake

Pitfall: “The category has the largest residual, so it is significantly different because its unadjusted p-value is below 0.05.”

Correction: A largest residual is a relative ranking, not a significance conclusion. When several categories are examined, use a stated multiple-comparison procedure such as Bonferroni, and interpret each result as a comparison with the specified null probability model.

Another frequent confusion is to call every Pearson residual a standard-normal score. The Pearson residual uses \(\sqrt{E_i}\) and is designed so its square equals a chi-square contribution. For an approximate category-level normal comparison under a fixed total, use the marginal standard deviation \(\sqrt{E_i(1-p_i)}\), then account for the family of comparisons.

Key takeaway: A signed residual identifies both the direction and relative size of a category’s departure from its expected count. Pearson residuals decompose the overall chi-square statistic; marginally standardized residuals support approximate category comparisons. Adjust those comparisons for multiplicity before describing a category as unusual.

Check Your Understanding

Try these questions without looking back.

  1. In a fixed-probability goodness-of-fit model, what does a positive Pearson residual say about a category’s observed count?
  2. Why does the square of a Pearson residual equal that category’s contribution to the Pearson chi-square statistic?
  3. For a category with null probability \(p_i\), why is the marginal standard deviation of its count \(\sqrt{np_i(1-p_i)}\) rather than \(\sqrt{np_i}\)?
  4. If six category comparisons are made at familywise level \(0.05\), what per-comparison level does the Bonferroni procedure use?
  5. Why can a large unadjusted category residual be insufficient evidence that the category is unusual after several categories have been examined?

Previous

Next

Back to Tutorials