Testing an Unequal Probability Vector

Why Test an Unequal Probability Vector?

A goodness-of-fit question does not always ask whether categories are equally likely. A clinic might have a planning model that predicts different proportions of appointments in the morning, afternoon, and evening. A manufacturer might specify different probabilities for several possible outcomes. In either case, the question is whether observed counts are reasonably consistent with the particular probabilities in that model.

The probabilities must be specified before examining the observed counts for the usual fixed-probability Pearson goodness-of-fit test. The test compares each observed count with the count expected under that null model, then combines the discrepancies into one statistic. The overall test assesses whether the full pattern of counts is inconsistent with the model; category-level quantities help describe which categories contribute to that pattern.

Definition: Suppose an outcome has \(k\) mutually exclusive and exhaustive categories. A fully specified probability vector gives a fixed null probability \(p_i\) for each category \(i\), where \(p_i>0\) for every category included and \(\sum_{i=1}^{k}p_i=1\). The null hypothesis is that the population category probabilities equal this vector. If \(O_i\) is the observed count in category \(i\), the expected count under the null is \(E_i=np_i\), where \(n=\sum_{i=1}^{k}O_i\).

Hypotheses, Expected Counts, and the Test Statistic

Write the hypotheses in terms of the category probabilities. If the specified null values are \(p_1^0,\ldots,p_k^0\), then \(H_0\) says that every population category probability equals its specified value. The alternative \(H_a\) says that at least one category probability differs. The alternative does not identify a particular category or direction of change; it describes any departure from the entire specified vector.

H_0: (p_1,\ldots,p_k)=(p_1^0,\ldots,p_k^0) \qquad\text{versus}\qquad H_a: \text{at least one }p_i\ne p_i^0

For each category, the expected count is the total sample size multiplied by that category’s null probability. Pearson’s goodness-of-fit statistic adds the squared observed-minus-expected discrepancy, scaled by the expected count, across all categories:

E_i=np_i^0, \qquad \chi^2=\sum_{i=1}^{k}\frac{(O_i-E_i)^2}{E_i}

When the null probabilities are fully specified and the usual chi-square approximation is appropriate, the reference distribution has \(df=k-1\). The p-value is the right-tail probability \(P(\chi^2_{k-1}\geq \chi^2_{\text{obs}})\). A large statistic means that the observed counts are collectively far from their null expectations. It does not by itself say which category is responsible.

Why the Expected Counts Have This Form

The expected-count formula follows directly from how a count is formed. For one category \(i\), define an indicator for each observation: it equals 1 if that observation falls in category \(i\), and 0 otherwise. The count \(O_i\) is the sum of these \(n\) indicators. Under \(H_0\), each indicator has expected value \(p_i^0\). By linearity of expectation, the expected value of their sum is \(np_i^0\), which is the expected count \(E_i\).

E(O_i)=E\left(\sum_{j=1}^{n} I_{ij}\right) =\sum_{j=1}^{n}E(I_{ij}) =\sum_{j=1}^{n}p_i^0 =np_i^0

This result explains why unequal probabilities matter: the expected counts need not be equal. For example, if one category has twice the null probability of another, it also has twice the expected count, for the same total sample size. Comparing observed counts with \(n/k\) in that situation would test the wrong model.

Worked Examples

Worked Example: A Three-Category Model

Hypothetical data. Suppose 120 independently sampled appointments are classified as morning, afternoon, or evening. A planning model specifies probabilities \(0.50\), \(0.30\), and \(0.20\), respectively. The observed counts are 68, 32, and 20. Assume each appointment contributes to exactly one category, the probability vector was specified in advance, and the observations are independent. The expected counts are all comfortably large, so the usual chi-square approximation is suitable.

Under \(H_0\), the expected counts are:

E_1=120(0.50)=60,\qquad E_2=120(0.30)=36,\qquad E_3=120(0.20)=24

The test statistic is:

\chi^2 =\frac{(68-60)^2}{60} +\frac{(32-36)^2}{36} +\frac{(20-24)^2}{24} =\frac{64}{60}+\frac{16}{36}+\frac{16}{24} =2.178

There are \(k=3\) fixed-probability categories, so \(df=3-1=2\). For 2 degrees of freedom, the right-tail probability is \(e^{-\chi^2/2}\), giving \(P(\chi^2_2\geq2.178)\approx0.3366\). At \(\alpha=0.05\), do not reject \(H_0\): these data do not provide strong evidence that the appointment-time probabilities differ from the specified vector. This is not proof that the model is exactly true.

The contributions to the statistic are approximately 1.067, 0.444, and 0.667. The morning count is above expectation and the other two counts are below expectation. The residuals \((O_i-E_i)/\sqrt{E_i}\) are approximately 1.033, \(-0.667\), and \(-0.816\), respectively. These describe the pattern, but the overall test has not established that any one category differs on its own.

Worked Example: Evidence Against a Nonuniform Model

Hypothetical data. Suppose a different independent sample of 100 observations is classified using the same prespecified probabilities: \(0.50\), \(0.30\), and \(0.20\). The observed counts are 40, 30, and 30. Each observation belongs to one category, and the expected counts are sufficiently large for the chi-square approximation.

The expected counts under \(H_0\) are \(100(0.50)=50\), \(100(0.30)=30\), and \(100(0.20)=20\). Thus:

\chi^2 =\frac{(40-50)^2}{50} +\frac{(30-30)^2}{30} +\frac{(30-20)^2}{20} =2+0+5=7.000

With \(df=2\), the right-tail p-value is \(e^{-7/2}\approx0.03020\). Since \(0.03020<0.05\), reject \(H_0\) at the 5% significance level. The sample provides evidence that the population category probabilities do not match the specified vector.

The contributions are 2, 0, and 5, so the third category contributes most to the total discrepancy. Its observed count is 10 greater than expected; the first category is 10 below expectation, and the second matches expectation. The residuals are \(-10/\sqrt{50}\approx-1.414\), \(0\), and \(10/\sqrt{20}\approx2.236\). These identify the direction and relative size of discrepancies. They are descriptive components of the overall test, not separate category-specific tests.

Worked Example: Interpreting a Modest Mismatch

Hypothetical data. A researcher records 100 independent responses in four mutually exclusive categories. A fully specified model gives probabilities \(0.10\), \(0.20\), \(0.30\), and \(0.40\). The observed counts are 8, 18, 34, and 40. The probabilities were set in advance, and all expected counts are at least 10, so the usual approximation is reasonable under the stated independent-sampling setup.

The expected counts are \(10\), \(20\), \(30\), and \(40\). The statistic is:

\chi^2 =\frac{(8-10)^2}{10} +\frac{(18-20)^2}{20} +\frac{(34-30)^2}{30} +\frac{(40-40)^2}{40} =0.400+0.200+0.5333+0 =1.133

Here \(df=4-1=3\), and the right-tail p-value is approximately \(0.7690\). At \(\alpha=0.05\), do not reject the specified probability vector. The observed differences are small relative to the expected counts and are not unusual under the null model. The category contributions, \(0.400\), \(0.200\), \(0.5333\), and \(0\), show that the third category contributes most, but its contribution is modest and does not establish a separate category-level effect.

Reading Overall and Category-Level Results

The p-value answers an overall question: if the specified vector were correct and the sampling assumptions held, how often would a Pearson statistic at least as large as the observed one occur? A small p-value indicates that the full observed pattern would be unusual under \(H_0\). It is not the probability that \(H_0\) is true, and it does not measure the practical importance of the mismatch.

The term \((O_i-E_i)^2/E_i\) is category \(i\)'s contribution to \(\chi^2\). A larger contribution means that category has more influence on the overall statistic. The Pearson residual \((O_i-E_i)/\sqrt{E_i}\) retains the direction of the difference: a positive residual means more observations than expected, and a negative residual means fewer. Squaring a residual gives the category's contribution. Category-level findings should be described as a pattern unless an appropriate separate inferential analysis is conducted.

The test relies on appropriate data and model conditions: observations are independent, categories are mutually exclusive and exhaustive, the null probabilities are fixed and sum to 1, and the expected counts are adequate for the chi-square approximation. A common guideline is that every expected count should be at least 5; the precise adequacy of the approximation depends on the setting. If the expected-count condition is not met, a routine chi-square p-value may not be reliable.

Zero Probabilities and a Common Mistake

Every category included in the displayed Pearson statistic must have a positive null probability, so its expected count is positive and the denominator is defined. If the model assigns probability zero to a category, a positive observed count is incompatible with that model. If the observed count is zero as well, its Pearson term would be \(0/0\), which is undefined; do not insert that term into the statistic. A genuinely impossible category can be excluded from the calculation when its observed count is zero, with the model’s support and remaining categories handled consistently.

Common mistake: “The categories have unequal observed counts, so I should compare each with \(n/k\).” This uses equal expected counts and therefore tests equal probabilities, not the specified unequal vector. The correction is to calculate each expected count as \(np_i^0\), then compare \(O_i\) with that category-specific expectation. Another mistake is to call a nonsignificant result proof that the model is correct; a failure to reject means only that the sample does not provide sufficiently strong evidence against the specified vector at the chosen significance level.

Key takeaway: For a fixed, nonuniform probability vector, calculate \(E_i=np_i^0\) separately for every category, then use \(\sum (O_i-E_i)^2/E_i\) with \(df=k-1\). Interpret the right-tail p-value as evidence about the overall fit and use contributions or residuals to describe, not overstate, category-level discrepancies.

Check Your Understanding

Answer each question using the specified unequal probabilities rather than assuming the categories are equally likely.

A model specifies probabilities \(0.60\) and \(0.40\) for two categories, with \(n=50\). What are the expected counts?
For four categories with fixed, fully specified probabilities, what are the degrees of freedom for Pearson’s goodness-of-fit test, and why?
If one category has a positive Pearson residual, is its observed count above or below its expected count?
Why is comparing every observed count with \(n/k\) inappropriate when the null probabilities are unequal?
What does a small right-tail p-value indicate about the specified probability vector, and what does it not establish about any single category?
Why can a category with null probability zero not be included in the usual Pearson statistic?