Sparse Categories and Unreliable Proportional-Odds Tests

Why Sparse Categories Matter

An ordinal model uses the order of response categories as well as their frequencies. In a proportional-odds cumulative-logit model with categories \(1,\ldots,K\), the convention here is

where \(\alpha_1<\cdots<\alpha_{K-1}\) are the cutpoints and \(\beta\) is the common slope vector. The probability of category \(j\) is the difference between two adjacent cumulative probabilities. Thus, a category with very few observations corresponds to a small difference between cumulative probabilities, and an empty category corresponds to a zero difference in the empirical distribution.

Sparse categories do not automatically make a model invalid. They do, however, make some parts of the likelihood weakly informed by the data. That can lead to unstable cutpoints, large or misleading standard errors, extreme fitted probabilities, and poor finite-sample approximations for tests of parallel slopes.

Cutpoints and Category Counts

In an intercept-only cumulative-logit model, let \(p_k=P(Y=k)\), and let \(F_j=P(Y\leq j)=\sum_{k=1}^j p_k\). The cutpoint is \(\alpha_j=\operatorname{logit}(F_j)\). The gap between adjacent cutpoints is

When category \(j\) is rare, \(F_j-F_{j-1}=p_j\) is small, so these cutpoints can be close. A small gap is not itself an error: it reflects the small probability assigned to that category. It does mean that the data must distinguish two nearby cumulative probabilities to estimate the corresponding thresholds reliably.

Derivation. Write the counts as \(n_1,\ldots,n_K\), with total \(n\), and let \(p_k\) be candidate category probabilities summing to one. Ignoring the multinomial coefficient, the likelihood is \(L=\prod_k p_k^{n_k}\). For any candidate with positive probabilities wherever \(n_k>0\), use \(\log u\leq u-1\):

Equality is achieved by \(p_k=n_k/n\) for positive counts and \(p_k=0\) for zero counts. If category \(j\) is empty, its fitted probability is zero; hence \(F_j=F_{j-1}\), and \(\operatorname{logit}(F_j)=\operatorname{logit}(F_{j-1})\) whenever these cumulative probabilities are interior. A model constrained to have strictly increasing cutpoints can approach this likelihood but cannot attain it at a finite, strictly ordered solution. This exact boundary conclusion is for the intercept-only model; with predictors, an empty overall category can have more complicated consequences, but it remains a serious warning about weak identification or boundary behavior. \(\square\)

Why Rare Cumulative Events Carry Little Information

A useful way to see the information problem is to focus on a single cumulative event, such as \(Y\leq j\), and write its probability as \(q=\operatorname{logit}^{-1}(\alpha)\). For \(n\) independent observations, the indicators of this event are Bernoulli with probability \(q\). The log-likelihood for \(\alpha\), up to a constant, is \(r\log q+(n-r)\log(1-q)\), where \(r\) is the number of events.

Derivation. Since \(dq/d\alpha=q(1-q)\), differentiating the log-likelihood gives the score \(r-nq\). Its variance is \(\operatorname{Var}(r)=nq(1-q)\), which is the Fisher information. Equivalently, the negative expected second derivative is \(nq(1-q)\). When \(q\) is close to zero or one, this quantity is small. In an ordinal model the cutpoints are jointly estimated and the cumulative indicators are dependent, so this calculation is not the full information matrix. It does establish why a threshold supported by very few observations on one side can be poorly estimated. \(\square\)

Symptoms in Fits and Proportional-Odds Tests

Sparse response categories can affect both the fitted model and tests of the parallel-slopes restriction. Likelihood-ratio, Wald, and score tests are commonly compared with chi-square reference distributions under the null hypothesis. Their standard large-sample justification requires, among other conditions, a correctly specified model, independent observations, an identifiable true parameter in the interior of the parameter space, differentiability and finite information, and a nonsingular information matrix. The model dimension is held fixed as sample size grows. If category probabilities are positive and fixed, expected counts grow with sample size; a category that remains rare as the sample grows, or whose probability approaches zero, need not satisfy the same useful approximation.

A small sample count is therefore a warning, not a proof that a test is invalid. Conversely, a large total sample does not guarantee safety if a response category is absent in a covariate group or if a fitted cumulative probability is nearly zero or one. For a Wald test, a very large standard error can make the statistic uninformative; near-boundary estimates can also make its normal approximation poor. A likelihood-ratio statistic can have a nonstandard finite-sample distribution near a boundary, while a score test still relies on adequate information under the null. None of the three tests repairs an absence of data support.

Practical warning signs include nonconvergence, extreme coefficients or cutpoints, large standard errors, a nearly singular estimated information matrix, strong parameter correlations, fitted category probabilities near zero, and very small expected counts within predictor patterns. Also inspect the observed and fitted counts by category and by scientifically important covariate groups. An overall category count can conceal a group with no observations in that category, which may be more consequential for a slope comparison.

When Is Pooling Defensible?

Pooling replaces adjacent response categories with a combined category. It changes the response being modeled: after pooling, the model no longer estimates a cutpoint separating the two original categories. This can be sensible if the distinction is not meaningful for the scientific question or cannot be measured reliably. It is not a purely numerical repair that leaves the estimand unchanged.

Before pooling, consider the following checks:

• Respect order. Pool adjacent categories, not separated categories, so the resulting outcome remains ordered.
• Preserve meaning. Explain why the merged categories can reasonably be treated as one outcome for this scientific purpose.
• Use a defensible decision rule. Prefer a prespecified or substantively justified rule over repeated data-driven pooling until a test becomes significant.
• Report the change. State which categories were combined, why, and how the resulting model changes the interpretation.
• Check alternatives. Compare conclusions with and without pooling when estimation permits, and be transparent if the unpooled fit is unstable.

Pooling does not guarantee reliable inference: the remaining categories can still be sparse, and sparse covariate patterns can persist. Nor does it establish the proportional-odds assumption. It addresses category support by changing the response scale; the model assumptions and tests still require separate assessment.

Common Mistake

Wrong reasoning: “The proportional-odds test is not significant, so parallel slopes are established, even though one response category has only two observations.” A nonsignificant statistic may reflect weak information rather than strong support for the null, and its reference distribution may be inaccurate in a sparse sample.

Correction: Inspect category and covariate-pattern counts, fitted probabilities, convergence, standard errors, and the conditions behind the test approximation. Report the test as limited evidence when sparse data make it unstable; do not treat a nonsignificant result as proof of proportional odds. Pool categories only if their substantive meaning justifies the changed outcome.