Likelihood for Grouped Ordinal Counts

From Ordinal Responses to Grouped Counts

When several individuals share the same covariate values, their ordinal responses can be summarized by the number in each response category. This can make a data set easier to report, but it does not change the model for an individual response. The grouped likelihood is the probability of the observed count vector under that model, with the number of possible individual response sequences accounted for.

The key is to form the category probabilities at each covariate pattern before writing the count likelihood. In a cumulative-link model, these probabilities are obtained by differencing adjacent cumulative probabilities. Counts from different covariate patterns generally have different cell probabilities, so their likelihood contributions must be kept separate.

Definition: Suppose the ordered response $Y$ has categories $1,\ldots,K$. At covariate vector $x_g$, let $n_{gj}$ be the number of responses in category $j$, and let $n_g=\sum_{j=1}^K n_{gj}$. The vector $(n_{g1},\ldots,n_{gK})$ is a grouped ordinal count vector for covariate pattern $g$.

Cell Probabilities Under a Cumulative-Link Model

Let $H$ be a strictly increasing cumulative distribution function. A cumulative-link model with ordered thresholds $\theta_1<\cdots<\theta_{K-1}$ and linear predictor $\eta_g=x_g^\mathsf{T}\beta$ specifies $P(Y\leq j\mid x_g)=H(\theta_j-\eta_g)$ for $j=1,\ldots,K-1$. The model gives cumulative probabilities; a multinomial likelihood requires the probability of each individual category.

Definition: Write $F_{gj}=H(\theta_j-\eta_g)$, and set $F_{g0}=0$ and $F_{gK}=1$. The probability of category $j$ at pattern $g$ is $p_{gj}=F_{gj}-F_{g,j-1}$, for $j=1,\ldots,K$.

Thus the first category has probability $H(\theta_1-\eta_g)$, an interior category has the difference between two adjacent cumulative probabilities, and the final category has probability $1-H(\theta_{K-1}-\eta_g)$. Since $H$ is increasing and the thresholds are ordered, every $p_{gj}$ is nonnegative. The probabilities sum to one because the differences telescope:

\sum_{j=1}^{K}p_{gj} =\sum_{j=1}^{K}(F_{gj}-F_{g,j-1}) =F_{gK}-F_{g0} =1.

For finite thresholds and a strictly increasing $H$, these category probabilities are positive. Positivity ensures that the log-likelihood expressions below are finite whenever the counts are finite.

The Multinomial Likelihood for One Pattern

Assume that, conditional on the covariates, responses from different individuals are independent, and that all $n_g$ individuals in pattern $g$ have the same category probabilities $p_{g1},\ldots,p_{gK}$. The probability of one specified sequence of responses containing $n_{gj}$ observations in category $j$ is $\prod_{j=1}^K p_{gj}^{n_{gj}}$. The observed grouped data do not specify which individual had which response, so the number of such sequences must also be included.

Theorem: Under the independence and common-probability assumptions above, the likelihood of the count vector at pattern $g$ is $$ L_g(\theta,\beta) =\frac{n_g!}{\prod_{j=1}^{K}n_{gj}!} \prod_{j=1}^{K}p_{gj}(\theta,\beta)^{n_{gj}}. $$ Here the multinomial coefficient counts the distinct individual response sequences that yield the observed counts.

Proof. Fix any one ordered sequence of $n_g$ responses with the stated counts. Independence gives its probability as $\prod_j p_{gj}^{n_{gj}}$: each factor $p_{gj}$ occurs once for every response in category $j$. To count the sequences, choose the $n_{g1}$ positions for category 1, then the $n_{g2}$ positions for category 2 from those remaining, and continue. The number is $(n_g!/\prod_j n_{gj}!)$. These sequences are disjoint events, each with the same probability, so adding their probabilities gives the displayed likelihood. $\square$

If the data instead retain the response for each identified individual, the probability of that particular sequence omits the multinomial coefficient. The coefficient depends only on the observed counts, not on $\theta$ or $\beta$. Consequently, the individual-sequence and grouped-count likelihoods differ by a parameter-independent factor and have the same likelihood maximizers. For a count-vector probability, however, the coefficient is part of the exact probability.

Worked Example: Three Categories at One Covariate Pattern

Suppose $Y$ has three ordered categories and follows a cumulative-logit model, so $H(t)=1/(1+\exp(-t))$. At a particular covariate pattern, take $\theta_1=-0.5$ and $\theta_2=1$, with $\eta=0$. The observed counts are $(12,10,8)$.

The cumulative probabilities are $F_1=H(-0.5)$ and $F_2=H(1)$. Therefore the three category probabilities are $p_1=H(-0.5)$, $p_2=H(1)-H(-0.5)$, and $p_3=1-H(1)$. Numerically these are approximately $0.3775407$, $0.3535179$, and $0.2689414$, respectively. They are positive and sum to one.

With $n=12+10+8=30$, the exact grouped likelihood is

L =\frac{30!}{12!\,10!\,8!} [H(-0.5)]^{12} [H(1)-H(-0.5)]^{10} [1-H(1)]^{8}.

The factorial coefficient counts the arrangements of 12 responses in category 1, 10 in category 2, and 8 in category 3. Replacing the exact probability terms by the displayed rounded decimals would give only an approximation to this likelihood, not an equality.

Combining Covariate Patterns

Suppose the data have $G$ distinct covariate patterns. If responses are conditionally independent across all individuals, then the count vectors from the patterns are independent. Each pattern may have its own probabilities because its linear predictor $\eta_g$ can differ. The full likelihood is the product of the pattern-specific multinomial likelihoods:

L(\theta,\beta) =\prod_{g=1}^{G} \left\{ \frac{n_g!}{\prod_{j=1}^{K}n_{gj}!} \prod_{j=1}^{K}p_{gj}(\theta,\beta)^{n_{gj}} \right\}.

This product uses common thresholds and slopes across patterns, while evaluating the category probabilities at each pattern’s covariate values. Pooling counts across patterns is generally invalid: the same category may have different probabilities at different $x_g$.

Worked Example: Two Covariate Patterns

Consider three response categories with thresholds $\theta_1=-1$ and $\theta_2=1$, using the logistic CDF $H$. Suppose the linear predictor is $\eta_1=0$ for the first pattern and $\eta_2=0.5$ for the second. The observed count vectors are $(8,12,10)$ and $(5,9,16)$.

At the first pattern, $p_{11}=H(-1)$, $p_{12}=H(1)-H(-1)$, and $p_{13}=1-H(1)$. Their approximate values are $0.2689414$, $0.4621172$, and $0.2689414$.

At the second pattern, the cumulative probabilities are $H(-1.5)$ and $H(0.5)$. Hence $p_{21}=H(-1.5)$, $p_{22}=H(0.5)-H(-1.5)$, and $p_{23}=1-H(0.5)$, approximately $0.1824255$, $0.4400338$, and $0.3775407$.

The exact likelihood is the product of two multinomial terms:

L= \frac{30!}{8!\,12!\,10!} [H(-1)]^8[H(1)-H(-1)]^{12}[1-H(1)]^{10} \frac{30!}{5!\,9!\,16!} [H(-1.5)]^5[H(0.5)-H(-1.5)]^9[1-H(0.5)]^{16}.

Each pattern has 30 observations, but the cell probabilities differ because the predictors differ. The exact expression retains the logistic probabilities; the decimal values above are only approximations for checking their sizes.

Log-Likelihood and Its Derivative

Taking logarithms turns the product over patterns and categories into a sum. With the convention that a zero count contributes zero, the grouped log-likelihood is

\ell(\theta,\beta) =\sum_{g=1}^{G}\log\left(\frac{n_g!}{\prod_{j=1}^{K}n_{gj}!}\right) +\sum_{g=1}^{G}\sum_{j=1}^{K}n_{gj}\log p_{gj}(\theta,\beta).

The first sum is constant with respect to the model parameters. Therefore it may be dropped when finding a maximum-likelihood estimate, although it should be retained when reporting the exact likelihood or an absolute log-likelihood.

Result: If each $p_{gj}(\theta)$ is differentiable and positive on the parameter region under consideration, then the gradient of the grouped log-likelihood is $$ \nabla_\theta\ell(\theta) =\sum_{g=1}^{G}\sum_{j=1}^{K} \frac{n_{gj}}{p_{gj}(\theta)}\nabla_\theta p_{gj}(\theta). $$

Derivation. The factorial terms do not depend on $\theta$, so their gradients are zero. For every positive differentiable cell probability, the chain rule gives $\nabla_\theta\log p_{gj}=(\nabla_\theta p_{gj})/p_{gj}$. Applying this identity to each term $n_{gj}\log p_{gj}$, and summing over patterns and categories, gives the stated expression. If a count is zero, its contribution is identically zero and can be omitted. $\square$

Worked Example: Four Categories and Nested Counts

Take four categories, thresholds $(-1,0,1)$, logistic $H$, and $\eta=0$. The category probabilities are $H(-1)$, $H(0)-H(-1)$, $H(1)-H(0)$, and $1-H(1)$. Numerically they are approximately $0.2689414$, $0.2310586$, $0.2310586$, and $0.2689414$. Suppose the observed counts are $(6,9,5,10)$, which sum to 30.

The exact likelihood is

L=\frac{30!}{6!\,9!\,5!\,10!} [H(-1)]^6[H(0)-H(-1)]^9[H(1)-H(0)]^5[1-H(1)]^{10}.

The corresponding cumulative counts are 6 observations in categories 1, 15 in categories 1 or 2, and 20 in categories 1, 2, or 3. These are counts of nested events: everyone counted in the first cumulative count is also counted in the second and third. They are not independent binomial observations. The multinomial likelihood above correctly represents the four mutually exclusive category counts without treating those nested totals as independent.

A Common Mistake

Wrong reasoning: “The data have several covariate patterns, but the responses all use the same ordered categories, so I can add each category’s counts and use one multinomial likelihood.”
Correction: A multinomial count vector has one probability vector. If predictors differ by pattern, the category probabilities differ too. Use one multinomial contribution for each pattern, evaluated at its own predictor, and multiply those contributions under conditional independence.
Another check: Cumulative probabilities are not category probabilities. Use adjacent differences to obtain the mutually exclusive cell probabilities before inserting counts into a multinomial likelihood.

Key takeaway: For each fixed covariate pattern, convert cumulative-link probabilities into category probabilities by differencing, then use the multinomial likelihood for its count vector. Multiply across patterns only after retaining each pattern’s own probabilities.

Check Your Understanding

Use the definitions and likelihoods above to answer these questions.

For $K$ categories, write the probability of the final category in terms of the last cumulative probability.
Why does the likelihood of a grouped count vector include a multinomial coefficient?
Two covariate patterns have different linear predictors. Under what condition could their category probabilities nevertheless be identical?
Explain why a zero cell count contributes zero to the log-likelihood when its probability is positive.
Why should cumulative counts such as the numbers with $Y\leq1$ and $Y\leq2$ not be treated as independent binomial counts?