Ordered-Response Probabilities And Likelihood

From Threshold Crossings to Category Probabilities

An ordered outcome has categories with a meaningful ranking, such as “low,” “medium,” and “high,” but the distance between adjacent categories is not generally measured on a common numerical scale. Ordered-response models represent this structure by supposing that an unobserved continuous variable crosses thresholds. The probability of a particular observed category is therefore the probability that the latent variable falls between two adjacent thresholds.

Let $Y_i$ be an observed outcome for unit $i$, with $J$ ordered categories $1,\ldots,J$. Write the latent index as $\eta_i=x_i'\beta$, where $x_i$ is a vector of observed covariates and $\beta$ is a vector of coefficients. The latent variable is $Y_i^*=\eta_i+\varepsilon_i$. The observed category is determined by cutpoints $\kappa_1,\ldots,\kappa_{J-1}$, ordered as $\kappa_1<\cdots<\kappa_{J-1}$. As in the earlier material on threshold identification, a location and scale normalization is needed; here the error distribution is standardized and its location is zero.

Definition: Set $\kappa_0=-\infty$ and $\kappa_J=+\infty$. The ordered-response rule is $Y_i=j$ if and only if $\kappa_{j-1}<Y_i^*\leq\kappa_j$. If the error $\varepsilon_i$ has cumulative distribution function $F$, then, conditional on $x_i$, $$ P(Y_i=j\mid x_i) =F(\kappa_j-\eta_i)-F(\kappa_{j-1}-\eta_i), \qquad j=1,\ldots,J. $$

The formula follows because the event defining category $j$ is $\kappa_{j-1}<\eta_i+\varepsilon_i\leq\kappa_j$. Subtracting $\eta_i$ from each part gives $\kappa_{j-1}-\eta_i<\varepsilon_i\leq\kappa_j-\eta_i$. The probability of this interval is the difference between the error CDF at its upper and lower endpoints. For the first and last categories, the infinite cutpoints make the formula work without separate interval rules.

In the ordered logit model, $F$ is the standard logistic CDF, $\Lambda(z)=1/(1+\exp(-z))$. In the ordered probit model, $F$ is the standard normal CDF, $\Phi(z)=P(Z\leq z)$ for $Z\sim N(0,1)$. The two models use the same threshold-crossing logic but different error distributions. In either case, the cutpoints must remain ordered for the category intervals to make sense.

Checking the Probability Formula

A valid set of category probabilities must be nonnegative and sum to one. Ordering the cutpoints ensures nonnegativity: because $F$ is nondecreasing and $\kappa_j-\eta_i>\kappa_{j-1}-\eta_i$, the CDF difference for each category cannot be negative. The sum has a useful telescoping form.

Write the adjacent differences.
For fixed $i$, the category probabilities are $F(\kappa_1-\eta_i)-F(\kappa_0-\eta_i)$, through $F(\kappa_J-\eta_i)-F(\kappa_{J-1}-\eta_i)$.

Add across categories.
The interior CDF terms cancel: each $F(\kappa_j-\eta_i)$ appears once with a plus sign and once with a minus sign.

Use the endpoint limits.
Since $F(-\infty)=0$ and $F(+\infty)=1$, the remaining difference is $1-0=1$.

Result: For every covariate vector $x_i$, the ordered-response formula gives nonnegative category probabilities that sum to one. The result relies on ordered cutpoints and a valid CDF $F$.

Worked Example: Three Ordered Logit Probabilities

Suppose a hypothetical three-category response follows an ordered logit model with cutpoints $\kappa_1=-0.8$ and $\kappa_2=0.6$. For a unit with latent index $\eta_i=0.4$, calculate the probabilities of all three categories.

The cutpoints are ordered because $-0.8<0.6$, so they define three valid intervals. Using $\Lambda(z)=1/(1+\exp(-z))$, the first-category probability is $$ P(Y_i=1\mid x_i) =\Lambda(-0.8-0.4) =\Lambda(-1.2) \approx 0.2314752. $$

The second category lies between the two cutpoints: $$ P(Y_i=2\mid x_i) =\Lambda(0.6-0.4)-\Lambda(-0.8-0.4) =\Lambda(0.2)-\Lambda(-1.2). $$ Using the unrounded CDF values, $\Lambda(0.2)=0.5498339973$ and $\Lambda(-1.2)=0.2314752165$, so $$ P(Y_i=2\mid x_i) =0.5498339973-0.2314752165 =0.3183587808 \approx 0.3184. $$ Thus the conditional probability of category 2 is about 31.84%.

For the final category, the probability is the upper tail beyond the second cutpoint: $$ P(Y_i=3\mid x_i) =1-\Lambda(0.6-0.4) =1-\Lambda(0.2) =1-0.5498339973 =0.4501660027 \approx 0.4502. $$ As a check, the unrounded probabilities add to $0.2314752165+0.3183587808+0.4501660027=1.0000000000$. The cutpoints and index have been specified, and the logistic CDF is used consistently for all three categories.

Worked Example: An Ordered Probit Probability

Now suppose a hypothetical three-category response follows an ordered probit model with cutpoints $\kappa_1=-0.5$ and $\kappa_2=0.7$. A unit has $\eta_i=0.2$. Find the probability of the middle category.

The cutpoints are ordered, $-0.5<0.7$, and the error is standard normal under the probit specification. The relevant standardized endpoints are $\kappa_1-\eta_i=-0.7$ and $\kappa_2-\eta_i=0.5$. Therefore, $$ P(Y_i=2\mid x_i) =\Phi(0.5)-\Phi(-0.7) \approx 0.6914624613-0.2419636522 =0.4494988091 \approx 0.4495. $$

The probability of the middle category is about 44.95%. As a quick validity check, the first-category probability is $\Phi(-0.7)\approx0.2420$, and the last-category probability is $1-\Phi(0.5)\approx0.3085$. Together with the middle probability, these values sum to one up to rounding. The calculation uses the standard normal CDF because this is an ordered probit model, not an ordered logit model.

Constructing the Ordered-Response Likelihood

Once the model assigns a probability to each possible category for each observation, the likelihood records how probable the observed sample is under specified model parameters. Suppose there are $n$ observations, and define $p_{ij}=P(Y_i=j\mid x_i)$ using the threshold formula. Assume the observations are conditionally independent given their covariates, and that their category probabilities follow the stated ordered-response model. These assumptions allow the joint conditional probability of the observed outcomes to be written as a product.

Definition: Let $\mathbf{1}(Y_i=j)$ equal 1 when observation $i$ is in category $j$, and 0 otherwise. The conditional likelihood is $$ L(\beta,\boldsymbol{\kappa}) =\prod_{i=1}^{n}\prod_{j=1}^{J} p_{ij}^{\mathbf{1}(Y_i=j)}, \qquad p_{ij}=F(\kappa_j-x_i'\beta)-F(\kappa_{j-1}-x_i'\beta). $$ The log-likelihood is $$ \ell(\beta,\boldsymbol{\kappa}) =\sum_{i=1}^{n}\sum_{j=1}^{J} \mathbf{1}(Y_i=j)\log(p_{ij}). $$

For each observation, exactly one indicator in the inner product equals 1. Consequently, that observation contributes only the probability assigned to its observed category. The product form is the joint probability under conditional independence; taking logarithms changes the product into a sum and is more convenient for calculation and optimization. The parameters in the likelihood are the coefficient vector and the ordered cutpoints, subject to the model’s identification normalization and the cutpoint-ordering constraints.

This likelihood is a function of unknown parameters for the observed data and covariates. It is not the probability that a parameter value is true. Estimation chooses parameter values that make the observed category pattern more likely under the specified model. The construction here is the ordered-category counterpart of likelihood methods used for multinomial outcomes, with the probabilities restricted by a common latent index and ordered cutpoints.

Worked Example: Likelihood for Three Hypothetical Responses

Consider three hypothetical independent observations from an ordered logit model with cutpoints $\kappa_1=-0.8$, $\kappa_2=0.6$. Suppose their indices and observed categories are: observation 1 has $\eta_1=0.4$ and $Y_1=2$; observation 2 has $\eta_2=-0.3$ and $Y_2=1$; observation 3 has $\eta_3=1.0$ and $Y_3=3$. The cutpoints are ordered, and the same logistic CDF is used for each independent observation.

For observation 1, the category-2 probability is $$ p_{12} =\Lambda(0.6-0.4)-\Lambda(-0.8-0.4) =\Lambda(0.2)-\Lambda(-1.2) =0.3183587808. $$ For observation 2, the observed category is 1, so $$ p_{21} =\Lambda(-0.8-(-0.3)) =\Lambda(-0.5) =0.3775406688. $$ For observation 3, the observed category is 3, so $$ p_{33} =1-\Lambda(0.6-1.0) =1-\Lambda(-0.4) =0.5986876601. $$

Conditional independence gives the likelihood contribution for this particular outcome vector: $$ L =p_{12}p_{21}p_{33} =0.3183587808\times0.3775406688\times0.5986876601 \approx0.0719583. $$ Thus the likelihood is approximately $0.07196$, or $0.0720$ to four decimal places. Its log is $$ \ell=\log(L)\approx\log(0.0719583)=-2.6317. $$ The likelihood is the model-assigned joint probability of these three observed categories at the stated indices and cutpoints. It is not a probability that the parameters themselves are correct.

Why Log-Likelihoods and Numerical Care Matter

A sample likelihood multiplies many probabilities, so it can become very small as the sample grows. The log-likelihood avoids repeatedly multiplying small numbers: it sums the logarithms of the probabilities assigned to the observed categories. A category probability of zero would make the log-likelihood negative infinity, which is one reason valid model calculations and appropriate numerical procedures matter.

A frequent numerical mistake is to round cumulative probabilities before subtracting them. In the first worked example, rounding $\Lambda(0.2)$ and $\Lambda(-1.2)$ to four decimal places before subtraction gives $0.5498-0.2315=0.3183$. But the unrounded difference is $0.3183587808$, which rounds to $0.3184$. Similarly, likelihood products should be calculated from unrounded probabilities and rounded only for reporting. Otherwise, rounding error can accumulate across observations and alter the reported likelihood or log-likelihood.

The same care applies to model interpretation. An ordered-response probability is conditional on the specified covariates and parameters, and refers to a particular outcome category. For example, the value $0.3184$ above is the model probability of category 2 for an observation with index $0.4$, given the stated cutpoints and logistic error distribution. It is not a numerical distance between categories, nor does it establish a causal effect of a covariate.

Common pitfall: It is incorrect to calculate an interior-category probability from rounded CDF values and then treat the result as exact at the same precision. Use the difference of unrounded CDF evaluations, retain adequate precision through likelihood multiplication and logarithms, and round only the final reported results.

Check Your Understanding

Use the threshold rule and likelihood definitions to answer the following questions.

For an ordered model with three categories, write the latent-index interval corresponding to category 2 and its probability in terms of $F$, $\kappa_1$, $\kappa_2$, and $\eta_i$.
Explain why ordered cutpoints ensure that each adjacent-difference category probability is nonnegative.
For an ordered logit model, calculate the probability of category 1 when $\kappa_1=-0.4$ and $\eta_i=0.1$. State the formula before evaluating it.
Why does conditional independence allow the likelihood for several observed outcomes to be written as a product?
Explain why the likelihood is not the probability that a particular parameter value is true.
What numerical error can arise from subtracting rounded cumulative probabilities, and how should the calculation be performed instead?

Ordered-Response Probabilities And Likelihood

What You'll Learn