Enforcing Ordered Thresholds During Estimation

Why Threshold Order Must Be Enforced

A cumulative-link model represents an ordinal response with cutpoints on a common scale. For $K$ ordered categories, there are $K-1$ thresholds. Their order is not merely a convention: differencing successive cumulative probabilities produces the probability of each category, and a reversed pair of thresholds can make one of those differences negative. Estimation therefore needs to keep the thresholds ordered, not just start from values that happen to be ordered.

Write the linear predictor for observation $i$ as $\eta_i=x_i^\mathsf{T}\beta$, with no separate intercept, and let $F$ be the link model's cumulative distribution function. Under the cumulative-link convention used here, $P(Y_i\leq j\mid x_i)=F(\alpha_j-\eta_i)$, for $j=1,\ldots,K-1$. The thresholds $\alpha_1,\ldots,\alpha_{K-1}$ must increase with $j$.

Definition: A strictly ordered threshold vector satisfies $\alpha_1<\alpha_2<\cdots<\alpha_{K-1}$. A convenient unconstrained parameterization uses a real starting value $\gamma_1$ and real increment parameters $\delta_2,\ldots,\delta_{K-1}$: $$ \alpha_1=\gamma_1,\qquad \alpha_j=\gamma_1+\sum_{r=2}^{j}\exp(\delta_r),\quad j=2,\ldots,K-1. $$ Each threshold after the first is obtained by adding positive gaps to the previous threshold.

The exponential is one standard way to make each gap positive. The parameters $\gamma_1$ and $\delta_r$ can range freely over the real line, while the resulting thresholds are always strictly ordered. In an implementation, the optimizer works with these unconstrained parameters and the model calculates the thresholds from them whenever it evaluates the likelihood.

Examples of the Increment Parameterization

Worked Example: Constructing Four Ordered Categories

Suppose there are four response categories, so the model needs three thresholds. Set $\gamma_1=-1$, $\delta_2=\log 2$, and $\delta_3=\log 0.5$. The first threshold is $\alpha_1=-1$. The first gap is $\exp(\delta_2)=2$, so $\alpha_2=-1+2=1$. The second gap is $\exp(\delta_3)=0.5$, so $\alpha_3=1+0.5=1.5$. Thus $-1<1<1.5$, as required.

For an additional check, take $\eta=0$ and the logistic cumulative distribution function $F(t)=1/(1+\exp(-t))$. The cumulative probabilities at the thresholds are approximately $F(-1)=0.26894$, $F(1)=0.73106$, and $F(1.5)=0.81757$. Differencing them gives category probabilities $0.26894$, $0.46212$, $0.08651$, and $0.18243$. They are all nonnegative and sum to $1$.

Worked Example: Detecting a Reversed Threshold

Consider a three-category logistic cumulative-link model at $\eta=0$, but suppose an unconstrained fitting routine returns $\alpha_1=1$ and $\alpha_2=0$. The cumulative probabilities are $F(1)=1/(1+\exp(-1))\approx0.73106$ and $F(0)=0.5$. The implied probability of category 2 is therefore $F(\alpha_2)-F(\alpha_1)=0.5-0.73106=-0.23106$.

A negative probability cannot describe any response distribution. The issue is the reversed order $1>0$, not a peculiarity of the logistic function. For comparison, choose $\gamma_1=0$ and $\delta_2=0$ in the increment parameterization. This gives thresholds $0$ and $1$, and the category probabilities are $0.5$, $F(1)-F(0)\approx0.23106$, and $1-F(1)\approx0.26894$. Each is nonnegative, and their sum is $1$.

Why Increasing Thresholds Give Valid Probabilities

For a fixed observation, write $q_j=F(\alpha_j-\eta)$, and set $q_0=0$ and $q_K=1$. The category probabilities are $p_j=q_j-q_{j-1}$, for $j=1,\ldots,K$. This includes the first category, whose probability is $q_1$, and the last category, whose probability is $1-q_{K-1}$.

Theorem: Let $F$ be a cumulative distribution function, let $\eta$ be finite, and let $\alpha_1<\cdots<\alpha_{K-1}$ be finite thresholds. Define $q_0=0$, $q_K=1$, $q_j=F(\alpha_j-\eta)$ for $1\leq j\leq K-1$, and $p_j=q_j-q_{j-1}$. Then every $p_j\geq0$ and $\sum_{j=1}^{K}p_j=1$. If $F$ is strictly increasing, every $p_j>0$.

Proof. A cumulative distribution function is nondecreasing and takes values in $[0,1]$. Since $\alpha_j-\eta<\alpha_{j+1}-\eta$, this gives $q_j\leq q_{j+1}$ for each pair of internal cutpoints. Also $0\leq q_1$ and $q_{K-1}\leq1$. Consequently every difference $p_j=q_j-q_{j-1}$ is nonnegative. The sum telescopes:

\sum_{j=1}^{K}p_j =\sum_{j=1}^{K}(q_j-q_{j-1}) =q_K-q_0 =1.

If $F$ is strictly increasing, each pair of successive arguments is strictly ordered, including the endpoint comparisons with probabilities $0$ and $1$; the resulting category probabilities are positive. For the usual logistic and normal cumulative-link distributions, this strict positivity holds for finite thresholds and finite $\eta$. $\square$

This result explains the practical point of the parameterization: if the thresholds are built from positive gaps, their order is guaranteed at every likelihood evaluation, so differencing the cumulative probabilities cannot create a negative category probability. The argument applies for each predictor value separately.

Reparameterization and Likelihood Estimation

Positive increments are not an additional statistical assumption beyond strict threshold ordering. They are a way to express the same set of allowable threshold vectors using coordinates that need no inequality constraints. The correspondence is exact.

Theorem: The map from $(\gamma_1,\delta_2,\ldots,\delta_{K-1})\in\mathbb{R}^{K-1}$ to $(\alpha_1,\ldots,\alpha_{K-1})$ defined by cumulative exponential increments is a one-to-one correspondence onto the set of finite strictly increasing threshold vectors. Therefore, with the same slope parameters and data, maximizing the ordinal-response likelihood over these transformed parameters has the same supremum as maximizing it over strictly ordered thresholds. If either maximum is attained at a finite parameter vector, the corresponding vector in the other parameterization attains the same likelihood.

Proof. Given any real $\gamma_1,\delta_2,\ldots,\delta_{K-1}$, the first threshold is $\gamma_1$ and each gap is $\alpha_j-\alpha_{j-1}=\exp(\delta_j)>0$. Thus the resulting thresholds are finite and strictly increasing.

Conversely, take any finite vector with $\alpha_1<\cdots<\alpha_{K-1}$. Set $\gamma_1=\alpha_1$, and for $j=2,\ldots,K-1$, set $\delta_j=\log(\alpha_j-\alpha_{j-1})$. The gaps are positive, so each logarithm is defined and finite. Substitution gives the original thresholds. These inverse formulas also show uniqueness: no other starting value or increment vector yields that same threshold vector.

For fixed slopes, the likelihood depends on the threshold parameters only through the thresholds they produce. Since the transformation covers exactly all strictly ordered finite threshold vectors, the set of likelihood values available in the transformed coordinates is exactly the set available in ordered-threshold coordinates. Their suprema are therefore equal. If a maximum is attained in one coordinate system, the forward or inverse map supplies a parameter vector in the other with the identical thresholds and likelihood. $\square$

Worked Example: Writing a Grouped-Count Likelihood in Increment Coordinates

Suppose a three-category response has counts $8,5,7$ at a covariate value with $\eta=0$. Choose thresholds $\alpha_1=-0.5$ and $\alpha_2=0.7$. In increment coordinates, $\gamma_1=-0.5$, and the gap is $0.7-(-0.5)=1.2$, giving $\delta_2=\log(1.2)\approx0.18232$. The positive increment reconstructs the second threshold as $-0.5+\exp(\log 1.2)=0.7$.

Using the logistic cumulative distribution function, the category probabilities are $p_1=F(-0.5)\approx0.37754$, $p_2=F(0.7)-F(-0.5)\approx0.66819-0.37754=0.29065$, and $p_3=1-F(0.7)\approx0.33181$.

Ignoring the multinomial coefficient, which does not depend on the parameters, the grouped-count likelihood is $L=p_1^8p_2^5p_3^7$. Its log-likelihood is

\ell =8\log(0.37754)+5\log(0.29065)+7\log(0.33181) \approx -7.7926-6.1782-7.7223 =-21.6931.

An optimizer can evaluate this same likelihood from $\gamma_1$ and $\delta_2$ at every trial point. Since $\exp(\delta_2)>0$ for every real $\delta_2$, it cannot evaluate the model with reversed thresholds and the resulting negative middle-category probability.

Practical Choices and Common Pitfalls

The exponential increment is convenient because its inverse is the logarithm of the gap and because it maps every real input to a strictly positive gap. Other positive transformations can be used, but their attainable gaps and numerical behavior should be understood. For example, the softplus function $\log(1+\exp(\delta))$ also produces positive increments. An implementation should evaluate such transformations in numerically stable form when parameters are very large in magnitude.

Strict positivity has a boundary implication. A zero gap would give two equal thresholds and, for a strictly increasing $F$, a category with probability zero. No finite $\delta_j$ gives $\exp(\delta_j)=0$; the gap approaches zero only as $\delta_j$ tends to negative infinity. Thus the transformed finite parameter space represents strictly ordered thresholds, while limiting configurations with tied thresholds lie on its boundary.

Mistake: checking order only at initialization. An optimizer may move freely through unconstrained threshold values later, so an initially ordered vector does not ensure valid probabilities throughout estimation. Use a parameterization or constraint that preserves order at every evaluation.
Mistake: assuming the transformation guarantees a successful fit. It guarantees threshold order, not existence of a finite maximum, convergence of an optimizer, or good numerical conditioning. Those issues still require appropriate optimization and diagnostics.
Mistake: adding a separate intercept while also estimating all thresholds. In this cumulative-link convention the thresholds supply the location terms; adding a common intercept creates a redundant shift. Keep the predictor and threshold convention consistent.

Key takeaway: Parameterize the first threshold freely and every later threshold as the previous one plus a positive increment. This makes every fitted threshold vector strictly ordered and keeps the category probabilities admissible throughout estimation.

Check Your Understanding

Use the cumulative-link convention and the positive-increment parameterization above.

For five response categories, how many thresholds and how many increment parameters are needed?
If $\gamma_1=-2$, $\delta_2=0$, and $\delta_3=\log 3$, calculate the first three thresholds.
Why does a nondecreasing cumulative distribution function, together with ordered thresholds, imply nonnegative category probabilities?
For thresholds $\alpha_1=0.4$ and $\alpha_2=0.1$, explain why the middle-category probability is negative for any strictly increasing $F$ at a fixed $\eta$.
Does the exponential parameterization guarantee that a finite maximum-likelihood estimate exists? Explain the distinction.