Identifiability of Adjacent-Category Regression

Identifiability and Estimation Are Different Questions

An adjacent-category regression describes the odds between neighboring response categories as functions of covariates. Before interpreting or estimating its coefficients, ask two separate questions: does the model assign a unique set of parameters to each possible distribution, and does the observed data yield a finite maximum-likelihood estimate? The first is structural identifiability; the second is a property of the likelihood for the particular sample. Full-rank design is central to the first question, but it does not guarantee the second.

Let the ordered response have \(K\) categories, numbered \(1,\ldots,K\), with probabilities \(p_k(x)=P(Y=k\mid x)\). For each adjacent pair define the log-odds \(\eta_j(x)=\log\{p_{j+1}(x)/p_j(x)\}\), for \(j=1,\ldots,K-1\). In the category-specific regression, the coefficients may vary across adjacent pairs. Write \(z(x)=(1,x^\mathsf{T})^\mathsf{T}\) for the design vector including an intercept.

Definition: A parameterization is structurally identifiable on a covariate support \(\mathcal X\) if two parameter values that give the same conditional response probabilities \(p_k(x)\) for every \(x\in\mathcal X\) must be equal. A finite maximum-likelihood estimate is a finite parameter value at which the likelihood attains its supremum. Identifiability does not, by itself, ensure that such an estimate exists.

For finite adjacent logits, every category probability is positive. The probabilities can be recovered from the adjacent logits: set \(s_1(x)=0\), \(s_k(x)=\sum_{j=1}^{k-1}\eta_j(x)\) for \(k\geq2\), and normalize the weights \(\exp\{s_k(x)\}\). Thus the logits and the conditional probability vector determine one another uniquely. This fact lets us assess identifiability through the regression equations for the logits.

Category-Specific Slopes: One Rank Check per Adjacent Pair

Suppose each adjacent logit has its own intercept and slope vector: \(\eta_j(x)=z(x)^\mathsf{T}\theta_j\), where \(\theta_j=(\alpha_j,\beta_j^\mathsf{T})^\mathsf{T}\). Let \(Z\) be the matrix whose rows are \(z(x_i)^\mathsf{T}\) for the distinct covariate values in the support. The same support is used for every adjacent pair; the relevant rank is the column rank of this design matrix.

Theorem: In the category-specific model, all adjacent-category intercepts and slopes are structurally identifiable on \(\mathcal X\) if and only if \(Z\) has full column rank. Equivalently, no nonzero vector \(v\) satisfies \(z(x)^\mathsf{T}v=0\) for every \(x\in\mathcal X\).

Proof. Suppose \(Z\) has full column rank and two parameter sets produce the same probabilities at every supported covariate value. Since probabilities determine adjacent logits, for each \(j\) their logit difference is zero at every design row. In matrix form, \(Z(\theta_j-\theta'_j)=0\). Full column rank implies that the null space of \(Z\) contains only zero, so \(\theta_j=\theta'_j\) for every \(j\).

Conversely, if \(Z\) is rank deficient, there is a nonzero \(v\) with \(Zv=0\). Change one adjacent-pair coefficient vector from \(\theta_j\) to \(\theta_j+v\), leaving the others unchanged. This does not change any adjacent logit on the support, and therefore does not change any conditional probability. The parameters are not identifiable. \(\square\)

The rank condition is about the covariate support, not simply the number of observations. Repeated observations at the same covariate values can improve precision, but they cannot add a new design direction. For an intercept and one predictor, at least two distinct supported predictor values are needed; for an intercept and \(p\) predictors, the supported design vectors must span \(p+1\) dimensions.

Worked Example: Checking Category-Specific Slopes

Consider three ordered categories and one predictor \(x\), with a separate intercept and slope for each of the two adjacent logits. The covariate support is \(x=0,1\). Suppose the conditional probabilities are \((0.5,0.3,0.2)\) at \(x=0\), and \((0.2,0.5,0.3)\) at \(x=1\).

At \(x=0\), the adjacent logits are \(\eta_1(0)=\log(0.3/0.5)=\log(0.6)\) and \(\eta_2(0)=\log(0.2/0.3)=\log(2/3)\). At \(x=1\), they are \(\eta_1(1)=\log(0.5/0.2)=\log(2.5)\) and \(\eta_2(1)=\log(0.3/0.5)=\log(0.6)\).

For either adjacent pair, the design matrix is \(Z=\begin{pmatrix}1&0\\1&1\end{pmatrix}\). Its determinant is \(1\), so it has full rank. The coefficient values are uniquely determined: \(\alpha_1=\log(0.6)\), \(\beta_1=\log(2.5)-\log(0.6)=\log(25/6)\); and \(\alpha_2=\log(2/3)\), \(\beta_2=\log(0.6)-\log(2/3)=\log(0.9)\). This establishes structural identifiability on these two support points; it does not assert that probabilities in a real sample will be known exactly.

Common Slopes: Covariate Variation Must Remain After Intercepts

In the common-slope model, each adjacent pair has its own intercept but all pairs share a slope vector: \(\eta_j(x)=\alpha_j+x^\mathsf{T}\beta\). The intercepts can absorb a constant shift in the linear predictor, so identifiability of the common slope depends on variation in \(x\), not merely on the rank of the uncentered covariate matrix.

Theorem: Suppose the same covariate support \(\mathcal X\) is used for all \(K-1\) adjacent logits, and each has its own intercept. The common-slope parameter \(\beta\) and all intercepts \(\alpha_j\) are identifiable if and only if the differences \(x-x_0\), for \(x\in\mathcal X\) and any fixed \(x_0\in\mathcal X\), span the predictor space.

Proof. If two parameter sets give the same logits, their differences satisfy \(\delta\alpha_j+x^\mathsf{T}\delta\beta=0\) for every \(x\) and \(j\). Subtract the equation at \(x_0\) from the equation at \(x\). This gives \((x-x_0)^\mathsf{T}\delta\beta=0\) for every support difference. If those differences span the predictor space, then \(\delta\beta=0\), and the original equations then give \(\delta\alpha_j=0\) for every \(j\).

If the differences do not span the predictor space, there is a nonzero \(\delta\beta\) orthogonal to all of them. Set \(\delta\alpha_j=-x_0^\mathsf{T}\delta\beta\) for every \(j\). Then \(\delta\alpha_j+x^\mathsf{T}\delta\beta=(x-x_0)^\mathsf{T}\delta\beta=0\) throughout the support. This nonzero parameter change leaves all logits unchanged, proving nonidentifiability. \(\square\)

Worked Example: A Common Slope With and Without Covariate Variation

For three categories and one predictor, suppose the common-slope model gives adjacent logits \((-0.4,0.2)\) at \(x=0\) and \((0.1,0.7)\) at \(x=1\). The change in each adjacent logit is \(0.5\), so the common slope is \(\beta=0.5\). The intercepts are the logits at zero: \(\alpha_1=-0.4\) and \(\alpha_2=0.2\). Because the support difference \(1-0=1\) spans the one-dimensional predictor space, these parameters are identifiable.

Now suppose the only supported value is \(x=2\). For any number \(c\), replace \(\beta\) by \(\beta+c\) and each \(\alpha_j\) by \(\alpha_j-2c\). At the only supported value, \((\alpha_j-2c)+2(\beta+c)=\alpha_j+2\beta\). Thus all fitted logits remain unchanged, and the intercepts and slope cannot be separately identified.

Empty Categories, Separation, and the Likelihood

A sample can have empty categories even when the model is structurally identifiable. An empty category is not a zero probability required by the finite-parameter model: it is a category with zero observed count. But such a sample can push likelihood maximization toward a boundary where one or more probabilities approach zero. In that case the likelihood may have a supremum without attaining it at any finite parameter value.

A useful way to see this is to write the category weights at covariate value \(x_i\) as \(\exp\{s_{ik}\}\), where \(s_{i1}=0\) and \(s_{ik}=\sum_{j=1}^{k-1}\eta_j(x_i)\). For an observation in category \(y_i\), its log-probability is \(s_{iy_i}-\log\sum_k\exp(s_{ik})\). If a parameter direction increases the score of each observed category relative to its competitors, the likelihood can keep improving as the direction grows. This is separation: it obstructs a finite maximum, rather than establishing a rank deficiency.

Worked Example: An Empty Middle Category and a Boundary Supremum

At a single covariate value, suppose the observed counts in three ordered categories are \(6,0,4\). Write \(\eta_1=\log(p_2/p_1)\) and \(\eta_2=\log(p_3/p_2)\). Consider \(\eta_1=-t\) and \(\eta_2=t+c\). The unnormalized category weights are \(1,e^{-t},e^c\), so the probabilities are \(p_1=1/D\), \(p_2=e^{-t}/D\), \(p_3=e^c/D\), where \(D=1+e^{-t}+e^c\).

The likelihood, omitting the constant multinomial coefficient, is \(L=p_1^6p_3^4\), and hence \(\ell=-10\log D+4c\). For fixed \(c\), its derivative with respect to \(t\) is \(10e^{-t}/D>0\). Thus increasing \(t\) always increases the likelihood, and no finite \(t\) maximizes it.

As \(t\) tends to infinity, \(p_2\) tends to zero. The limiting log-likelihood is \(-10\log(1+e^c)+4c\). Its derivative in \(c\) is \(4-10e^c/(1+e^c)\), which is zero at \(e^c=4/6=2/3\); its second derivative is \(-10e^c/(1+e^c)^2<0\), so this is the maximum over \(c\). The likelihood therefore has a finite supremum approached as \(t\) tends to infinity, but no finite maximum. This is a boundary problem associated with the empty middle category, not a failure of structural identifiability for the finite-parameter model.

The same distinction matters when checking data. A rank-deficient design makes some coefficient combinations invisible at every possible response distribution on the observed covariate support. Separation instead gives a direction along which the likelihood improves while parameters diverge. An empty category can contribute to separation, but its presence alone does not prove separation in every design or sample. Check the fitted design and the observed category pattern together.

Practical Checks and a Common Mistake

Specify the model.
Decide whether each adjacent logit has its own slopes or the model imposes a common slope; the identifiability condition differs.

Check support and rank.
For category-specific coefficients, check the rank of the design matrix. For common slopes with separate intercepts, check whether covariate differences span the predictor space.

Inspect observed categories.
Look for empty categories and patterns where observed categories can be made increasingly favored over unobserved alternatives.

Report the issue precisely.
Distinguish nonidentifiable coefficients, a finite but imprecise estimate, and a likelihood whose supremum occurs only at a boundary.

Common mistake: “Every category has observations, so the regression coefficients must be identifiable and have finite estimates.” This reasoning conflates two separate requirements. Identifiability depends on covariate support and design rank; finite estimation depends on the likelihood for the realized category counts. Conversely, an empty category does not automatically mean that the model is structurally nonidentifiable. It may instead create a separated comparison and a nonattained likelihood supremum, as in the worked example.

Key takeaway: Use design rank to assess whether distinct coefficient vectors can produce the same adjacent logits on the covariate support. Then inspect the data separately for empty categories and separation, which can prevent finite maximum-likelihood estimation even when the model is structurally identifiable.

Check Your Understanding

Use the support and the observed category pattern separately in each answer.

With an intercept and two predictors, what must be true of the supported design vectors for category-specific adjacent slopes to be identifiable?
In a common-slope model with category-specific intercepts, why does a single supported covariate value fail to identify the slope?
Does an empty response category, by itself, prove that the regression parameterization is structurally nonidentifiable? Explain.
In the three-category example with counts \(6,0,4\), which probability approaches zero along the likelihood-improving sequence?
Explain the difference between a rank-deficient design and separation in terms of what happens to the likelihood and parameters.