Likelihood Comparisons Across Logit Structures

Why Compare Logit Structures?

Adjacent-category, continuation-ratio, and baseline-category logits describe category probabilities using different odds. When their regression structures impose different restrictions, they can yield different fitted probabilities and different likelihoods for the same observations. A comparison should therefore start with a precise question: are the models nested, or are they nonnested alternatives?

The likelihood measures how well a model’s fitted probabilities account for the observed outcomes. Deviance expresses the likelihood loss relative to a saturated model, while AIC and BIC balance fit against model complexity. These measures are meaningful only when the models are fitted to the same response data using a consistent likelihood convention.

Definition: For independent observations \(Y_1,\ldots,Y_n\) with model probabilities \(p_i(y;\theta)\), the likelihood is \(L(\theta)=\prod_{i=1}^n p_i(Y_i;\theta)\), and the log-likelihood is \(\ell(\theta)=\sum_{i=1}^n\log p_i(Y_i;\theta)\). For grouped multinomial data, the log-likelihood also includes a multinomial coefficient. That coefficient is constant across models fitted to the same counts, so it may be omitted when comparing them, provided the same convention is used throughout.

Likelihood, Saturation, and Deviance

Suppose independent groups \(i=1,\ldots,G\) have counts \(y_{ij}\) in categories \(j=1,\ldots,J\), with group totals \(n_i=\sum_j y_{ij}\). A fitted model supplies probabilities \(\hat p_{ij}\), where \(\sum_j\hat p_{ij}=1\). Omitting the multinomial coefficients, its maximized log-likelihood is

\ell_{\text{fit}}=\sum_{i=1}^G\sum_{j=1}^J y_{ij}\log(\hat p_{ij}).

A saturated model estimates a separate category probability for every group, so its fitted probabilities are \(\hat p_{ij}^{\text{sat}}=y_{ij}/n_i\). Terms with \(y_{ij}=0\) contribute zero, using the convention \(0\log 0=0\).

Theorem: For grouped multinomial data, the deviance of a fitted model relative to the saturated model is

D=2\{\ell_{\text{sat}}-\ell_{\text{fit}}\} =2\sum_{i=1}^G\sum_{j:y_{ij}>0}y_{ij}\log\left(\frac{y_{ij}}{n_i\hat p_{ij}}\right).

Proof. For group \(i\), the saturated log-likelihood contribution, omitting the common multinomial coefficient, is \(\sum_{j:y_{ij}>0}y_{ij}\log(y_{ij}/n_i)\). The fitted contribution is \(\sum_{j:y_{ij}>0}y_{ij}\log\hat p_{ij}\); zero-count terms vanish in both expressions. Subtracting gives

\ell_{\text{sat}}-\ell_{\text{fit}} =\sum_i\sum_{j:y_{ij}>0}y_{ij} \left\{\log\left(\frac{y_{ij}}{n_i}\right)-\log(\hat p_{ij})\right\} =\sum_i\sum_{j:y_{ij}>0}y_{ij}\log\left(\frac{y_{ij}}{n_i\hat p_{ij}}\right).

Multiplying by two proves the formula. The deviance is nonnegative: for each group with \(n_i>0\), put \(q_{ij}=y_{ij}/n_i\). The inequality \(\log t\leq t-1\) for \(t>0\) gives

\sum_{j:q_{ij}>0}q_{ij}\log\left(\frac{q_{ij}}{\hat p_{ij}}\right) \geq \sum_{j:q_{ij}>0}(q_{ij}-\hat p_{ij})\geq 0,

where the final inequality follows because \(\sum_jq_{ij}=1\) and \(\sum_{j:q_{ij}>0}\hat p_{ij}\leq1\). Thus each group’s contribution, and hence the deviance, is nonnegative. \(\square\)

Worked Example: Deviance for Two Nested Fits

Two covariate groups have three-category counts \((8,2,0)\) and \((2,4,4)\). Compare a null model with one common probability vector across groups to a model with a separate probability vector for each group. The pooled counts are \((10,6,4)\), so the null fit is \((0.5,0.3,0.2)\) in both groups. The separate-probability fit uses the observed proportions in each group.

Under the null, the expected counts in each group are \((5,3,2)\). Substituting these and the observed counts into the deviance formula gives

\begin{aligned} D &=2\left[ 8\log(8/5)+2\log(2/3) +2\log(2/5)+4\log(4/3)+4\log(4/2) \right]\\ &=2\left[ 3.760604-0.810930-1.832581+1.150728+2.772589 \right]\\ &\approx 10.0808. \end{aligned}

There are two free probabilities for each group in the separate model, for four in total; the common-probability model has two. The difference is two parameters. Under the regularity conditions for Wilks’ theorem, the likelihood-ratio statistic is approximately \(\chi^2_2\) under the null. Here the null model is nested within the separate-probability model because setting the two group probability vectors equal recovers the null. The approximation should be treated cautiously with small expected counts, such as the expected count of two in this example.

Likelihood-Ratio Tests Require Nesting

A model \(M_0\) is nested in \(M_1\) if every probability distribution allowed by \(M_0\) is also allowed by \(M_1\). For regular nested models, the likelihood-ratio statistic is \(2\{\ell_1-\ell_0\}\), where each log-likelihood is maximized within its model. If the models have \(k_0\) and \(k_1\) free parameters, respectively, Wilks’ theorem says this statistic converges in distribution to \(\chi^2_{k_1-k_0}\) under the null.

This approximation assumes independent observations and regularity conditions: the null parameter is an interior point of the larger model’s parameter space, the model is identifiable, and the likelihood is sufficiently smooth with nonsingular information near the true parameter. Boundary parameters, nonidentifiability, or sparse data can invalidate the usual reference distribution. In particular, a difference between adjacent-category and continuation-ratio fits is not automatically a likelihood-ratio test: if neither model contains the other, there is no standard degrees-of-freedom difference to use.

Worked Example: A Common Fit Under Three Logit Labels

In one group, the observed category counts are \((6,3,1)\). An intercept-only model with three category probabilities has two free parameters. With no covariates or additional constraints, each of the adjacent-category, continuation-ratio, and baseline-category parameterizations can represent any strictly positive three-category probability vector. All three therefore achieve the saturated maximum at \((0.6,0.3,0.1)\).

Omitting the multinomial coefficient, the shared maximized log-likelihood is

\ell_{\text{fit}} =6\log(0.6)+3\log(0.3)+1\log(0.1) \approx-3.064954-3.611918-2.302585 =-8.979457.

With \(k=2\), the AIC is

\mathrm{AIC}=-2\ell_{\text{fit}}+2k =17.958914+4 \approx21.9589.

There are \(N=10\) observations, so the BIC is

\mathrm{BIC}=-2\ell_{\text{fit}}+k\log N =17.958914+2\log(10) \approx22.5641.

Both criteria are identical across the three fits because they have the same maximized likelihood and parameter count. This does not show that the three structures will fit equally well once different covariate restrictions are imposed; it shows that the logit labels alone do not force different fits in this unrestricted intercept-only case.

AIC and BIC for Competing Fits

For a fitted model with maximized log-likelihood \(\ell\) and \(k\) estimated parameters, the Akaike information criterion and Bayesian information criterion are

\mathrm{AIC}=-2\ell+2k,\qquad \mathrm{BIC}=-2\ell+k\log N.

Here \(N\) is the number of independent observational units, not necessarily the number of rows in a grouped table. The parameter count includes all estimated regression coefficients and other free parameters. Smaller values indicate a preferred trade-off between fit and complexity among the candidates being compared; neither criterion is a hypothesis-test \(p\)-value. For grouped counts, the multinomial coefficient may be omitted from \(\ell\) because it is the same for all models fitted to those same counts. Do not omit different constants from different models or compare values computed under inconsistent likelihood conventions.

Worked Example: Information Criteria for Nonnested Fits

Suppose three models are fitted to the same \(N=80\) independent outcomes. A fitting report gives maximized log-likelihoods of \(-91.2\) for an adjacent-category model, \(-89.7\) for a continuation-ratio model, and \(-90.4\) for a baseline-category model. Each model estimates \(k=6\) parameters. Take these values as the reported maximum-likelihood fits; the task is to compare their fit-complexity criteria.

The AIC values are

\begin{aligned} \mathrm{AIC}_{\text{adjacent}}&=-2(-91.2)+12=194.4,\\ \mathrm{AIC}_{\text{continuation}}&=-2(-89.7)+12=191.4,\\ \mathrm{AIC}_{\text{baseline}}&=-2(-90.4)+12=192.8. \end{aligned}

Since \(\log(80)\approx4.3820266\), the BIC penalty is \(6\log(80)\approx26.2922\). Thus

\begin{aligned} \mathrm{BIC}_{\text{adjacent}}&=182.4+26.2922=208.6922,\\ \mathrm{BIC}_{\text{continuation}}&=179.4+26.2922=205.6922,\\ \mathrm{BIC}_{\text{baseline}}&=180.8+26.2922=207.0922. \end{aligned}

Both criteria rank the continuation-ratio fit first here, because all three models have the same parameter count and it also has the largest maximized log-likelihood. The models may be nonnested, so the likelihood difference does not by itself justify a chi-square test. AIC and BIC provide criterion-based comparisons, not proof that the winning structure is correct.

Common Mistakes and Practical Interpretation

Using a likelihood-ratio test for any two models. The usual chi-square result requires nesting and regularity conditions. For nonnested structures, compare criteria such as AIC or BIC, or use a method designed for the specific nonnested comparison.
Comparing log-likelihoods from different data. They are comparable only when the response observations and likelihood convention match.
Forgetting the parameter penalty. A larger model can increase maximized likelihood simply by having more flexibility; AIC and BIC explicitly penalize that flexibility.
Treating a lower criterion as a test result. AIC and BIC rank candidate fits under their respective penalties; neither supplies a conventional null-hypothesis \(p\)-value.
Assuming different logit labels guarantee different fits. Unrestricted parameterizations can represent the same probabilities. Differences arise from the models’ imposed covariate structures or constraints, not merely from the way odds are written.

In practice, report the response data used, the log-likelihood convention, the number of estimated parameters, and the criterion values. When a nested comparison is scientifically relevant, state the null restriction and check whether the regularity conditions make the chi-square approximation credible. When models are nonnested, present the criterion ranking as comparative evidence rather than a definitive test.

Check Your Understanding

Use the definitions and calculations above to answer these questions.

Why may the multinomial coefficient be omitted from every model’s log-likelihood when the same grouped counts are being compared?
In the nested example, what makes the separate-probability model a larger model than the common-probability model?
State two regularity conditions needed for the usual Wilks chi-square approximation.
For two nonnested logit structures, why is a difference in maximized log-likelihoods not automatically a likelihood-ratio test statistic with a known chi-square distribution?
If two models have the same maximized log-likelihood and the same number of parameters on the same data, what can you conclude about their AIC and BIC values?