Baseline-Category Logit Models

Why Use a Baseline Category?

A response can have more than two possible outcomes without those outcomes having a natural order. A patient might improve, remain unchanged, or worsen; a respondent might choose one of several unordered alternatives. A binary logit compares two outcomes. For \(K\) categories, the baseline-category logit model instead compares each of \(K-1\) categories with one selected reference category.

The reference category is a comparison point, not a claim that its outcome is more important, more common, or lower in an ordering. The model needs no ordering of the response categories. Its equations describe the conditional probabilities of the categories through their log odds relative to that reference.

Definition: Let \(Y_i\) be an outcome for observation \(i\), taking one of \(K\geq 2\) unordered categories labeled \(1,\ldots,K\). Given a covariate vector \(x_i\), write \(\pi_{ij}=P(Y_i=j\mid x_i)\). Choose category \(K\) as the reference. A baseline-category logit model specifies, for \(j=1,\ldots,K-1\), \(\log(\pi_{ij}/\pi_{iK})=x_i^\mathsf{T}\beta_j\), where \(\beta_j\) is the coefficient vector for category \(j\) versus the reference. These logits require \(\pi_{ij}>0\) and \(\pi_{iK}>0\) at the covariate values considered. The reference equation is conventionally written as \(\eta_{iK}=0\); it is a normalization, not an additional fitted equation.

If \(x_i\) has \(p\) components, each \(\beta_j\) has \(p\) components. When an intercept is included, the first component of \(x_i\) is 1. The remaining components can represent continuous covariates, indicator variables, or other specified predictors. Each nonreference category has its own coefficient vector: the model does not require the covariates to have the same coefficients for every category.

A logit is the logarithm of odds. In this model, the odds of category \(j\) versus the baseline category are \(\pi_{ij}/\pi_{iK}\). An odds ratio, by contrast, ordinarily compares odds across two conditions or groups. Keeping these terms separate prevents a common misreading of the model equation.

Writing the \(K-1\) Equations

Once the reference category is chosen, write one equation for each remaining category. The left side is the log of that category's probability divided by the reference probability. The right side is its own linear predictor. For example, if \(K=3\) and category 3 is the reference, the equations are

\log\left(\frac{\pi_{i1}}{\pi_{i3}}\right)=x_i^\mathsf{T}\beta_1, \qquad \log\left(\frac{\pi_{i2}}{\pi_{i3}}\right)=x_i^\mathsf{T}\beta_2.

There are two equations because there are two nonreference categories. If the response has \(K\) categories, the model has \(K-1\) category-specific equations. This count refers to equations, not to the total number of coefficients: with \(p\) covariates in each linear predictor, there are \(p\) coefficients per equation.

The category labels are interchangeable in the sense that none is inherently the baseline. In a particular analysis, however, state the chosen reference explicitly and keep it fixed while writing and interpreting the model. Changing the reference changes which comparisons the equations directly express.

Worked Example: Specifying a Model for Four Outcomes

Suppose a study records one of four unordered treatment outcomes: improvement (category 1), no change (category 2), adverse response (category 3), or discontinuation (category 4). Let \(x_i=(1,\text{age}_i,\text{active}_i)^\mathsf{T}\), where age is measured in years and \(\text{active}_i\) is 1 for active treatment and 0 for control. Choose discontinuation as the reference category.

There are \(K=4\) categories, so there must be \(K-1=3\) equations. With a separate coefficient vector for each nonreference category, write

\log\left(\frac{\pi_{i1}}{\pi_{i4}}\right) =\beta_{10}+\beta_{11}\text{age}_i+\beta_{12}\text{active}_i,

\log\left(\frac{\pi_{i2}}{\pi_{i4}}\right) =\beta_{20}+\beta_{21}\text{age}_i+\beta_{22}\text{active}_i,

\log\left(\frac{\pi_{i3}}{\pi_{i4}}\right) =\beta_{30}+\beta_{31}\text{age}_i+\beta_{32}\text{active}_i.

Each equation compares one outcome with discontinuation. The model has three coefficient vectors, each with three entries: an intercept, an age coefficient, and an active-treatment coefficient. It does not impose a ranking such as improvement above no change above adverse response; the outcomes are treated as distinct categories.

What a Baseline Logit Represents

For a fixed observation and a nonreference category, the logit equation specifies a ratio of two category probabilities. Exponentiating both sides gives the corresponding odds:

\frac{\pi_{ij}}{\pi_{iK}} =\exp\left(x_i^\mathsf{T}\beta_j\right).

This statement concerns category \(j\) versus category \(K\) for the same covariate profile \(x_i\). It is not, by itself, an odds ratio comparing two profiles. The standard binary logit is recovered when \(K=2\): there is one nonreference category and one log odds equation.

Result: For two positive category probabilities \(\pi_{ij}\) and \(\pi_{iK}\), define \(q_{ijK}=\pi_{ij}/(\pi_{ij}+\pi_{iK})\), the probability of category \(j\) conditional on the outcome being either \(j\) or \(K\). Then \(\operatorname{logit}(q_{ijK})=\log(\pi_{ij}/\pi_{iK})\).

Proof. Since both probabilities are positive, \(0<q_{ijK}<1\), and \(1-q_{ijK}=\pi_{iK}/(\pi_{ij}+\pi_{iK})\). Therefore \(q_{ijK}/(1-q_{ijK})=\pi_{ij}/\pi_{iK}\). Taking logarithms gives the stated equality. Thus a baseline-category logit is also the ordinary binary logit for the two-category comparison after restricting attention to those two outcomes. \(\square\)

The result clarifies the role of the denominator: the baseline-category logit is a logarithm of odds between two categories. It does not mean that all observations outside those categories have been modeled as one combined outcome; the full model still has \(K\) response categories and \(K-1\) comparisons to the chosen baseline.

Worked Example: Calculating Baseline Logits From Probabilities

At a particular covariate profile, suppose a three-category response has probabilities \(\pi_{i1}=0.18\), \(\pi_{i2}=0.32\), and \(\pi_{i3}=0.50\). Use category 3 as the reference and calculate both baseline logits and odds.

For category 1 versus category 3, the odds are \(\pi_{i1}/\pi_{i3}=0.18/0.50=0.36\). The logit is \(\log(0.36)\approx -1.0217\).

For category 2 versus category 3, the odds are \(\pi_{i2}/\pi_{i3}=0.32/0.50=0.64\). The logit is \(\log(0.64)\approx -0.4463\).

The signs are consistent with the odds: in each comparison, the nonreference category has a probability smaller than category 3, so its odds relative to category 3 are below 1 and its log odds are negative. The two logits are the two required equations evaluated at this one profile; they are not odds ratios across profiles.

Constant Logits and Constant Category Distributions

A useful feature of the baseline representation is that all \(K-1\) comparisons together determine whether the category distribution changes across covariate profiles. The next result establishes this without requiring an explicit formula that converts a set of logits into probabilities.

Result: Consider two covariate profiles \(x_a\) and \(x_b\), with strictly positive probabilities for all \(K\) categories at both profiles. If their baseline logits are equal for every \(j=1,\ldots,K-1\), then their full category probability vectors are equal.

Proof. Write the probabilities at the two profiles as \(\pi_{aj}\) and \(\pi_{bj}\). Equality of the baseline logits implies equality of their exponentials, and hence \(\pi_{aj}/\pi_{aK}=\pi_{bj}/\pi_{bK}\) for every \(j<K\). Define \(c=\pi_{aK}/\pi_{bK}\), which is positive. Multiplying the ratio equalities by \(\pi_{aK}\) gives \(\pi_{aj}=c\pi_{bj}\) for every \(j<K\). The same equality holds for \(j=K\) by the definition of \(c\). Summing over all categories yields \(1=\sum_{j=1}^K\pi_{aj}=c\sum_{j=1}^K\pi_{bj}=c\). Thus \(c=1\), so \(\pi_{aj}=\pi_{bj}\) for every category. \(\square\)

In particular, if the model has intercepts but no covariate terms, each baseline logit is constant across profiles. The result shows that the category probability vector is then constant as well. More generally, if every fitted baseline logit has the same value at two profiles, the model implies the same category distribution at those profiles.

Worked Example: Evaluating Category-Specific Predictors

Consider three response categories with category 3 as the reference and predictor vector \(x_i=(1,z_i)^\mathsf{T}\). Suppose \(\beta_1=(-0.4,0.3)^\mathsf{T}\) and \(\beta_2=(0.2,-0.2)^\mathsf{T}\). Calculate the two logits and odds when \(z_i=2\).

The predictor vector is \(x_i=(1,2)^\mathsf{T}\). For category 1, \(x_i^\mathsf{T}\beta_1=(1)(-0.4)+(2)(0.3)=-0.4+0.6=0.2\). Thus \(\log(\pi_{i1}/\pi_{i3})=0.2\), and \(\pi_{i1}/\pi_{i3}=\exp(0.2)\approx1.2214\).

For category 2, \(x_i^\mathsf{T}\beta_2=(1)(0.2)+(2)(-0.2)=0.2-0.4=-0.2\). Thus \(\log(\pi_{i2}/\pi_{i3})=-0.2\), and \(\pi_{i2}/\pi_{i3}=\exp(-0.2)\approx0.8187\).

The two calculations use different coefficient vectors, as the model allows. They give the fitted log odds and odds relative to category 3 at this profile. They do not by themselves report the three category probabilities.

Common Mistakes and Practical Checks

A model is easier to check when its categories, baseline, and equations are stated explicitly. Before using or reporting a fitted model, verify that the number of equations is \(K-1\), that each left side uses the same chosen reference category, and that every category-specific linear predictor uses the corresponding coefficient vector.

Calling a logit an odds ratio. A logit is the logarithm of odds. Here those odds are \(\pi_{ij}/\pi_{iK}\). An odds ratio compares odds across conditions or groups.
Writing \(K\) equations. With category \(K\) as the reference, there are \(K-1\) equations. The reference category is represented by the zero-predictor convention, not another fitted logit equation.
Treating the outcomes as ordered. Baseline-category logits do not assume that category 1 is below category 2 or that category \(K\) is an endpoint on a scale. They are comparisons between named categories.
Giving every category the same coefficient vector without justification. The standard baseline-category model permits a distinct \(\beta_j\) for each nonreference category. Equality constraints, if desired, are additional modeling choices.
Reporting an exponentiated coefficient without naming the comparison. First identify the nonreference category and its reference. The equation determines which category odds the coefficient belongs to.

The baseline-category formulation provides a compact way to describe a multinomial response: choose one category as the comparison point, then specify a separate log-odds equation for every other category. Keeping the probability ratio and its reference category visible in each equation makes the model's structure clear.

Check Your Understanding

Use the definitions and results above to answer these questions.

A response has five unordered categories. If category 5 is the reference, how many baseline logit equations are specified?
In a baseline-category model, what is the distinction between a logit and an odds ratio?
For probabilities \(\pi_{i1}=0.25\) and \(\pi_{i3}=0.50\), calculate the odds and logit for category 1 versus category 3.
Why is the reference-category equation represented by a zero predictor rather than counted as an additional fitted logit?
If every baseline logit is equal at two profiles and all category probabilities are positive, what does the proved result imply about their category distributions?

What You'll Learn