Eliminating Stratum Intercepts in Logistic Regression

Why Condition on the Number of Cases?

In a stratified logistic regression, each stratum can have its own baseline risk while the covariate effects are shared. For example, hospitals may differ in baseline outcome rates even when the association between a patient characteristic and the outcome is assumed common across hospitals. If every stratum receives its own intercept, estimating all those intercepts can be awkward, especially when some strata contain few observations. A direct alternative is to condition on the observed number of cases within each stratum.

The key is that the case count carries the stratum-specific intercept, while the arrangement of those cases among individuals carries information about the common covariate effects. Conditioning on the count removes the intercept from the probability of that arrangement. This tutorial derives that conditional probability and the resulting likelihood. The argument requires a common covariate-effect vector across strata; it does not imply that the strata have the same baseline risk.

Model and Conditional Likelihood

There are \(S\) strata. Stratum \(s\) contains \(n_s\) individuals, with binary outcomes \(Y_{si}\in\{0,1\}\) and covariate vectors \(x_{si}\in\mathbb{R}^p\), for \(i=1,\ldots,n_s\). Treat the covariates as fixed. Conditional on them, assume all outcomes are independent and

\operatorname{logit}\{P(Y_{si}=1)\}=\alpha_s+x_{si}^{\mathsf T}\beta,

where \(\alpha_s\) is an unrestricted intercept for stratum \(s\), and \(\beta\in\mathbb{R}^p\) is common to all strata. Write \(K_s=\sum_{i=1}^{n_s}Y_{si}\) for the observed number of cases in stratum \(s\).

Definition: The conditional likelihood for \(\beta\) is the probability, under this model, of the observed outcome configuration conditional on the stratum case counts \(K_1,\ldots,K_S\) and the covariates. It is a likelihood formed from conditional probabilities; it is not obtained by substituting estimated intercepts into the ordinary likelihood.

For a particular stratum, let \(y_s=(y_{s1},\ldots,y_{sn_s})\) be a possible outcome vector with \(\sum_i y_{si}=k_s\). Define its covariate total \(T_s(y_s)=\sum_i y_{si}x_{si}\). The conditional probability of this configuration is

P(Y_s=y_s\mid K_s=k_s,x_s) = \frac{\exp\{\beta^{\mathsf T}T_s(y_s)\}} {\displaystyle\sum_{\substack{A\subseteq\{1,\ldots,n_s\}\\|A|=k_s}} \exp\left\{\beta^{\mathsf T}\sum_{i\in A}x_{si}\right\}}.

The sum in the denominator ranges over every possible set \(A\) of \(k_s\) case positions. In particular, it is a sum over configurations, not a product of individual Bernoulli denominators.

Theorem: Under the stated independent Bernoulli logistic model, for finite stratum intercepts and finite covariates, conditioning on \(K_s=k_s\) removes \(\alpha_s\). The conditional likelihood across independent strata is the product of the stratum probabilities above, evaluated at the observed outcome configurations.

Proof. For a possible outcome vector \(y_s\), independence and the logistic model give

P(Y_s=y_s\mid x_s) = \frac{\exp\left\{\alpha_s\sum_i y_{si} +\beta^{\mathsf T}\sum_i y_{si}x_{si}\right\}} {\displaystyle\prod_{i=1}^{n_s} \left[1+\exp\{\alpha_s+x_{si}^{\mathsf T}\beta\}\right]}.

For every vector in the event \(K_s=k_s\), the first sum in the exponent is \(k_s\). The product in the denominator is also the same for every configuration in that event. Summing the displayed probability over all configurations with \(k_s\) cases therefore gives the probability of the conditioning event. Dividing the probability of a particular configuration by that sum cancels both the common product and \(\exp(\alpha_s k_s)\), leaving the stated conditional probability. With independent outcomes across strata, the stratum outcome vectors are independent; conditioning separately on their respective counts preserves the product form. For finite \(\alpha_s\) and finite covariates, every configuration has positive probability, so each count \(k_s\in\{0,\ldots,n_s\}\) is an event of positive probability. \(\square\)

Worked Calculations

Worked Example: One Case Among Three Individuals

A stratum has three individuals with scalar covariates \(x=(0,1,2)\). The observed case count is \(K=1\), and the case is the individual with \(x=2\). Find the stratum’s conditional likelihood contribution.

Because exactly one of the three individuals is a case, the possible case sets are \(\{1\}\), \(\{2\}\), and \(\{3\}\). Their covariate totals are \(0\), \(1\), and \(2\), so their respective unnormalized weights are \(1\), \(e^\beta\), and \(e^{2\beta}\). The observed configuration has the third weight. Thus

L_c(\beta)=\frac{e^{2\beta}}{1+e^\beta+e^{2\beta}}.

At \(\beta=\log 2\), the weights are \(1,2,4\), their sum is \(7\), and the observed configuration has conditional probability \(4/7\). The intercept does not enter this calculation. The contribution compares the observed covariate value for the case with the values available to the other possible case configurations.

Worked Example: Multiplying Contributions Across Strata

Suppose there are two independent strata with a common scalar effect \(\beta\). In stratum 1, the covariates are \((0,1,2)\), one case is observed, and that case has \(x=1\). In stratum 2, the covariates are \((0,1,3)\), two cases are observed, and the cases have \(x=1\) and \(x=3\). Derive the joint conditional likelihood and evaluate it at \(\beta=\log 2\).

For stratum 1, the possible single-case weights are \(1,e^\beta,e^{2\beta}\). The observed weight is \(e^\beta\), so its contribution is \(e^\beta/(1+e^\beta+e^{2\beta})\).

For stratum 2, enumerate the three possible pairs of cases. Their covariate totals are \(0+1=1\), \(0+3=3\), and \(1+3=4\), giving weights \(e^\beta,e^{3\beta},e^{4\beta}\). The observed pair has total \(4\), so its contribution is \(e^{4\beta}/(e^\beta+e^{3\beta}+e^{4\beta})\). Independence across strata gives

L_c(\beta)= \frac{e^\beta}{1+e^\beta+e^{2\beta}} \frac{e^{4\beta}}{e^\beta+e^{3\beta}+e^{4\beta}}.

At \(\beta=\log 2\), the first contribution is \(2/(1+2+4)=2/7\). The second is \(16/(2+8+16)=16/26=8/13\). Their product is \((2/7)(8/13)=16/91\). Notice that each stratum has its own denominator, because the possible case configurations and case count differ between strata.

Log-Likelihood, Score, and Information

For the observed data, let \(T_s=\sum_iY_{si}x_{si}\). Define the finite sum

Z_s(\beta;k_s)= \sum_{\substack{A\subseteq\{1,\ldots,n_s\}\\|A|=k_s}} \exp\left\{\beta^{\mathsf T}\sum_{i\in A}x_{si}\right\}.

The conditional log-likelihood is therefore

\ell_c(\beta)= \sum_{s=1}^S\left\{\beta^{\mathsf T}T_s-\log Z_s(\beta;K_s)\right\}.

Result: The conditional score is the observed total covariate among cases minus its conditional expectation. The negative Hessian is the sum of conditional covariance matrices of the case covariate totals.

Derivation. For each stratum, the sum defining \(Z_s\) is finite, so it can be differentiated term by term. Differentiating each exponential gives

\nabla_\beta\log Z_s(\beta;k_s) = \frac{\displaystyle\sum_{|A|=k_s} \left(\sum_{i\in A}x_{si}\right) \exp\left\{\beta^{\mathsf T}\sum_{i\in A}x_{si}\right\}} {Z_s(\beta;k_s)} = E_\beta[T_s(Y_s)\mid K_s=k_s,x_s].

The last equality follows because each possible case set has conditional probability equal to its exponential weight divided by \(Z_s\). Differentiating \(\ell_c\) now yields

\nabla_\beta\ell_c(\beta) = \sum_s\left\{T_s-E_\beta[T_s(Y_s)\mid K_s,x_s]\right\}.

To obtain the second derivative, differentiate the expectation, or equivalently differentiate the finite sum defining \(Z_s\) a second time. The result is the covariance of the case-total vector under those same conditional probabilities. Hence

-\nabla_\beta^2\ell_c(\beta) = \sum_s\operatorname{Cov}_\beta\{T_s(Y_s)\mid K_s,x_s\}.

Each covariance matrix is positive semidefinite, so the conditional log-likelihood is concave in \(\beta\). It need not be strictly concave: the covariate totals may not vary in some direction among the configurations permitted by the observed counts.

Worked Example: A Stratum That Cannot Inform the Effect

Consider a stratum of three individuals, all with the same scalar covariate \(x=5\), and suppose exactly one is a case. Every possible case set has covariate total \(5\), hence weight \(e^{5\beta}\). The denominator contains three identical terms, so the conditional probability of the observed configuration is

\frac{e^{5\beta}}{e^{5\beta}+e^{5\beta}+e^{5\beta}}=\frac13.

This contribution is constant in \(\beta\): the case can occupy any one of three positions, but the covariates cannot distinguish those positions. More generally, a stratum with \(K_s=0\) or \(K_s=n_s\) has exactly one possible case configuration and contributes conditional probability \(1\). Such strata provide no information about \(\beta\) through this conditional likelihood.

Interpretation and Common Pitfalls

Conditioning does not estimate each \(\alpha_s\) and then remove it. It changes the likelihood to the distribution of the outcome arrangement given the observed case counts. This is useful when the stratum baselines are nuisance parameters and the scientific target is a common within-stratum covariate effect. It also has a cost: information in the marginal distribution of the case counts is no longer used, so the conditional likelihood does not describe the baseline risks.

Pitfall: treating a case count as if it were a covariate. The count defines which configurations are included in the conditional sample space. It is fixed during the conditional calculation; it is not itself part of the case-set weight.
Pitfall: using a product of Bernoulli denominators in the conditional denominator. Those denominators cancel when conditioning. The correct denominator sums exponential weights over all case sets of the required size.
Pitfall: assuming every stratum contributes information. Strata with no cases, all cases, or no variation in possible case covariate totals can contribute no information about some or all components of \(\beta\).
Pitfall: allowing effects to differ by stratum without changing the model. The derivation assumes a common \(\beta\). If covariate effects vary across strata, the displayed likelihood does not represent that alternative model.

Key takeaway: In an independent Bernoulli logistic model with a separate intercept for each stratum and a common covariate effect, conditioning on each observed case count cancels that stratum’s intercept. The resulting likelihood compares the observed case covariate totals with all totals possible at the same case count.

Check Your Understanding

Use the conditional sample space of case configurations to answer the following.

In a stratum with four individuals and two observed cases, how many case sets appear in the conditional denominator?
Explain which factors in the joint probability of an outcome configuration cancel when conditioning on its stratum case count.
For covariates \(0\) and \(2\) in a stratum with one case, write the conditional probability that the individual with covariate \(2\) is the case.
Why does a stratum with all individuals observed as cases contribute no information about the common covariate effect through this conditional likelihood?
State the modeling assumption that permits the contributions from different strata to be multiplied into one conditional likelihood.