Sufficient Statistics for Conditional Logistic Models

From Stratum Totals to Slope Sufficiency

In a conditional logistic model, each stratum has its own intercept. Conditioning on the number of cases in each stratum removes those intercepts, but it does not remove the slope parameter: the relative covariate values of the cases still carry information about the slopes. The statistic that summarizes this information is the sum of the case covariate vectors across strata.

The distinction between two kinds of conditioning matters. Conditioning on the stratum totals produces a model that still depends on the slopes. Conditioning further on the slope sufficient statistic produces a distribution that no longer depends on them. That second distribution is over the possible labeled case assignments for the fixed covariates; it is uniform over assignments with the same sufficient statistic.

Definition: Consider fixed covariate vectors $x_{ij}\in\mathbb{R}^p$ and binary outcomes $Y_{ij}\in\{0,1\}$, for strata $i=1,\ldots,K$ and units $j=1,\ldots,n_i$. Under the stratum-intercept logistic model, $$ P(Y_{ij}=1\mid x_{ij})=\frac{\exp(\alpha_i+\beta^\mathsf{T}x_{ij})}{1+\exp(\alpha_i+\beta^\mathsf{T}x_{ij})}, $$ where $\alpha_i$ is the intercept for stratum $i$ and $\beta\in\mathbb{R}^p$ is the common slope vector. Define the stratum total $M_i=\sum_{j=1}^{n_i}Y_{ij}$ and the slope statistic $$ T(Y)=\sum_{i=1}^K\sum_{j=1}^{n_i}Y_{ij}x_{ij}. $$ The conditional slope sufficient statistic is $T(Y)$, after conditioning on all the totals $M_1,\ldots,M_K$.

Here the covariates are treated as fixed, and the outcomes are independent given them and the model parameters. For a particular stratum, conditioning on $M_i=m_i$ restricts the possible outcomes to subsets of $m_i$ units designated as cases. This is the conditional logistic likelihood for matched sets: a subset receives weight proportional to the exponential of $\beta$ times its case covariate sum.

P(Y=y\mid M_1=m_1,\ldots,M_K=m_K;\beta) = \frac{\exp\{\beta^\mathsf{T}T(y)\}} {\displaystyle\prod_{i=1}^K \left[ \sum_{\substack{A\subseteq\{1,\ldots,n_i\}\\|A|=m_i}} \exp\!\left\{\beta^\mathsf{T}\sum_{j\in A}x_{ij}\right\} \right]}.

The denominator sums over every allowed case assignment in each stratum. It depends on the fixed covariates, the conditioned totals, and $\beta$, but not on the observed assignment. The numerator depends on the assignment through $T(y)$ alone.

Deriving the Conditional Distribution

The first result establishes sufficiency directly. The second gives the distribution after conditioning on the sufficient statistic itself. The sample space in both results is finite: it consists of all labeled outcome vectors having the specified stratum totals.

Theorem: Given $M_i=m_i$ for every stratum, $T(Y)$ is sufficient for $\beta$. If $t$ is attainable, then for any allowed assignment $y$ with $T(y)=t$, $$ P(Y=y\mid M_1=m_1,\ldots,M_K=m_K,\ T(Y)=t;\beta) = \frac{1}{N(t)}, $$ where $N(t)$ is the number of allowed labeled assignments with statistic $t$. In particular, this conditional distribution does not depend on $\beta$.

Proof. The conditional probability displayed above for the stratum-total model factors into a term depending on $y$ and $\beta$ only through $\exp\{\beta^\mathsf{T}T(y)\}$, and a factor independent of $y$. The factorization criterion therefore gives sufficiency. Alternatively, summing that probability over all allowed assignments with $T(y)=t$ gives $$ P(T(Y)=t\mid M_1=m_1,\ldots,M_K=m_K;\beta) = \frac{N(t)\exp\{\beta^\mathsf{T}t\}} {\displaystyle\prod_{i=1}^K \left[ \sum_{|A|=m_i} \exp\!\left\{\beta^\mathsf{T}\sum_{j\in A}x_{ij}\right\} \right]}. $$ For any particular such assignment $y$, divide its conditional probability given the stratum totals by this probability for $T(Y)=t$. Its numerator is $\exp\{\beta^\mathsf{T}t\}$, so the common exponential and denominator cancel, leaving $1/N(t)$. The result is parameter-free. $\square$

This is a distribution over assignments of case labels to the observed units, not a claim that the covariates themselves were sampled from a uniform distribution. If several assignments produce the same multiset of case covariates, that multiset’s probability is the sum of the probabilities of its assignments.

Worked Example: A Matched Pair and Its Sufficient Statistic

One matched pair has covariate values $x_{11}=0$ and $x_{12}=2$, and exactly one member is a case. The two allowed assignments have case covariate sums 0 and 2. Derive their conditional probabilities and evaluate them at $\beta=\log 3$.

For the assignment in which the first member is the case, the weight is $\exp(\beta\cdot 0)=1$. For the assignment in which the second member is the case, the weight is $\exp(\beta\cdot 2)=\exp(2\beta)$. Therefore the conditional probabilities are $$ P(\text{first is case}\mid M_1=1)=\frac{1}{1+\exp(2\beta)},\qquad P(\text{second is case}\mid M_1=1)=\frac{\exp(2\beta)}{1+\exp(2\beta)}. $$

At $\beta=\log 3$, $\exp(2\beta)=\exp(2\log 3)=9$. The probabilities are $1/10$ and $9/10$, respectively. The observed case covariate, $T(Y)$, is 0 or 2 and indexes which of these assignments occurred. Once $T(Y)$ is also fixed, only one assignment remains, so its conditional probability is 1.

Worked Example: Two Slopes and Two Matched Pairs

There are two strata, each with one case. In the first, the covariates are $(0,0)$ and $(1,0)$; in the second, they are $(0,0)$ and $(0,1)$. Let $\beta=(\beta_1,\beta_2)^\mathsf{T}$. Find the conditional probability that the nonzero-covariate member is the case in both strata when $\beta_1=\log 2$ and $\beta_2=\log 3$.

The first stratum’s two assignment weights are $1$ and $\exp(\beta_1)$; its normalizing sum is $1+\exp(\beta_1)$. The second stratum’s weights are $1$ and $\exp(\beta_2)$, with normalizing sum $1+\exp(\beta_2)$. The conditional likelihood for an assignment is the product of its two stratum weights divided by $(1+\exp(\beta_1))(1+\exp(\beta_2))$.

For the assignment specified, the case covariate vectors are $(1,0)$ and $(0,1)$, so $T=(1,1)$. Its weight is $\exp(\beta_1+\beta_2)=2\cdot3=6$, and the denominator is $(1+2)(1+3)=12$. Thus the probability is $6/12=1/2$. The two coordinates of $T$ record the case covariate contributions for both slopes; neither stratum intercept appears.

Minimal Sufficiency and What the Statistic Retains

Sufficiency says that the conditional likelihood depends on the observed assignment through $T$. Under a regular parameter-space condition, $T$ also distinguishes assignments that cannot be combined into a smaller sufficient partition.

Theorem: Suppose the conditional sample space is finite and the allowed parameter values for $\beta$ contain a nonempty open subset of $\mathbb{R}^p$. Then $T(Y)$ is minimal sufficient for $\beta$ in the model conditional on the stratum totals.

Proof. For two allowed assignments $y$ and $y'$, their conditional probability ratio given the stratum totals is $$ \frac{P(Y=y\mid M;\beta)}{P(Y=y'\mid M;\beta)} = \exp\{\beta^\mathsf{T}[T(y)-T(y')]\}, $$ where $M$ denotes the vector of conditioned stratum totals. The common normalizing denominator cancels. If $T(y)=T(y')$, this ratio is constant in $\beta$. Conversely, suppose it is constant on the allowed open subset. Set $d=T(y)-T(y')$. If $d\ne0$, choose an interior point $\beta_0$ and a sufficiently small nonzero scalar $s$ such that $\beta_0+sd$ remains in that subset. The log ratio then changes by $s d^\mathsf{T}d$, which is nonzero. This contradicts constancy. Thus a constant ratio requires $d=0$. By the likelihood-ratio criterion for minimal sufficiency on a finite sample space, assignments are in the same minimal sufficient class exactly when their $T$ values agree. $\square$

The open-set condition is substantive. If only a restricted set of slope values is allowed, or if the design makes some directions in the covariates indistinguishable, different $T$ values can produce the same likelihood ratio throughout the allowed parameter space. Sufficiency still follows from the factorization; the stated minimality conclusion need not.

Worked Example: Repeated Covariates and Assignment Multiplicity

One stratum has four labeled units with scalar covariates $0,1,1,2$, and exactly two cases. The two units with covariate 1 are distinct individuals. Enumerate the possible case sums and determine the distribution given $T=1$.

There are $\binom{4}{2}=6$ labeled assignments. Choosing the covariate-0 unit and either one of the two covariate-1 units gives two assignments with $T=1$. Choosing the 0 and 2 gives one assignment with $T=2$; choosing both 1s gives another assignment with $T=2$. Choosing either 1 and the 2 gives two assignments with $T=3$. The assignment counts for $T=1,2,3$ are therefore $2,2,2$.

The conditional probability of each assignment given the stratum total is proportional to $\exp(\beta T)$. Consequently, $$ P(T=1\mid M=2;\beta) = \frac{2\exp(\beta)}{2\exp(\beta)+2\exp(2\beta)+2\exp(3\beta)}. $$ Given $T=1$, precisely two labeled assignments remain, and each has probability $1/2$. Both assignments have the same case covariate multiset, $\{0,1\}$, but differ in which labeled unit with covariate 1 is a case. Thus the probability of that covariate pattern is 1, while the probability of either particular assignment is $1/2$.

Common Pitfall and Why the Distinction Matters

A common mistake is to claim that conditioning on the slope sufficient statistic gives equal probability to each distinct covariate pattern. The theorem gives equal probability to each allowed labeled assignment. If patterns have different numbers of assignments representing them, their probabilities are proportional to those multiplicities. In the repeated-covariate example, merging the two labeled assignments into one pattern does not preserve the probability of either assignment.

Another mistake is to condition on $T$ and then treat the resulting law as if it still contained information about $\beta$. Once $T$ is fixed, every allowed assignment has the same $\beta$-dependent factor $\exp(\beta^\mathsf{T}T)$, which cancels in the conditional distribution. The slope information lies in the distribution of $T$ under the stratum-total-conditioned model, not in the assignment among outcomes with the same $T$.

This separation is useful because it identifies exactly what the conditional model retains. The stratum totals remove the nuisance intercepts; the covariate sum captures the remaining slope dependence; and conditioning on that sum yields a parameter-free reference distribution over compatible assignments. Keeping these stages distinct prevents the sufficient statistic from being mistaken either for a nuisance statistic or for a distribution over distinct covariate values.

Check Your Understanding

Answer each question using the fixed-covariate conditional logistic model described above.

For stratum totals $M_i=m_i$, write the slope statistic $T(Y)$ and explain which part of the conditional likelihood depends on the observed assignment.
Why does conditioning on $T(Y)=t$ make the probability of each compatible labeled assignment independent of $\beta$?
In the repeated-covariate example, why are there two assignments with $T=1$ even though they have the same case covariate multiset?
What condition on the allowed values of $\beta$ is used in the minimal-sufficiency proof?
Explain why the distribution of $T$ given the stratum totals can depend on $\beta$, while the distribution of assignments given both the stratum totals and $T$ does not.

Sufficient Statistics for Conditional Logistic Models

What You'll Learn