When Collapsing Factor Levels Is a Bad Remedy

Why a Sparse Level Is Not Just a Numerical Problem

A factor level with few observations can create separation: its observed outcomes may line up so completely with the level that the corresponding coefficient has no finite maximum-likelihood estimate. Merging that level with another can sometimes remove the pattern, but it is not a neutral repair. The categories may represent distinct clinical states, exposures, or processes, and combining them changes what the model is asked to estimate. A finite estimate from the merged model may answer a different question from the one that motivated the analysis.

The key question is therefore not simply whether merging makes the software finish. It is whether the merged categories can reasonably be treated as the same level for the scientific purpose, and whether the reduced model still has a separating direction. The data summaries and calculations below are hypothetical and are used to make those distinctions explicit.

Model and Meaning of a Level Merger

For binary outcomes \(Y_{ij}\) observed for unit \(i\) in cluster \(j\), consider a random-intercept logistic GLMM:

Y_{ij}\mid b_j \sim \operatorname{Bernoulli}(p_{ij}),\qquad \operatorname{logit}(p_{ij})=\beta_0+\mathbf{x}_{ij}^{\mathsf T}\boldsymbol{\beta}+b_j,\qquad b_j\sim N(0,\tau^2).

Here observations are conditionally independent given the cluster effects and covariates; the \(b_j\) are independent draws from the stated normal distribution; and the fixed-effects design must encode the factor levels and any interactions being modeled. The logit is \(\operatorname{logit}(p)=\log\{p/(1-p)\}\). A factor with levels A, B, and C may be represented, using C as reference, by indicators for A and B, with coefficients \(\beta_A\) and \(\beta_B\). These coefficients describe conditional log-odds contrasts with C, holding other fixed predictors and \(b_j\) constant.

Definition: Collapsing two factor levels replaces them with a common category in the model. In the simple treatment-coded model above, merging A and B imposes the restriction \(\beta_A=\beta_B\): both levels have the same fixed-effects contribution relative to C. It does not merely rename the original estimates or improve their precision while leaving the model unchanged.

This description assumes that the merged observations can be treated as exchangeable with respect to the role of this factor in the model. If the factor has interactions, merging can also constrain the interaction structure. The GLMM’s random effects account for modeled cluster heterogeneity; they do not establish that distinct factor levels have the same scientific meaning, nor do they guarantee finite fixed-effect estimates.

What the Restriction Does: A Short Derivation

With separate indicators, the fixed part of the linear predictor for an observation in A is \(\beta_0+\beta_A\), while in B it is \(\beta_0+\beta_B\). Once the categories are merged into M, there is one indicator and one coefficient, say \(\beta_M\). For the merged model to reproduce the separate model’s linear predictors in both A and B, it must satisfy

\beta_M=\beta_A \quad\text{and}\quad \beta_M=\beta_B, \quad\text{so}\quad \beta_A=\beta_B.

Thus, the merged model only permits the part of the original parameter space in which A and B have equal fixed effects. In a model with additional predictors, this restriction applies to their factor contributions, conditional on those predictors and the random effects. It does not assert that the raw event rates in A and B must be equal in every finite sample; it asserts that the fitted factor effect is shared.

Collapsing can remove a separating direction if that direction depended on assigning different coefficients to A and B. But it need not do so. If the remaining design still permits a direction that improves the likelihood by moving fitted probabilities toward the observed zeros and ones, the reduced model can remain separated. The precise likelihood behavior in a GLMM also depends on its random-effects structure and likelihood formulation; a changed factor coding alone is not evidence that a finite maximum has been obtained.

Worked Example: Opposing Sparse Levels Become a Null Contrast

Suppose a hypothetical multicenter study records a binary outcome for patients in three factor levels. Across the observed clusters, the counts are A: 0 events among 6 patients; B: 6 events among 6; and C: 4 events among 8. Assume A and B denote distinct clinical subtypes. The table is a descriptive summary; it does not by itself provide a fitted GLMM estimate.

Level	Events	No events	Observed event proportion
A	0	6	0/6 = 0
B	6	0	6/6 = 1
C	4	4	4/8 = 0.5

In a factor-only logistic model, the observed proportions in A and B lie at the boundaries. Their empirical logits are \(\log\{0/(1-0)\}=-\infty\) and \(\log\{1/(1-1)\}=+\infty\); correspondingly, the separate A and B contrasts are separated in this simple illustration. The same outcome pattern across clusters can also signal a separating fixed-effect direction in a GLMM; the random intercept does not make A and B scientifically equivalent.

If A and B are merged, their combined proportion is

\hat p_M=\frac{0+6}{6+6}=\frac{6}{12}=0.5.

C also has proportion \(4/8=0.5\). In the simple factor-only binomial calculation, the merged-versus-C log-odds contrast is therefore \(\log\{0.5/(1-0.5)\}-\log\{0.5/(1-0.5)\}=0-0=0\). This arithmetic shows how pooling can produce an apparently ordinary contrast while concealing the opposite outcomes in the original levels. It is not an estimate from the GLMM: cluster adjustment and other predictors can change fitted contrasts. If the subtype distinction is scientifically relevant, the zero contrast for the merged category answers the wrong question.

When Collapsing Does Not Remove Separation

A merger only changes the design columns associated with the merged factor. It cannot automatically fix an outcome pattern carried by the remaining categories or other predictors. Before interpreting a finite coefficient as a remedy, inspect the outcomes after applying the proposed grouping and check the rest of the fixed-effects design as well.

Worked Example: The Merged Category Is Still Separated

In a second hypothetical multicenter dataset, level A has 0 events among 4 patients, level B has 0 among 3, and level C has 4 among 4. Suppose A and B are genuinely interchangeable for the scientific question, so combining them is defensible. The merged level M has

\hat p_M=\frac{0+0}{4+3}=\frac{0}{7}=0, \qquad \hat p_C=\frac{4}{4}=1.

The merger has not produced a mixed outcome pattern across M and C: all seven observations in M have no event and all four in C have an event. In the corresponding factor-only logistic illustration, the M-versus-C contrast remains separated, with boundary logits \(-\infty\) and \(+\infty\). In a GLMM, this summary is a warning to investigate the design and likelihood, not a substitute for fitting or diagnosing the model. The arithmetic is enough to show that a scientifically permissible merger does not guarantee that the separation disappears.

How to Decide Whether a Merger Is Defensible

Use the scientific meaning of the categories, not the observed outcome pattern alone, to decide whether levels belong together. A merger is more plausible when levels are different labels for the same underlying state or were distinguished by a coding convention irrelevant to the study question. It is much harder to justify when levels represent different interventions, disease subtypes, exposure intensities, or populations for which different outcome associations are of substantive interest.

The choice should also respect the intended target. If the question asks whether A differs from B, a model that imposes \(\beta_A=\beta_B\) cannot estimate that contrast. If a pre-specified scientific question concerns a broader combined category, the merged model may be appropriate, but its result should be described as the effect for that combined category, not as evidence that the original levels were equivalent.

For each proposed merger, check the following:

Meaning: Is there a substantive reason to regard the levels as one category, independent of the observed outcomes?
Target: Does the merged contrast still answer the stated scientific question?
Pattern: After merging, do the combined levels contain outcome variation, and are other predictors or factor levels still separating?
Structure: Does the merger alter interactions, cluster composition, or the interpretation of other coefficients?
Sensitivity: How do estimates and conclusions compare under the separate-level specification and scientifically plausible alternatives?

Sparse cells make estimated rates unstable, and a zero or one observed proportion is not proof that the underlying event probability is exactly zero or one. Conversely, pooling can mask real heterogeneity: a combined proportion is a weighted summary, with weights determined by the numbers observed in each original level. It should not be read as evidence that the component levels have a common risk.

Worked Example: Check the Scientific Target Before Pooling

A hypothetical safety analysis distinguishes exposure levels A and B because they correspond to different doses. In the observed data, A has 1 event among 5 observations and B has 5 among 5. Suppose a researcher proposes merging them because A is sparse. The combined observed event proportion is

\hat p_M=\frac{1+5}{5+5}=\frac{6}{10}=0.6.

The calculation is correct, but it does not establish that the dose levels share an effect. In a model with a reference level C, a merged factor estimates one common conditional contrast for M versus C; the separate model allows the A-versus-C and B-versus-C contrasts to differ. Because dose is the scientific distinction of interest, the merger removes the very contrast the analysis is meant to assess. A pooled estimate of 0.6 describes the observed mixture, not a dose-specific probability and not a causal effect.

A defensible next step is to retain the dose distinction and consider remedies suited to the estimation problem, such as additional outcome information or an appropriate regularization approach. If a combined dose category is scientifically meaningful and chosen on that basis, it can be analyzed as a different target, with the changed interpretation made explicit.

Diagnostics, Alternatives, and Limitations

Treat a merger as a modeling decision and then repeat the separation diagnostics for the resulting model. Examine outcome counts by factor level and cluster, the fitted fixed-effects design, score and information behavior, profile likelihood, and convergence warnings. A coefficient that remains extreme, a nearly flat likelihood direction, or poor information after collapsing suggests that the underlying problem remains. A finite estimate alone is not proof of stable inference: pooling can simply conceal the original contrast or make the reduced model’s estimate appear well-behaved.

When distinct levels must remain distinct, alternatives include improving the data collection so that informative outcomes occur across levels and clusters, or using an appropriate regularization method for separated effects. The earlier material on proper priors, Firth correction, and ridge penalization addresses those approaches. They impose different assumptions or penalties and do not make the data more informative, but they can preserve the factor distinctions rather than forcing equal effects through recoding. No remedy removes the need to state the target and examine sensitivity.

Collapsing also has limits when the original data are extremely sparse: a reduced model may still have too little information for precise estimation, particularly with complex interactions or cluster-specific patterns. If only some levels are combined, document the rule and its scientific rationale. Avoid trying many outcome-informed groupings and reporting only the one that yields finite estimates; that practice makes the analysis dependent on the observed outcomes and can understate uncertainty.

Common Mistake

Common mistake: “The level was sparse and the merged model returned a finite coefficient, so collapsing solved separation without affecting the analysis.”

Correction: A finite coefficient only describes the restricted model that was fitted. Merging imposes shared factor effects, may erase a scientifically important contrast, and may leave another separating direction intact. Justify the grouping independently of the outcome pattern, confirm that the reduced model addresses the intended question, and repeat the diagnostics.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a treatment-coded model with levels A, B, and reference C, what restriction does merging A and B impose on their fixed effects?
A has 0 events among 5 observations and B has 5 among 5. What pooled event proportion results from merging them, and what important information does it conceal?
Two scientifically interchangeable levels each have only non-events, while a remaining level has only events. Does merging the first two guarantee removal of separation? Explain.
Why is a finite merged-model coefficient not evidence that the original A-versus-B contrast has been estimated?
Name two diagnostics to revisit after a proposed merger and one alternative when scientifically distinct levels must be retained.

Solutions. (1) It imposes \(\beta_A=\beta_B\), so the two levels share one fixed-effects contribution. (2) The pooled proportion is \((0+5)/(5+5)=0.5\). It conceals that the observed outcomes were opposite in A and B. (3) No. The merged category still contains only non-events while the remaining level contains only events, so the separating pattern can persist. (4) The merged model constrains A and B to have the same effect; it estimates a combined-category contrast, not their separate contrast. (5) For example, revisit score or information behavior and profile likelihood, along with outcome patterns by level and cluster. If the distinction must remain, consider additional data or a suitable regularization method.