Common Misinterpretations of Firth Logistic Regression

What Firth Correction Does—and Does Not—Claim

Firth logistic regression is often introduced in settings where ordinary maximum likelihood estimates can be severely biased in small samples or diverge under separation. That motivation can invite overstatements: that the method is just Bayesian estimation with a Jeffreys prior, that it pulls every coefficient toward zero, or that a finite estimate has somehow removed uncertainty. Each statement confuses a property of the fitting procedure with a broader claim that does not follow.

For independent binary responses, write \(Y_i\sim\operatorname{Bernoulli}(p_i)\), with \(\operatorname{logit}(p_i)=x_i^\mathsf{T}\beta\). Let \(I(\beta)\) denote the expected Fisher information. Firth fitting maximizes the log-likelihood with the Jeffreys-information penalty:

The penalty changes the estimating equations and, in important settings, yields finite estimates where ordinary maximum likelihood does not. It is a bias-reduction method; the penalty alone does not turn the fitted model into a complete Bayesian analysis.

Misinterpretation 1: “Firth Is Just Bayesian Estimation”

The resemblance is real, but the conclusion is too broad. In a chosen coordinate system, the Jeffreys prior has density proportional to \(|I(\beta)|^{1/2}\). Consequently, the expression maximized by Firth fitting is proportional to the likelihood times that prior density. If this defines a proper posterior with an interior mode, the posterior mode in those same coordinates is the Firth estimate.

That numerical relationship does not make Firth inference equivalent to Bayesian inference. A Bayesian analysis specifies a prior and uses the posterior distribution, which can support posterior probabilities, credible intervals, and predictive distributions. Firth fitting is a penalized-likelihood procedure aimed at reducing first-order bias; by itself, it supplies neither posterior uncertainty nor posterior predictive probabilities. Jeffreys priors can also be improper, and a posterior mode, when it exists, depends on the coordinates in which a density is expressed. The mode connection must therefore be stated with its conditions, not treated as an identity between the two inferential frameworks.

Misinterpretation 2: “Every Coefficient Moves Toward Zero”

Firth correction does not apply a universal shrinkage factor to the regression coefficients. The adjustment depends on the design and fitted probabilities. Some estimates may move toward zero, others may move away, and the meaning of “toward zero” itself depends on how predictors and contrasts are coded. A coefficient is not an invariant summary of fitted probabilities under arbitrary recoding.

Misinterpretation 3: “Firth Shrinks Probabilities Toward Zero”

Coefficient movement and probability movement are not interchangeable. Probabilities are obtained by applying the inverse-logit function to a linear predictor; the result depends on the full coefficient vector and the predictor value. Even in the preceding example, the group probabilities move from 0.6 and 0.9 to approximately 0.5909 and 0.8996—not toward zero, but slightly toward 0.5.

Misinterpretation 4: “Finite Estimates Remove Uncertainty”

Finiteness is a property of the estimate, not a statement that the data identify the parameter precisely. Bias reduction does not make a small sample large, recover information that was never observed, or eliminate sampling variability. Uncertainty remains relevant after fitting and must be assessed with an appropriate inferential method.

The estimate formula follows by maximizing the penalized log-likelihood. With \(\eta=\operatorname{logit}(p)\), the derivative is \(Y-np+\tfrac12(1-2p)\). Setting it to zero yields \(Y+\tfrac12=(n+1)p\), hence \(\tilde p=(Y+\tfrac12)/(n+1)\). To obtain its sampling variance, use that \(E[Y]=np\) and \(\operatorname{Var}(Y)=np(1-p)\):

The strict inequality holds for \(n>0\) and \(0<p<1\). The estimate is finite for every possible observed count, but it still varies across repeated samples. This exact calculation demonstrates that avoiding an infinite estimate is not the same as eliminating uncertainty.

Why These Distinctions Matter

The practical value of Firth correction is clearer when its claims are kept narrow. It modifies the likelihood to reduce first-order bias and can prevent divergence under separation. Those are meaningful properties, but they do not imply universal coefficient shrinkage, probability shrinkage, or certainty. Nor does the Jeffreys-information penalty by itself provide the posterior distribution that a Bayesian analysis would use.

A common mistake is to reason, “The penalty comes from a Jeffreys prior, so the estimate is Bayesian; the penalty pulls the model toward zero, so every effect gets smaller; and because the estimate is finite, the problem is resolved.” The correction is to check each claim separately: the prior connection concerns an objective function and, under conditions, a posterior mode in matching coordinates; the coefficient example shows that a slope can increase in magnitude; and the exact variance calculation shows that finite estimates retain sampling uncertainty.

When interpreting a fitted model, distinguish parameter estimates from predicted probabilities and point estimates from uncertainty statements. A correction to the estimating procedure changes neither the need to understand the model’s covariate coding nor the need to quantify uncertainty.