Taylor Linearization Of The Binary Mean

Why Linearize a Binary Mean?

In a binary generalized linear mixed model, the conditional success probability is a nonlinear function of a predictor that includes both fixed and random effects. Penalized quasi-likelihood (PQL) makes this nonlinear structure more manageable by replacing the mean locally with a linear approximation. The key is that the approximation is local: its accuracy depends on the expansion point, the size of the predictor change, and the curvature of the mean function.

This tutorial derives that approximation for a general binary inverse link and then specializes to the logit link. The expansion is for the conditional mean at fixed covariates and random effects. It is not an expansion of the observed binary outcome, nor does it integrate over the random-effects distribution.

For the usual Bernoulli model, \(Y_{ij}\mid\mathbf{b}_i\) has a Bernoulli distribution with success probability \(\mu_{ij}\). The Taylor identity below is a statement about the function \(h\); it does not require a particular distribution for \(\mathbf{b}_i\). A model may additionally assume, for example, that \(\mathbf{b}_i\sim N(\mathbf{0},\mathbf{D})\), and that outcomes are conditionally independent given the random effects and covariates.

Expansion at Current Fixed- and Random-Effect Estimates

Suppose the current values are \(\boldsymbol\beta^{(0)}\) and \(\mathbf{b}_i^{(0)}\). Define the current predictor and current conditional mean by

For candidate values \(\boldsymbol\beta\) and \(\mathbf{b}_i\), their predictor change from this expansion point is

Thus the first-order expansion includes changes in both components. Holding the random effect fixed would omit the second term; holding the fixed effects fixed would omit the first. Neither omission is appropriate when both are being updated.

The integral expression is an exact remainder, not an additional approximation. It follows by applying the fundamental theorem of calculus twice to \(g(t)=h(\eta_{ij}^{(0)}+t\Delta\eta_{ij})\). Since \(g'(t)=\Delta\eta_{ij}h'(\eta_{ij}^{(0)}+t\Delta\eta_{ij})\) and \(g''(t)=(\Delta\eta_{ij})^2h''(\eta_{ij}^{(0)}+t\Delta\eta_{ij})\), the second-order integral identity for \(g(1)\) gives the displayed formula. If \(h''\) is continuous, the same remainder can also be written as \(\tfrac12 h''(\eta_{ij}^{(0)}+\theta\Delta\eta_{ij})(\Delta\eta_{ij})^2\) for some \(0<\theta<1\) when \(\Delta\eta_{ij}\ne0\).

The segment condition matters: differentiability only in a small neighborhood of the expansion point is not enough if the candidate predictor lies outside that neighborhood. A useful bound follows if \(|h''(u)|\le M\) everywhere on the segment:

This makes the local nature explicit. With bounded curvature, the discarded error is at most of second order in the predictor change.

Logit Mean and Its Local Slope

For the logit link, \(h(\eta)=\exp(\eta)/(1+\exp(\eta))\). Its derivatives are

The linearized mean therefore uses the current success probability as its intercept and the current probability slope as its multiplier:

The slope is largest at a mean of \(0.5\) and becomes small near 0 or 1. The second derivative changes sign at \(\mu=0.5\): the logistic mean is locally convex when \(\eta<0\), locally concave when \(\eta>0\), and has zero second derivative at \(\eta=0\). Consequently, the tangent line can lie below or above the true mean depending on the portion of the predictor scale being traversed.

How Fixed- and Random-Effect Changes Enter

The expansion can be applied directly to the parameter and random-effect updates. It does not require introducing a separate approximation for each part: their changes first add on the predictor scale, and the derivative of \(h\) then converts that total change to the mean scale.

When the Tangent Approximation Is Not Adequate

A small remainder is not guaranteed merely because the inverse link is smooth. A large predictor change can cross a region of substantial curvature, and the local slope may poorly represent the mean farther from the expansion point. The next calculation illustrates that problem.

Several checks help identify when the linearization is being used beyond its reliable range:

• Inspect predictor changes. Large values of \(|\Delta\eta_{ij}|\) make the second-order remainder more consequential, especially where \(|h''|\) is appreciable.
• Compare with the exact mean. At a candidate update, calculate \(h(\eta_{ij})\) as well as the linearized value. A large discrepancy signals that the local approximation should not be interpreted as accurate at that update.
• Check the probability range. The exact binary mean lies between zero and one, but a tangent line need not. A linearized value outside \([0,1]\) is a clear sign that the approximation is not a valid probability at that point.
• Check the expansion point. The derivative must be evaluated at the current predictor \(\eta_{ij}^{(0)}\), and the remainder condition must cover the entire segment to the candidate predictor.

What This Linearization Does—and Does Not Do

Taylor linearization is a local approximation to the conditional mean as the fixed and random effects change. It is one component of the PQL approximation described in the earlier material on Penalized Quasi-Likelihood. It does not by itself specify the full estimation procedure, its criterion, or the updates used to fit a working mixed model.

It is also distinct from marginalizing over random effects. The quantity expanded here is \(h(\mathbf{x}_{ij}^{\mathsf T}\boldsymbol\beta+\mathbf{z}_{ij}^{\mathsf T}\mathbf{b}_i)\) at a particular random-effect value. The marginal mean would instead involve an expectation such as \(E_{\mathbf b}[h(\mathbf{x}_{ij}^{\mathsf T}\boldsymbol\beta+\mathbf{z}_{ij}^{\mathsf T}\mathbf b)]\), which generally is not obtained by simply applying \(h\) to the mean predictor. No marginalization occurs in the Taylor identity above.

The main alternative at a given candidate parameter value is to evaluate the nonlinear conditional mean directly, rather than replace it by a tangent line. That preserves the exact inverse-link relationship but does not supply the local linear surrogate used by PQL. A second-order Taylor approximation retains the quadratic curvature term, but it remains an approximation and requires the same care about the interval and remainder. The first-order version is computationally useful precisely because it is simpler; its limitation is that it discards curvature.