Linearization Error In Binary PQL

Why a Local Approximation Can Miss the Binary Mean

In binary PQL, the conditional mean is approximated by a first-order Taylor expansion of the inverse-logit function around a current linear predictor. This is useful because it replaces a nonlinear mean relationship with a locally linear one. But “locally” matters: the tangent line need not track the logistic curve well when the predictor moves far from the expansion point. The issue is not whether the logistic model is correct; it is how accurately its mean is represented by this particular linear approximation.

The goal here is to quantify that discrepancy. We focus on the deterministic difference between the exact conditional mean and its tangent approximation. It is separate from the randomness of a binary response and from numerical error in evaluating an integral.

Model and Taylor Remainder

For a binary response \(Y\), suppose that, conditional on the random effects, \(Y\) is Bernoulli with mean \(\mu\), and that the logit model is \(\operatorname{logit}(\mu)=\eta\), where \(\eta\) is the linear predictor. Thus the inverse-logit mean is \(f(\eta)=1/(1+\exp(-\eta))\). In a mixed model, \(\eta\) can include fixed effects and random effects, for example \(\eta=x^\mathsf{T}\beta+z^\mathsf{T}b\). The Taylor calculation below applies to any real-valued \(\eta\); it does not require a particular design or distribution for \(b\).

The relevant derivatives follow by differentiating the inverse-logit. Writing \(p=f(\eta)\), the derivative is \(f'(\eta)=p(1-p)\). Differentiating once more gives \(f''(\eta)=p(1-p)(1-2p)\). The first derivative controls the tangent’s slope; the second derivative controls its curvature and hence the leading size and direction of the error.

This is Taylor’s theorem with Lagrange remainder. It applies because the inverse-logit has a continuous second derivative for every real predictor. In particular, the exact remainder equals \(\tfrac12 f''(\xi)\delta^2\) for some intermediate predictor \(\xi\). The remainder is not generally equal to \(\tfrac12 f''(\eta_0)\delta^2\); that expression is a second-order approximation to the remainder when the change is small.

A Bound on the Error

For every real \(\eta\), \(0<f(\eta)<1\), so \(0<p(1-p)\leq 1/4\) and \(|1-2p|\leq 1\). Consequently, \(|f''(\eta)|\leq 1/4\). Substituting this derivative bound into the Lagrange remainder gives a simple global guarantee.

The bound holds for every expansion point and every real predictor change, including changes in either direction. It is conservative: the maximum curvature bound need not be attained anywhere between the two predictors. Even so, it shows why the size of the change matters. If \(|\delta|\) doubles, the bound increases by a factor of four. A small change supports a small absolute error; a large change may leave the bound too broad to be informative.

Curvature also explains the direction of the discrepancy. The logistic curve is convex when \(p<1/2\), corresponding to \(\eta<0\), and concave when \(p>1/2\), corresponding to \(\eta>0\). If the interval from \(\eta_0\) to \(\eta\) stays entirely on one side of zero, the tangent lies below the curve in the convex region and above it in the concave region. If the interval crosses zero, curvature changes sign, so the net remainder must be evaluated rather than inferred from the curvature at just one endpoint.

Worked Numerical Comparisons

What This Means for Binary PQL

At a PQL iteration, the expansion point is the current linear predictor, and the predictor change reflects the proposed change in fixed and random effects. The local mean representation is

This is the particular approximation whose error the calculations above describe. It is not the error in a fitted coefficient, nor is it the difference between an integrated likelihood and a numerical approximation to that likelihood. Those are distinct questions. A Taylor remainder is present even if all arithmetic is exact and the binary-logit model is correctly specified.

A useful first diagnostic is the size of the predictor change over the range relevant to an update. The bound \(\delta^2/8\) gives a guaranteed absolute error for any individual change, while direct comparison of \(f(\eta_0+\delta)\) and \(T_1(\eta_0+\delta)\) gives the exact remainder when both predictors are known. In a mixed model, examining only the change at an average random effect can miss larger changes for other random-effect values.

The tangent is unbounded as a function of \(\delta\), whereas a binary probability must remain between zero and one. Thus a tangent value below zero or above one is an immediate warning that the local approximation is being used beyond a sensible range. The converse is not a guarantee of accuracy: the tangent may remain inside the unit interval and still differ materially from the logistic mean, as the examples show. There is no universal cutoff for an acceptable remainder; it depends on the precision needed for the analysis.

How to Assess an Approximation and What Alternatives Change

A practical assessment has three parts. First, identify the expansion point and the predictor changes the procedure needs to represent. Second, use the bound or evaluate the exact remainder at representative changes. Third, judge the resulting discrepancy in probability units and in the context of the intended use. If the approximation is consequentially inaccurate, the local expansion is a weak basis for representing those means.

The main alternative is to retain the nonlinear logistic mean and evaluate the random-effects integral using a likelihood-based method such as adaptive quadrature or a Laplace approximation. Those approaches address integration of the nonlinear model rather than replacing its mean by a tangent. Their accuracy has its own numerical and approximation considerations, so they are not error-free by definition. The important distinction is that a PQL linearization error is not repaired merely by computing the resulting working-model calculations more precisely.

The Taylor bound is useful but limited. It is an absolute bound on the mean error for a specified predictor change, not a relative-error guarantee, and it may be loose. Near probabilities close to zero or one, an absolute error that seems small can be large relative to the mean itself. Also, a pointwise bound does not by itself quantify how an approximation affects an entire fitted model: that depends on the observations, design, iterative updates, and estimation procedure. This tutorial therefore uses the remainder to diagnose the local approximation, not to claim a general bound on parameter error.

Common Mistakes

• Using only the curvature at the expansion point. A small or zero \(f''(\eta_0)\) does not imply a small remainder over a finite change. Taylor’s theorem uses curvature at an intermediate point.
• Reversing the sign interpretation. Since \(R=f(\eta)-T_1(\eta)\), a negative remainder means the tangent is above the exact mean; a positive remainder means it is below.
• Calling the global bound an estimate. \(\delta^2/8\) is a guaranteed upper bound, not the expected error or the actual error. Calculate the exact remainder when the two predictor values are available.
• Confusing approximation error with response noise. The remainder compares two deterministic means. A Bernoulli response can differ from either mean because it is random.
• Assuming an in-range tangent is accurate. A tangent value between zero and one is not proof that it closely approximates the logistic curve.