Working Weights For Binary PQL

Why Binary PQL Uses Fitted-Probability Weights

In a binary generalized linear mixed model, the response is Bernoulli, but its conditional variance depends on its conditional mean. PQL locally represents the response on the linear-predictor scale, where a working linear mixed model can be used. The uncertainty of that locally transformed response is therefore not constant across observations. Its inverse variance gives the observation-level working weight.

For binary logit PQL, the weight has a particularly simple form: it is the fitted probability multiplied by one minus that probability. This tutorial derives that result and clarifies what it does—and does not—mean. The weight is not the probability that an observation is an event, nor is it the inverse variance of the original binary response.

Definition: For observation \(i\), let \(Y_i\) be a Bernoulli response with conditional mean \(\mu_i\), and let \(\eta_i\) be its conditional linear predictor. The PQL working response is the local linearization on the predictor scale. Its working weight \(w_i\) is the inverse of its conditional variance under the first-order approximation. For a Bernoulli response with a logit link and unit dispersion, \(w_i=\mu_i(1-\mu_i)\), evaluated at the current fitted probability.

Model and Assumptions

Write \(i\) for an observation, which belongs to a cluster or subject represented by random effects. Conditional on those random effects and the model covariates, assume

Y_i\mid b \sim \operatorname{Bernoulli}(\mu_i),\qquad \operatorname{logit}(\mu_i)=\eta_i=x_i^\mathsf{T}\beta+z_i^\mathsf{T}b.

Here \(b\) denotes the random effects, \(x_i\) and \(z_i\) are the fixed- and random-effects design vectors, and \(\beta\) is the fixed-effects coefficient vector. Conditional independence of the Bernoulli responses is assumed given the random effects and covariates. The derivation uses the Bernoulli variance, a logit link, and unit dispersion; it does not require a particular random-effects distribution to calculate the observation-level weight.

At a given PQL linearization, let \(\eta_i^{(0)}\) be the current predictor value and \(\mu_i^{(0)}\) its fitted probability. For a finite predictor,

\mu_i^{(0)}=\frac{\exp(\eta_i^{(0)})}{1+\exp(\eta_i^{(0)})}, \qquad 0<\mu_i^{(0)}<1.

The superscript \((0)\) labels the current point of linearization; it does not indicate that the probability is known to be the true probability. As the linearization point changes, the fitted probability and the weight change with it.

Deriving the Working Weight

The logit link is \(g(\mu)=\log\{\mu/(1-\mu)\}\). Its inverse is the logistic mean function, whose derivative with respect to the predictor is

\frac{d\mu_i}{d\eta_i} =\mu_i(1-\mu_i).

A first-order Taylor expansion of the mean around \(\eta_i^{(0)}\) gives

\mu_i\ \approx\ \mu_i^{(0)} +\left.\frac{d\mu_i}{d\eta_i}\right|_{\eta_i^{(0)}} (\eta_i-\eta_i^{(0)}) =\mu_i^{(0)}+\mu_i^{(0)}(1-\mu_i^{(0)})(\eta_i-\eta_i^{(0)}).

Rearranging this local approximation produces the working response \(Y_i^{*}=\eta_i^{(0)}+(Y_i-\mu_i^{(0)})/\{\mu_i^{(0)}(1-\mu_i^{(0)})\}\). To find its working variance, hold the current linearization point fixed and take conditional variance. The Bernoulli variance is \(\mu_i^{(0)}(1-\mu_i^{(0)})\). Therefore,

\begin{aligned} \operatorname{Var}(Y_i^{*}\mid b) &\approx \frac{\operatorname{Var}(Y_i\mid b)} {\left\{\left.d\mu_i/d\eta_i\right|_{\eta_i^{(0)}}\right\}^{2}}\\ &= \frac{\mu_i^{(0)}(1-\mu_i^{(0)})} {\{\mu_i^{(0)}(1-\mu_i^{(0)})\}^{2}}\\ &=\frac{1}{\mu_i^{(0)}(1-\mu_i^{(0)})}. \end{aligned}

The working weight is the reciprocal of this approximate working-response variance:

w_i=\frac{1}{\operatorname{Var}(Y_i^{*}\mid b)} =\mu_i^{(0)}(1-\mu_i^{(0)}).

This is a local, first-order variance calculation, not an exact claim that the transformed binary response has a normal distribution. In the corresponding weighted linear mixed-model representation, the working response has approximate variance \(1/w_i\) under the unit-dispersion convention. Equivalently, the working weight is the squared derivative of the mean with respect to the predictor divided by the response variance:

w_i= \frac{\left\{\left.d\mu_i/d\eta_i\right|_{\eta_i^{(0)}}\right\}^{2}} {\operatorname{Var}(Y_i\mid b)} = \frac{\{\mu_i^{(0)}(1-\mu_i^{(0)})\}^{2}} {\mu_i^{(0)}(1-\mu_i^{(0)})}.

Worked Example: A Fitted Probability Below One-Half

Suppose a hypothetical observation has current predictor \(\eta_i^{(0)}=-1.20\). Calculate its fitted probability and PQL working weight.

First convert the predictor to a probability: \(\mu_i^{(0)}=\exp(-1.20)/\{1+\exp(-1.20)\}=0.2315\), to four significant figures. The complementary probability is \(1-0.2315=0.7685\).

The working weight is \(w_i=0.2315(0.7685)=0.1779\). Thus this observation contributes working precision of about \(0.1779\) on the local predictor-scale representation. This number is not a 17.79% event probability: the fitted event probability is \(0.2315\).

How Fitted Probabilities Control Weight

For any fitted probability \(p\) between zero and one, \(w=p(1-p)\). This function is symmetric around one-half: a fitted probability \(p\) and its complement \(1-p\) have the same weight. It reaches its maximum at \(p=0.5\), where the weight is \(0.25\), and approaches zero as \(p\) approaches either boundary.

The maximum follows directly by completing the square:

p(1-p)=p-p^2=\frac{1}{4}-\left(p-\frac{1}{2}\right)^2\leq\frac{1}{4}.

Equality holds exactly when \(p=1/2\). The conditional Bernoulli response has greatest variance at that probability, but the working weight is an inverse variance after the response has been locally transformed. The transformation’s slope also depends on \(p\); combining the slope and Bernoulli variance yields \(p(1-p)\) as the precision weight.

Consequently, the weight decreases for fitted probabilities near zero or one. On the predictor scale, the logit mean changes slowly in those regions: a change in \(\eta_i\) produces a relatively small change in \(\mu_i\). The local transformation from response to predictor scale then has high approximate variance, and hence low working weight.

Worked Example: Comparing Central and Extreme Probabilities

Consider three hypothetical observations with current fitted probabilities \(0.50\), \(0.95\), and \(0.99\). Calculate and compare their weights.

For \(0.50\), \(w=0.50(1-0.50)=0.2500\). For \(0.95\), \(w=0.95(0.05)=0.04750\). For \(0.99\), \(w=0.99(0.01)=0.009900\).

The \(0.95\) observation has \(0.04750/0.2500=0.1900\) times the weight of the \(0.50\) observation. The \(0.99\) observation has \(0.009900/0.2500=0.03960\) times its weight. Under this local working model, the most extreme fitted probability has much less precision than the central one. This comparison concerns working weights, not whether either observation is correctly classified.

Worked Example: The Outcome Does Not Set the Weight

Two hypothetical Bernoulli observations have the same current fitted probability, \(\mu_i^{(0)}=0.80\), but their recorded outcomes are \(Y_1=1\) and \(Y_2=0\). Find each working weight and explain the comparison.

For the first observation, \(w_1=0.80(1-0.80)=0.80(0.20)=0.1600\). For the second, \(w_2=0.80(1-0.80)=0.1600\) as well. The outcome changes the working residual and therefore the working response, but it does not enter the weight formula. The current fitted probability determines the weight.

Assumptions, Diagnostics, and Alternatives

The derivation relies on the conditional Bernoulli variance being \(\mu_i(1-\mu_i)\) and on the logit derivative being \(\mu_i(1-\mu_i)\). If the response is a count of successes from multiple trials rather than one Bernoulli outcome, the variance and weight must reflect the number of trials; applying the single-observation formula without adjustment would be inappropriate. Likewise, another link function changes the derivative and generally changes the weight formula.

A useful diagnostic is to inspect the fitted probabilities alongside the weights. Many probabilities extremely close to zero or one imply very small working weights, while probabilities near one-half imply weights close to their maximum of \(0.25\). Check that these patterns are plausible given the covariates and the model specification. Also check for numerically rounded probabilities of exactly zero or one: for finite predictors the logistic formula gives probabilities strictly between the boundaries, so exact boundary values often reflect numerical overflow or underflow. Such values make the working variance or weight calculation unstable and should not be treated as ordinary finite-predictor cases.

The weight formula is one ingredient in PQL’s local Gaussian approximation. It does not by itself establish that the approximation is adequate, nor does a small weight prove that an observation is erroneous. PQL uses a linearization, whereas likelihood-based approaches such as Laplace approximation or adaptive quadrature approximate the random-effects integration in the likelihood rather than relying on this same working-response variance construction. Ordinary logistic-regression IRLS uses the same Bernoulli logit weight form when there are no random effects, but it does not account for within-cluster dependence through random effects. These approaches address different modeling and approximation choices; the weight alone is not a rule for choosing among them.

Common Mistakes

Calling \(w_i\) the event probability. The event probability is \(\mu_i^{(0)}\); the weight is \(\mu_i^{(0)}(1-\mu_i^{(0)})\).
Using the raw-response variance as the working weight. The raw Bernoulli variance is \(\mu_i^{(0)}(1-\mu_i^{(0)})\), while the working-response variance is its reciprocal. The weight is the reciprocal of the latter.
Thinking extreme probabilities receive high weight because they seem decisive. The weight tends to zero near zero and one because the local predictor-scale working response has high approximate variance there.
Letting the observed outcome determine the weight. At a fixed linearization point, the weight depends on the fitted mean, not on whether the observed outcome is zero or one.

Key takeaway: For a Bernoulli response with a logit link and unit dispersion, the PQL observation-level weight at a current fitted probability \(\mu_i^{(0)}\) is \(w_i=\mu_i^{(0)}(1-\mu_i^{(0)})\). It is at most \(0.25\), is largest at probability one-half, and approaches zero near either probability boundary.

Check Your Understanding

Answer each question, then compare with the solutions below.

For a hypothetical observation with current predictor \(\eta_i^{(0)}=0\), calculate the fitted probability and working weight.
Why can two observations with different binary outcomes have equal working weights?
Which has the larger weight, a fitted probability of \(0.10\) or \(0.90\)? Calculate both.
State the conditional variance assumption used to derive the Bernoulli logit working weight.

Solutions. (1) The logistic probability is \(1/(1+1)=0.5000\), so \(w_i=0.5000(0.5000)=0.2500\). (2) At a fixed linearization point, \(w_i\) depends only on the fitted probability; if the probabilities match, the weights match even when the outcomes differ. (3) They are equal by symmetry: \(0.10(0.90)=0.09000\) and \(0.90(0.10)=0.09000\). (4) Conditional on the random effects and covariates, the response is Bernoulli with variance \(\mu_i(1-\mu_i)\); the derivation also uses the logit link and unit dispersion.

What You'll Learn