Why GLM Fitting Uses Working Quantities
A generalized linear model (GLM) relates a predictor \(\eta_i\) to the mean response \(\mu_i\) through a link function. For a logistic model, the mean is a probability; for a Poisson model, it is an expected event count. In either case, fitting the model requires solving equations that are generally not linear in the coefficients. Iteratively reweighted least squares (IRLS) makes each fitting step a weighted least-squares problem by constructing a working response and an observation weight from the current coefficient estimate.
These are temporary quantities used in an iteration, not transformed data that replace the original observations. After a coefficient update, the fitted means change, so the working responses and weights are recalculated. This tutorial focuses on calculating and interpreting those quantities for logistic and Poisson GLMs, aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The formula uses the coefficient estimate at the current iteration. For the usual Bernoulli and Poisson models, \(\phi=1\). In grouped binomial data represented as a proportion from \(m_i\) trials, the number of trials supplies the prior weight \(a_i=m_i\). A Poisson exposure offset, in contrast, enters the linear predictor; it is not automatically a prior weight.
Logistic Working Responses and Weights
For a logistic GLM, \(g(\mu)=\log\{\mu/(1-\mu)\}\), so \(g'(\mu)=1/\{\mu(1-\mu)\}\). For a Bernoulli observation, the variance function is \(V(\mu)=\mu(1-\mu)\). Substitution gives
When observation \(i\) is a proportion \(y_i/m_i\) from \(m_i\) binomial trials, use that proportion as the response and \(a_i=m_i\). Its working weight is \(m_i\mu_i(1-\mu_i)\). Its working response has the same form, with \(y_i\) in the numerator interpreted as the observed proportion.
Worked Example: One Logistic Update for Bernoulli Observations
A hypothetical insurer models whether a submitted application requires manual review. Three applications have responses \(y=(1,0,1)\). Use an intercept-only logistic model, start at the coefficient estimate \(\beta^{(0)}=0\), and calculate the working quantities and the next coefficient estimate. State the method and check the update.
Method. Calculate each fitted probability from the current intercept, construct \(z_i\) and \(w_i\), then fit an intercept to the working responses by weighted least squares. The observations are Bernoulli, so their prior weights are 1.
The current predictor is \(\eta_i=0\) for every application. Therefore, \(\mu_i=1/(1+\exp(0))=0.5\), and \(\mu_i(1-\mu_i)=0.5(0.5)=0.25\). Each weight is \(0.25\). For an observed review indicator of 1, \(z_i=0+(1-0.5)/0.25=2\). For an indicator of 0, \(z_i=0+(0-0.5)/0.25=-2\). Thus the working responses are \(2,-2,2\).
In an intercept-only weighted least-squares fit, the updated intercept is the weighted mean of the working responses: \[ \beta^{(1)}=\frac{0.25(2)+0.25(-2)+0.25(2)}{0.25+0.25+0.25} =\frac{0.5}{0.75}=\frac{2}{3}=0.6667. \] The displayed decimal is rounded to four places. As a check, the initial logistic score is \(1-0.5+0-0.5+1-0.5=0.5\), and its Fisher information is \(3(0.25)=0.75\). The score divided by the information is \(0.5/0.75=2/3\), agreeing with the working least-squares update. The common mistake avoided here is treating \(z_i\) as a new observed outcome: it is a temporary response constructed from the current probability.
Worked Example: Grouped Binomial Working Quantities
A hypothetical screening program records positive results in two risk groups. The groups have \(m_1=m_2=10\) trials, observed positive proportions \(0.3\) and \(0.7\), and predictor values \(x_1=0\) and \(x_2=1\). At the current estimates \(\beta_0=-\log(3)\) and \(\beta_1=\log(3)\), calculate the working responses and weights, then carry out one weighted least-squares update. Round the updated coefficients to four decimal places.
Method. Use \(\eta_i=\beta_0+\beta_1x_i\), and use \(m_i=10\) as the prior weight because the responses are proportions from ten trials. The first predictor is \(-\log(3)\), giving \(\mu_1=0.25\); the second is 0, giving \(\mu_2=0.5\).
For the first group, the logistic variance is \(0.25(0.75)=0.1875\). Its weight is \(w_1=10(0.1875)=1.875\), and its working response is \[ z_1=-\log(3)+\frac{0.3-0.25}{0.1875} =-1.0986123+0.2666667=-0.8319456. \] For the second group, the variance is \(0.5(0.5)=0.25\). Its weight is \(w_2=10(0.25)=2.5\), and its working response is \[ z_2=0+\frac{0.7-0.5}{0.25}=0.8. \]
With two observations and two coefficients, the weighted least-squares fit to the two working responses gives fitted working values \(z_1\) at \(x=0\) and \(z_2\) at \(x=1\). Hence \(\beta_0^{(1)}=z_1=-0.8319456\) and \(\beta_1^{(1)}=z_2-z_1=1.6319456\). Rounded to four decimal places, the updated coefficients are \(-0.8319\) and \(1.6319\). The check is that \(\beta_0^{(1)}+\beta_1^{(1)}=0.8000=z_2\). The common mistake avoided is using weight \(0.1875\) for the first group: that is the variance for one Bernoulli trial, while this grouped proportion has ten trials and weight \(1.875\).
Poisson Working Responses and Weights
For a Poisson GLM with a log link, \(g(\mu)=\log(\mu)\), so \(g'(\mu)=1/\mu\). The Poisson variance function is \(V(\mu)=\mu\). For an observation with prior weight 1, the working quantities therefore simplify to
If a model predicts counts proportional to exposure \(e_i>0\), a common specification is \(\eta_i=\log(e_i)+x_i^\mathsf{T}\beta\). The term \(\log(e_i)\) is an offset, so \(\mu_i=e_i\exp(x_i^\mathsf{T}\beta)\). Calculate \(z_i\) using the full current predictor, including the offset. When solving for coefficients, fit the adjusted working response \(z_i-\log(e_i)\) against the coefficient predictors.
Worked Example: Poisson Update with an Exposure Offset
A hypothetical service team records counts of equipment alarms over two monitoring periods. The periods have exposures \(e_1=1\) and \(e_2=3\) exposure units and observed counts \(y_1=3\) and \(y_2=3\). Fit an intercept-only log-rate model with offset \(\log(e_i)\), starting at \(\beta^{(0)}=\log(2)\). Calculate both sets of working quantities and the updated intercept. Round the final intercept and rate to four decimal places.
Method. The current predictor is \(\eta_i=\log(e_i)+\beta^{(0)}\), so the fitted means are \(\mu_1=1(2)=2\) and \(\mu_2=3(2)=6\). For Poisson observations, each weight equals its fitted mean: \(w_1=2\) and \(w_2=6\).
For the first period, \[ z_1=\log(2)+\frac{3-2}{2}=\log(2)+0.5=1.1931472. \] For the second period, \[ z_2=\log(6)+\frac{3-6}{6}=\log(6)-0.5=1.2917595. \] The offsets are \(0\) and \(\log(3)=1.0986123\), respectively. The adjusted working responses for the intercept are therefore \(1.1931472\) and \(1.2917595-1.0986123=0.1931472\).
The weighted least-squares intercept is \[ \beta^{(1)} =\frac{2(1.1931472)+6(0.1931472)}{2+6} =\frac{2.3862944+1.1588832}{8} =0.4431472. \] Thus the updated intercept is \(0.4431\), and its rate is \(\exp(0.4431472)=1.5576\), rounded to four decimal places. As a check on the update calculation, the Poisson score for the intercept is \((3-2)+(3-6)=-2\), and the Fisher information is \(2+6=8\). The coefficient increment is \(-2/8=-0.25\), so the updated intercept is \(\log(2)-0.25=0.4431472\), as above. The common mistake avoided is subtracting the exposure offset from the observed count; the offset is part of the predictor, not an adjustment to \(y_i\).
Why the Working Least-Squares Step Holds
The working response is chosen so that a local change in the predictor corresponds to a local change in the mean. This gives a weighted least-squares update for the coefficients rather than a direct least-squares fit to the original outcomes.
At the current predictor, a small change \(\Delta\eta_i\) changes the mean by approximately \(\Delta\mu_i=(d\mu_i/d\eta_i)\Delta\eta_i=\Delta\eta_i/g'(\mu_i)\).
The linearized predictor needed to move from \(\mu_i\) toward \(y_i\) is \(\eta_i+(y_i-\mu_i)g'(\mu_i)\), which is \(z_i\).
The mean has variance \(\phi V(\mu_i)/a_i\). The corresponding local predictor variance is approximately \(\phi V(\mu_i)\{g'(\mu_i)\}^2/a_i\); its reciprocal is \(w_i\).
Fit the linear predictor to \(z_i\) by weighted least squares, using \(z_i-o_i\) as the target if the model includes a fixed offset \(o_i\).
More formally, for design matrix \(X\), coefficient vector \(\beta\), and fixed offset vector \(o\), the IRLS update solves
where \(W\) is the diagonal matrix of current working weights. This follows from the weighted least-squares normal equations for fitting \(z-o\) using \(X\). In practice, a fitting algorithm may solve this system without explicitly forming a matrix inverse. The result is one update based on a local approximation; the process recalculates \(\eta_i,\mu_i,z_i,\) and \(w_i\) before the next step.
Interpreting the Weights and Avoiding Traps
A working weight describes the information an observation contributes to the local coefficient update. In a logistic model, \(\mu_i(1-\mu_i)\) is largest at \(\mu_i=0.5\) and becomes small as the probability approaches 0 or 1. In a Poisson model, the weight is the fitted mean, so a larger fitted count has a larger working weight, all else equal. These are iteration-specific weights, not necessarily the weights that should be used for every other statistical task.
- Use the current fitted mean. Compute \(\mu_i\) from the current coefficient estimate before calculating either \(z_i\) or \(w_i\).
- Respect the response scale. For grouped binomial data, distinguish a success count from a success proportion. If using a proportion, include the number of trials as the prior weight.
- Include the offset in the predictor. Compute the working response using the full \(\eta_i\), then account for the offset in the weighted regression target.
- Do not treat \(z_i\) as the original response. It is constructed for one local update and generally changes at the next iteration.
- Do not infer model quality from one update. Working quantities explain how coefficients are updated; they do not, by themselves, assess overall fit.
Check Your Understanding
For each problem, calculate the requested working quantities at the stated current estimate. Round probabilities and final decimal values to four decimal places where needed.
- A Bernoulli logistic observation has \(y=1\) and current predictor \(\eta=0.4\). Find \(\mu,z,\) and \(w\). (Answer: \(\mu=0.5987,\ z=2.0703,\ w=0.2403\))
- A Poisson observation has \(y=2\) and current predictor \(\eta=\log(4)\). Find its working response and weight. (Answer: \(z=0.8863,\ w=4\))
- A grouped binomial observation has 8 trials, observed success proportion \(0.375\), and current fitted probability \(0.25\). Find its working response and weight. (Answer: \(z=-0.4319,\ w=1.5\))
- A Poisson count has exposure \(e=2\), observed count \(y=5\), and current log-rate intercept \(\beta=\log(1.5)\). Find the full predictor, working response, weight, and offset-adjusted working response for the intercept. (Answer: \(\eta=\log(3)=1.0986,\ z=1.7653,\ w=3,\ z-\log(2)=1.0721\))
- An intercept-only logistic model has two Bernoulli observations \(y_1=1,y_2=0\), both with current predictor \(0\). Find the working responses, weights, and one updated intercept. (Answer: \(z_1=2,\ z_2=-2,\ w_1=w_2=0.25,\ \beta^{\text{new}}=0\))