Purpose of a Hand-Worked IRLS Update
A generalized linear model is usually estimated by repeated updates rather than by solving a single linear equation for its coefficients. In a Poisson frequency model with a log link, each update turns the current nonlinear estimation problem into a weighted least-squares problem. Hand-calculating one iteration makes the roles of the fitted means, working responses, and weights explicit.
The goal here is one complete update, not a fully converged model. Each example states the current coefficient vector, computes every row’s intermediate quantities, and then solves the resulting normal equations. A single update need not be close to the final estimate; it is the prescribed next step from the stated starting values.
Quantities in One Poisson IRLS Step
Suppose the current coefficient estimate is \(\boldsymbol{\beta}^{(0)}\). For each observation, first calculate the current linear predictor and fitted mean. The working response is a temporary response on the linear-predictor scale, and the weight indicates how strongly that response contributes to the next weighted least-squares fit.
Here \(y_i\) is the observed count. The superscript \((0)\) labels quantities calculated from the current coefficients; it does not mean that the observed count is zero. The updated coefficients are obtained by regressing the working responses on the model’s predictor columns using weights \(w_i^{(0)}\):
This inverse form requires the weighted design matrix to have full column rank. Equivalently, the normal equations must have a unique solution. If the predictor columns are redundant, the displayed inverse does not exist and the model is not identifiable in that parameterization.
Why These Working Quantities Give a Fisher-Scoring Step
The justification follows directly from the Poisson log-likelihood. Ignoring terms that do not depend on \(\boldsymbol{\beta}\), the log-likelihood is \(\ell=\sum_i(y_i\eta_i-\mu_i)\). Since \(\partial\mu_i/\partial\eta_i=\mu_i\), its score vector is \(\mathbf{X}^{\mathsf{T}}(\mathbf{y}-\boldsymbol{\mu})\). The expected information matrix is \(\mathbf{X}^{\mathsf{T}}\mathbf{W}\mathbf{X}\), with diagonal entries of \(\mathbf{W}\) equal to \(\mu_i\). A Fisher-scoring update therefore adds the information-weighted score to the current coefficients.
To connect this update to weighted least squares, note that \(\mathbf{z}^{(0)}=\mathbf{X}\boldsymbol{\beta}^{(0)}+\mathbf{W}^{(0)-1}(\mathbf{y}-\boldsymbol{\mu}^{(0)})\). Multiplying the difference between the working response and the current fitted linear predictor by the weights gives the original score residuals. Thus the weighted normal equations for \(\mathbf{z}^{(0)}\) produce exactly the Fisher-scoring update:
This is the reason IRLS is called iteratively reweighted least squares: the working response and the weights are recalculated from the latest coefficients at each iteration. For this Poisson log-link model, the weight is the fitted mean, not its reciprocal.
Worked Updates
Worked Example: Starting From Equal Fitted Means
A hypothetical portfolio has three equal-exposure records. The predictor values are \(x=(0,1,2)\), the observed claim counts are \(y=(1,2,3)\), and the model is \(\log(\mu_i)=\beta_0+\beta_1x_i\). Starting from \(\boldsymbol{\beta}^{(0)}=(0,0)\), calculate one IRLS update. The counts are modeled as independent Poisson variables.
Method. Calculate the current means, working responses, and weights, then solve the weighted normal equations. With both initial coefficients zero, each current linear predictor is zero, so each fitted mean is \(\exp(0)=1\).
For \(x=0\), \(y=1\): \(z=0+(1-1)/1=0\), and \(w=1\). For \(x=1\), \(y=2\): \(z=0+(2-1)/1=1\), and \(w=1\). For \(x=2\), \(y=3\): \(z=0+(3-1)/1=2\), and \(w=1\). Thus \(\mathbf{z}=(0,1,2)^\mathsf{T}\) and \(\mathbf{W}\) is the \(3\) by \(3\) identity matrix.
The design matrix has rows \((1,0),(1,1),(1,2)\). Therefore,
The determinant is \(3(5)-3(3)=6\), so the matrix is invertible. Solving \(3\beta_0^{(1)}+3\beta_1^{(1)}=3\) and \(3\beta_0^{(1)}+5\beta_1^{(1)}=5\), subtraction gives \(2\beta_1^{(1)}=2\), hence \(\beta_1^{(1)}=1\) and \(\beta_0^{(1)}=0\). The updated vector is \((0,1)\). As a check, the updated fitted means at \(x=0,1,2\) are \(1,\exp(1),\exp(2)\), approximately \(1.0000,2.7183,7.3891\), respectively.
Common mistake avoided: The observed counts are not the next regression response. The response in this update is \((0,1,2)\), calculated from the current means. Regressing \((1,2,3)\) directly would skip the working-response construction.
Worked Example: An Intercept and a Binary Predictor
A hypothetical equal-exposure study has two records at each of \(x=0\) and \(x=1\). Their counts are \(1,3\) for \(x=0\), and \(2,6\) for \(x=1\). Fit the model \(\log(\mu_i)=\beta_0+\beta_1x_i\) for one IRLS step from \(\boldsymbol{\beta}^{(0)}=(\log 2,0)\). Assume independent Poisson counts.
Method. At the starting values, every record has \(\eta^{(0)}=\log 2\) and \(\mu^{(0)}=2\); all four weights are therefore \(2\). For the two \(x=0\) records, the working responses are \(\log 2+(1-2)/2=\log 2-0.5\) and \(\log 2+(3-2)/2=\log 2+0.5\). Their mean is \(\log 2\).
For the \(x=1\) records, the working responses are \(\log 2+(2-2)/2=\log 2\) and \(\log 2+(6-2)/2=\log 2+2\). Their mean is \(\log 2+1\). Because each group has equal weights, the weighted least-squares fitted working response in each group is its arithmetic mean. With an intercept and a binary predictor, the fitted value for \(x=0\) is \(\beta_0^{(1)}\), while the fitted value for \(x=1\) is \(\beta_0^{(1)}+\beta_1^{(1)}\). Consequently,
The update is \((\log 2,1)\), approximately \((0.6931,1.0000)\). The resulting means are \(2\) for \(x=0\) and \(2\exp(1)\approx5.4366\) for \(x=1\). As a check on the weighted fit, the average observed count in the first group is \(2\), and in the second it is \(4\); the updated means are not required to equal those observed averages after just one step.
Common mistake avoided: Each observation has weight \(2\), not weight \(1/2\). The two members of a group have equal weights, so their average working response is unchanged by multiplying both weights by \(2\).
Worked Example: Unequal Weights and a Nonzero Slope
A hypothetical portfolio has three equal-exposure records with \(x=(0,1,2)\) and counts \(y=(2,1,5)\). Use the model \(\log(\mu_i)=\beta_0+\beta_1x_i\), starting from \(\boldsymbol{\beta}^{(0)}=(0,\log 2)\). Calculate one update. Assume independent Poisson counts.
Method. The current linear predictors are \(0,\log 2,\log 4\), giving fitted means \(1,2,4\) and weights \(1,2,4\). The working responses are \(0+(2-1)/1=1\), \(\log 2+(1-2)/2=\log 2-0.5\), and \(\log 4+(5-4)/4=\log 4+0.25\). Numerically, these are \(1.0000,0.1931,1.6363\).
The weighted normal-equation matrix and right-hand side can be assembled from the rows \((1,x_i)\):
The weighted response sums are \(S_0=\sum_iw_iz_i=1+2(\log2-0.5)+4(\log4+0.25)=1+10\log2\), and \(S_1=\sum_iw_ix_iz_i=2(\log2-0.5)+8(\log4+0.25)=1+18\log2\). The normal equations are \(7\beta_0+10\beta_1=S_0\) and \(10\beta_0+18\beta_1=S_1\). Their determinant is \(7(18)-10(10)=26\), so
The updated coefficients are approximately \((0.3077,0.5778)\). The computed normal-equation determinant is positive, confirming that this two-coefficient update has a unique solution. The calculation also checks by substitution: \(7(4/13)+10(\log2-3/26)=1+10\log2=S_0\), and \(10(4/13)+18(\log2-3/26)=1+18\log2=S_1\).
Common mistake avoided: The rows must not be treated as equally weighted. In particular, the \(x=2\) row has weight \(4\), so it contributes \(4\) to the intercept-intercept entry and \(16\) to the slope-slope entry of the matrix.
Exam Checks and Practical Interpretation
A reliable hand calculation keeps the update in a fixed order. First calculate \(\eta_i^{(0)}\), then exponentiate to get \(\mu_i^{(0)}\), and only then calculate \(z_i^{(0)}\) and \(w_i^{(0)}\). After building the normal equations, solve for the new coefficients and verify them by substitution. This order helps prevent mixing quantities from the current step with quantities from the updated model.
Confirm that the response is a count, the distribution is Poisson, the link is log, and the stated starting coefficients are used.
Find \(\eta_i^{(0)}\), \(\mu_i^{(0)}=\exp(\eta_i^{(0)})\), \(z_i^{(0)}\), and \(w_i^{(0)}=\mu_i^{(0)}\).
Form \(\mathbf{X}^{\mathsf{T}}\mathbf{W}^{(0)}\mathbf{X}\) and \(\mathbf{X}^{\mathsf{T}}\mathbf{W}^{(0)}\mathbf{z}^{(0)}\), solve for the updated vector, and substitute it back into the normal equations.
A common exam trap is to use the observed counts directly as the working responses or to use reciprocal fitted means as the weights. Both choices change the normal equations and therefore do not calculate the Poisson log-link IRLS step. Another important check is the requested task: one iteration requires using the supplied starting coefficients once; it does not mean repeating the process until the coefficients stop changing.
Check Your Understanding
For each problem, calculate only the requested single IRLS update unless stated otherwise. Assume independent Poisson counts and the log link.
- A model has an intercept only, counts \(1,3,4\), and starting coefficient \(\beta_0^{(0)}=\log2\). Find the updated intercept. (Answer: \(\log2+\frac13\approx1.0265\))
- Two equal-exposure records have \(x=0,1\), counts \(1,3\), and starting coefficients \((0,0)\) for \(\log(\mu)=\beta_0+\beta_1x\). Find the updated coefficients. (Answer: \((0,2)\))
- For \(x=0,1,2\), counts \(2,2,4\), and starting coefficients \((0,0)\), find the updated coefficients in the intercept-and-slope model. (Answer: \((\frac23,1)\))
- Two records have \(x=0,1\), counts \(0,4\), and starting coefficients \((\log2,0)\). Find the updated coefficients. (Answer: \((\log2-1,2)\approx(-0.3069,2.0000)\))
- For \(x=0,1,2\), counts \(1,2,4\), and starting coefficients \((0,0)\), find the updated coefficients. (Answer: \((-\frac16,\frac32)\approx(-0.1667,1.5000)\))