Deriving Weighted GLM Score Equations

Why Derive a Weighted Score Equation?

A generalized linear model (GLM) uses a specified response distribution, a linear predictor, and a link function to relate the mean response to predictors. When observations have different weights, the fitted coefficients are determined by a weighted objective: a case with a larger weight contributes more to the estimation equations. Deriving those equations makes the effect precise and helps distinguish weighting from changing the model itself. The material is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

This tutorial treats weights as fixed, nonnegative multipliers on observation-level log-likelihood contributions. A weight of zero removes that case from the objective; a larger positive weight gives its contribution more influence. The equations below do not, by themselves, decide why a particular weighting scheme is appropriate. That interpretation depends on the data and the modeling purpose.

Deriving the General Weighted Score

Suppose each response follows an exponential-family model with log-likelihood contribution, apart from terms not involving the model parameters, \(\ell_i=(y_i\theta_i-b(\theta_i))/\phi\). Here \(\theta_i\) is the natural parameter, \(\phi\) is a common dispersion parameter, and the mean is \(\mu_i=b'(\theta_i)\). The variance has the form \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\). The GLM specifies \(\eta_i=x_i^\mathsf{T}\beta\) and \(g(\mu_i)=\eta_i\), where \(x_i\) is the predictor vector and \(g\) is the link.

Assume the responses are conditionally independent, the family and link are differentiable, and the means lie in the interior of the allowed parameter range. These conditions allow differentiation of the likelihood. The dispersion is treated as fixed for this coefficient-score derivation.

Combining the steps gives the general weighted score vector. The estimate is found by setting this vector equal to zero and solving for \(\beta\).

This formula also shows why a weight changes a case’s contribution to estimation: it multiplies that case’s score. It does not appear inside \(g\), nor does it change the relationship \(g(\mu_i)=\eta_i\). A noncanonical link generally leaves the factor \((d\mu_i/d\eta_i)/V(\mu_i)\) in the score. It is not generally correct to use the simpler canonical-link equation for every GLM.

Connection to Working Weights

The score can be written in the form used by iteratively reweighted least squares (IRLS). Define the working response \(z_i=\eta_i+(y_i-\mu_i)/(d\mu_i/d\eta_i)\), and define the working weight \(W_i=w_i(d\mu_i/d\eta_i)^2/(\phi V(\mu_i))\). Then \(W_i(z_i-\eta_i)\) equals the \(i\)th scalar residual contribution to the score before multiplication by \(x_i\). Thus the score equation is \(X^\mathsf{T}W(z-\eta)=\mathbf{0}\), where \(W\) is the diagonal matrix of working weights.

The working weights vary with the current fitted means for a general GLM. They combine the observation weight with the model’s variance and link behavior. They are not a replacement link function: the model still uses the stated \(g\). IRLS repeatedly updates the working response and working weights to find a solution to the original score equations.

Weighted Gaussian Regression

For a normal response with \(\mu_i=x_i^\mathsf{T}\beta\), identity link, and common variance \(\sigma^2\), the score is \(\sum_i w_ix_i(y_i-\mu_i)/\sigma^2=\mathbf{0}\). Since the nonzero common factor \(1/\sigma^2\) cancels, the coefficient estimate solves the weighted least-squares normal equations \(X^\mathsf{T}WX\beta=X^\mathsf{T}Wy\). This result relies on the identity link and the stated Gaussian variance structure.

Grouped Binomial Responses and Weights

For grouped binomial data, let \(r_i\) be the number of successes out of \(m_i\) trials, and let \(p_i\) be the success probability. Under a logit link, the canonical score for additional case weights \(w_i\) is \(\sum_iw_ix_i(r_i-m_ip_i)=\mathbf{0}\). The number of trials \(m_i\) is part of the binomial response likelihood; an additional case weight \(w_i\) multiplies that entire grouped likelihood contribution.

What the Weights Change—and What They Do Not

The weighted score makes the distinction direct. The link determines how a linear predictor maps to the mean; for example, a log link requires \(\mu_i=e^{\eta_i}\), while a logit link requires \(p_i=e^{\eta_i}/(1+e^{\eta_i})\). Multiplying a likelihood contribution by \(w_i\) does not change either mapping. It changes how strongly that observation’s residual pattern affects the coefficient estimate.

Weights also do not guarantee that an estimate is a solution in closed form. The Poisson binary-predictor example and the intercept-only binomial example admitted simple weighted means because their predictors allowed direct group or intercept solutions. More general GLMs usually require numerical optimization or IRLS. If weights are fractional or are not literal replication counts, a weighted likelihood can be a useful estimation objective, but it should not automatically be interpreted as an ordinary likelihood for an expanded sample.

A frequent exam trap is to conflate the weight with a predictor effect. A weight does not add a term to \(\eta_i\); it multiplies the observation’s contribution to the score. Another trap is to use the canonical simplification under a noncanonical link. In that case, return to the general score and retain the derivative and variance-function factors.