Weighted Likelihood in Generalized Linear Models

Why Weight Likelihood Contributions?

In a generalized linear model (GLM), each observation contributes information about the model parameters. An observation-level weight changes how strongly that observation contributes to a specified fitting criterion. This can be useful when the analysis calls for some observations to count more than others, but the reason for the weights must be clear: a weight is not automatically an exposure, a number of trials, or a change to the response distribution.

The central distinction is between weighting a likelihood contribution and changing the probability model for the response. The first modifies the objective used for fitting. The second changes assumptions about how the response is distributed. They can produce related-looking calculations, but they are not interchangeable.

Definition: Suppose observations $Y_1,\ldots,Y_n$ are conditionally independent given their predictors, and a specified GLM gives observation $i$ a probability or density contribution $f_i(y_i;\beta)$. Its log-likelihood contribution is $\ell_i(\beta)=\log f_i(y_i;\beta)$. For fixed nonnegative observation-level weights $w_i$, the weighted log-likelihood criterion is $$ \ell_w(\beta)=\sum_{i=1}^{n} w_i\ell_i(\beta) =\sum_{i=1}^{n}w_i\log f_i(y_i;\beta). $$

The response distribution $f_i$ and its parameters are specified before applying this criterion. A positive weight increases or decreases the contribution of an observation relative to other observations. A zero weight removes that observation’s contribution. Unless a separate probability model is specified, weighting does not change the family, link, conditional mean, or conditional variance assumed for $Y_i$.

Weights may have different meanings in different analyses. The formula above defines how they enter the criterion; it does not establish why a particular set of weights is appropriate. For example, a fixed weight used to emphasize an observation is not automatically an exposure offset or a count of independent records. The intended interpretation must be supplied by the analysis.

Why the Contributions Add

Under conditional independence, the ordinary likelihood is the product of the observation-level probability or density contributions. Taking a logarithm turns that product into a sum. The weighted criterion applies a weight to each term in that sum.

L(\beta)=\prod_{i=1}^{n} f_i(y_i;\beta) \quad\Longrightarrow\quad \log L(\beta)=\sum_{i=1}^{n}\log f_i(y_i;\beta) \quad\Longrightarrow\quad \ell_w(\beta)=\sum_{i=1}^{n}w_i\log f_i(y_i;\beta).

The first implication follows from the logarithm rule $\log(ab)=\log(a)+\log(b)$. The final expression is the weighted criterion by definition. If every $w_i=1$, it is the ordinary log-likelihood. If the weights are not all one, the criterion can be regarded as a weighted or power-likelihood objective. In general, it need not be the ordinary log-likelihood of the original independent observations under the original response model.

A useful consequence is that multiplying every weight by the same positive constant $c$ multiplies the entire criterion by $c$:

\ell_{cw}(\beta)=\sum_i cw_i\ell_i(\beta)=c\ell_w(\beta).

Because $c>0$, this common rescaling preserves which parameter value maximizes the criterion. It does not preserve the numerical value of the criterion. Therefore, reported likelihood values or likelihood-based comparisons can depend on the weight scale, even when the maximizing coefficient estimates do not.

Specify the response model.
State the response family and its parameters, including how the GLM predictors determine the conditional mean or other model parameters.

Find each log contribution.
For every observation, calculate $\ell_i(\beta)=\log f_i(y_i;\beta)$ under that specified model.

Apply the stated weights.
Multiply each observation’s log contribution by its own $w_i$, then sum the weighted terms.

Interpret the result cautiously.
Describe it as a weighted criterion unless the weight construction also justifies a particular sampling or probability model.

Worked Likelihood Calculations

The following examples use specified candidate model values so that the weighted contributions can be checked directly. The weights are part of each hypothetical analysis; they do not, by themselves, change the response family.

Worked Example: Comparing Poisson Candidate Means

A hypothetical portfolio has three claim-count observations $y=(0,1,2)$ with fixed weights $w=(1,2,1)$. Assume the observations are conditionally independent Poisson variables with the stated candidate means. Compare candidate A, with means $(0.5,1,2)$, and candidate B, with means $(1,1,1)$, using the weighted log-likelihood. Which candidate has the larger value?

Method. For $Y_i\sim\operatorname{Poisson}(\mu_i)$, the probability mass function is $f_i(y_i;\mu_i)=e^{-\mu_i}\mu_i^{y_i}/y_i!$. Therefore,

\ell_i=y_i\log(\mu_i)-\mu_i-\log(y_i!).

For candidate A, observation 1 has $y_1=0,\mu_1=0.5,w_1=1$, so its weighted contribution is $1(0\log(0.5)-0.5-\log(0!))=-0.5$. Observation 2 has $y_2=1,\mu_2=1,w_2=2$, giving $2(1\log(1)-1-\log(1!))=2(0-1-0)=-2$. Observation 3 has $y_3=2,\mu_3=2,w_3=1$, giving $1(2\log(2)-2-\log(2!))=2\log(2)-2-\log(2)=\log(2)-2$, approximately $-1.3069$.

Thus candidate A has $\ell_w(A)=-0.5-2-1.3069=-3.8069$. For candidate B, the first contribution is $1(0\log(1)-1-0)=-1$; the second is $2(1\log(1)-1-0)=-2$; and the third is $1(2\log(1)-1-\log(2))=-1-\log(2)$, approximately $-1.6931$. Therefore $\ell_w(B)=-1-2-1.6931=-4.6931$.

Candidate A has the larger weighted log-likelihood, since $-3.8069>-4.6931$. The calculation applies each weight to the complete log contribution, including its $-\mu_i$ and factorial terms. A common mistake is to weight only $y_i\log(\mu_i)$; that does not calculate the defined weighted log-likelihood.

Worked Example: Weighted Bernoulli Contributions

In a hypothetical screening study, three outcomes are $y=(1,0,1)$. Assume independent Bernoulli responses, with $P(Y_i=1)=p_i$. The fixed weights are $(2,1,0.5)$. Candidate A predicts probabilities $(0.8,0.3,0.6)$; candidate B predicts $(0.7,0.2,0.7)$. Calculate each weighted log-likelihood and select the larger.

For a Bernoulli response, $f_i(y_i;p_i)=p_i^{y_i}(1-p_i)^{1-y_i}$. Thus a success contributes $\log(p_i)$, a failure contributes $\log(1-p_i)$, and the weighted contribution is $w_i$ times the applicable logarithm.

For candidate A, the three weighted contributions are $2\log(0.8)=-0.4463$, $1\log(1-0.3)=\log(0.7)=-0.3567$, and $0.5\log(0.6)=-0.2554$. Adding them gives $\ell_w(A)=-1.0584$.

For candidate B, the contributions are $2\log(0.7)=-0.7133$, $1\log(1-0.2)=\log(0.8)=-0.2231$, and $0.5\log(0.7)=-0.1783$. Hence $\ell_w(B)=-1.1148$. Candidate A has the larger value because $-1.0584>-1.1148$.

The weight of 2 makes the first outcome contribute twice its Bernoulli log probability; it does not, without an additional model, turn that observation into two Bernoulli trials. Treating the weight as a trial count would impose a different interpretation and requires its own justification.

Worked Example: Weighted Normal Log-Likelihood

A hypothetical severity model has responses $y=(10,14,13)$, fixed weights $w=(1,3,2)$, and known conditional standard deviation $\sigma=2$. Assume independent Normal responses with means equal to the predictions. Candidate A predicts $(11,13,12)$, and candidate B predicts $(10,12,15)$. Compare their weighted log-likelihoods.

Under $Y_i\sim N(\mu_i,\sigma^2)$ with $\sigma^2=4$, the log-density is $-\frac12\log(2\pi\cdot4)-(y_i-\mu_i)^2/8$. The weighted squared residual total for candidate A is $1(10-11)^2+3(14-13)^2+2(13-12)^2=1+3+2=6$. For candidate B it is $1(10-10)^2+3(14-12)^2+2(13-15)^2=0+12+8=20$. The sum of the weights is $1+3+2=6$, so both candidates have the same weighted total for the constant log-density terms.

Consequently, candidate A has weighted log-likelihood $-3\log(8\pi)-6/8$, approximately $-9.6725-0.7500=-10.4225$. Candidate B has $-3\log(8\pi)-20/8$, approximately $-9.6725-2.5000=-12.1725$. Candidate A has the larger value.

This calculation uses the same Normal variance of 4 for every observation and weights each complete log-density. It is not the same as specifying a different variance for each response. For example, the separate model $Y_i\sim N(\mu_i,4/w_i)$ changes the response distribution and its density normalization; it is not implied merely by multiplying a log-density by $w_i$.

Weights Do Not Automatically Change the Response Model

A GLM response distribution specifies the possible response values and their probabilities or densities, conditional on predictors and model parameters. The weighted criterion above changes the relative influence of the specified observation-level contributions. It does not automatically replace one distribution with another or alter the distribution’s parameters.

For instance, if a Poisson model specifies $Y_i\sim\operatorname{Poisson}(\mu_i)$, the response remains Poisson with mean and variance $\mu_i$ under the weighted criterion. A weight on its log-likelihood contribution does not itself change that variance to $\mu_i/w_i$. Likewise, a weight attached to a Bernoulli log contribution does not by itself change the response from a single Bernoulli outcome to a binomial count.

This distinction matters when explaining estimates and uncertainty. The weighted objective determines which observations have more influence on the fitted criterion. Any claims about a sampling distribution, variance, or standard error must also be supported by an appropriate model for how the data and weights arose; they do not follow solely from the algebraic act of weighting.

Common Mistake: Weighting Only Part of the Log-Likelihood

A frequent error is to apply the weight to the response-dependent term while leaving the rest of the log-likelihood unweighted. In the Poisson example, the full contribution is $y_i\log(\mu_i)-\mu_i-\log(y_i!)$. The weighted contribution is $w_i[y_i\log(\mu_i)-\mu_i-\log(y_i!)]$, not $w_i y_i\log(\mu_i)-\mu_i-\log(y_i!)$. The latter uses a different criterion.

Another trap is to treat a weight as an exposure multiplier without checking the model. An exposure rate multiplier enters a rate model through the mean structure, whereas an observation-level weight multiplies a likelihood contribution. They serve different roles. Do not substitute one for the other merely because both appear in a calculation involving observations of unequal importance or size.

Key takeaway: First write the response model and its observation-level log contribution. Then multiply the entire contribution by the stated weight and sum. Interpret the result as a weighted criterion unless the weight construction supports a more specific probability-model interpretation.

Check Your Understanding

For each problem, write the response model or criterion before calculating. Probabilities and log-likelihood values are rounded to four decimal places where needed.

A Poisson observation has $y=2$, candidate mean $\mu=1.5$, and weight $w=2$. Calculate its weighted log-likelihood contribution. (Answer: $2[2\log(1.5)-1.5-\log(2!)]=-2.7644$.)
A Bernoulli observation has $y=0$, predicted probability $p=0.25$, and weight $w=3$. Calculate its weighted log-likelihood contribution. (Answer: $3\log(0.75)=-0.8630$.)
For three Normal observations, residuals are $2,-1,3$ and weights are $1,2,0.5$. Calculate the weighted sum of squared residuals. (Answer: $1(2^2)+2((-1)^2)+0.5(3^2)=10.5$.)
Two candidate models have weighted log-likelihoods $-8$ and $-9$. All weights are then multiplied by 4. Give the new criterion values and state whether the preferred candidate changes. (Answer: $-32$ and $-36$; the first candidate remains preferred because $-32>-36$.)
Suppose $Y\sim N(\mu,4)$ and an observation has weight $w=2$. Write its weighted log-density contribution. Compare this with the ordinary log-density under a different model $Y\sim N(\mu,2)$. (Answer: The weighted contribution is $2[-\frac12\log(8\pi)-(y-\mu)^2/8]=-\log(8\pi)-(y-\mu)^2/4$. Under $N(\mu,2)$, the log-density is $-\frac12\log(4\pi)-(y-\mu)^2/4$; the differing constant shows these are not the same model or criterion.)

What You'll Learn