Likelihood Deviance With Offsets and Weights

Why Weights and Offsets Matter for Deviance

A deviance compares the likelihood of a fitted model with the likelihood of a saturated model, one that can fit each observation separately. In a generalized linear model, a fixed offset contributes to each fitted linear predictor, while weights determine how strongly each observation contributes to the likelihood. Both details matter when calculating the comparison.

The key distinction is that the offset changes the fitted mean, whereas the saturated model can usually absorb the same fixed offset into its unrestricted predictor. Weights, by contrast, scale the likelihood contribution under the weight convention being used. A frequency weight, a precision weight, and a binomial trial count can therefore lead to different likelihood interpretations even when they appear in similar places in software output.

Likelihood Contributions and Fixed Offsets

For an exponential-dispersion family, write the parameter-dependent part of an observation’s log-likelihood as \(\{y_i\theta_i-b(\theta_i)\}/a(\phi)\), where \(\theta_i\) is the canonical parameter, \(b\) is the cumulant function, and \(\phi\) is the dispersion parameter. The mean is \(\mu_i=b'(\theta_i)\). Terms that do not depend on the model mean are omitted here. The unit deviance is twice the loss in this numerator relative to the saturated fit.

When the observed value is on the boundary of the mean parameter space, such as zero successes in a binomial observation, the saturated expression is interpreted by its limit. The familiar convention \(0\log 0=0\) gives the same finite deviance formulas.

An offset \(o_i\) is a known, fixed quantity included in the linear predictor, for example \(\eta_i=x_i^\mathsf{T}\beta+o_i\). The fitted mean is determined by the inverse link applied to this predictor. In a saturated model, each observation has an unrestricted predictor \(\eta_i^{\mathrm{sat}}\). For any admissible saturated mean, that predictor can be written as a free component plus the same offset \(o_i\). Thus, for a fixed offset and an unrestricted saturated model, the offset does not prevent the saturated fit from attaining its usual observation-by-observation maximum. It still changes the fitted-model mean, and therefore changes the deviance indirectly.

How Weight Conventions Enter

Suppose a row has a nonnegative weight \(w_i\). When its meaning is frequency replication, \(w_i\) is an integer number of identical records. Its log-likelihood contribution is repeated \(w_i\) times. When a valid precision-weight likelihood is specified, the likelihood contribution is also weighted, but the underlying sampling model gives \(w_i\) a precision interpretation. In either case, if the log-likelihood has the weighted exponential-family form below, the unscaled deviance contribution is \(w_i d(y_i,\hat{\mu}_i)\).

The likelihood-ratio statistic is \(2(\ell_{\mathrm{sat}}-\ell_{\mathrm{fit}})=D_w/a(\phi)\) when the same known dispersion parameter applies to both likelihoods. For Poisson and binomial models, \(a(\phi)=1\). For a Gaussian model with variance \(\phi/w_i\), the unscaled deviance is weighted squared error, and the likelihood-ratio statistic divides that deviance by \(\phi\).

A weight can multiply a log-likelihood algebraically without defining a valid sampling distribution. In particular, software may use prior or analytic weights in a quasi-likelihood or estimating-equation calculation. If there is no specified likelihood, the resulting weighted deviance is not automatically a likelihood-ratio statistic. Its formula and inferential interpretation depend on the convention and model actually used.

Worked Deviance Calculations

Comparing Two Fitted Models

Deviance also compares nested fitted models. Let model 0 be a submodel of model 1, fitted to the same data with the same likelihood, weights, and fixed offsets. Their saturated log-likelihood is the same. Write \(D_0=2(\ell_{\mathrm{sat}}-\ell_0)\) and \(D_1=2(\ell_{\mathrm{sat}}-\ell_1)\), where each log-likelihood is maximized within its model. Subtracting gives

Thus, the reduction in deviance is exactly the likelihood-ratio statistic for the nested comparison when the likelihood is genuine. With known dispersion, the exponential-family unscaled deviance reduction is divided by \(a(\phi)\) to obtain that statistic. Under the usual regularity conditions for likelihood-ratio inference—including an identifiable, regular model, an interior true parameter, and a fixed-dimensional difference in model dimension—the statistic is asymptotically chi-squared with degrees of freedom equal to that difference. If dispersion is estimated or the weights define only a quasi-likelihood, do not assume this simple scaling or reference distribution without additional justification.

Common Mistake: Treating Every Number Called a Weight Alike

A frequent error is to see a number \(w_i\) beside a row and automatically multiply that row’s deviance by it, then interpret the result as a likelihood-ratio statistic. This overlooks whether the weight describes replication, precision, or something else. For a genuine frequency weight, multiplying the row log-likelihood by an integer replication count is justified. For a precision weight, the likelihood must match the specified sampling variance or other precision model. For a binomial observation, the trial count is part of the binomial response likelihood and is not an interchangeable generic weight.

A second error is to think a fixed offset must be removed from the saturated model, or that it contributes an extra deviance term. The correction is to use the same offset in both predictors. The saturated model’s unrestricted predictor absorbs it, so the offset does not create a separate saturated-versus-fitted term; it affects the fitted mean and therefore the resulting deviance.