Weight Semantics and Standard Errors

Why the Meaning of a Weight Matters

In a generalized linear model, weights can enter the coefficient covariance calculation through the same-looking information matrix. That algebraic resemblance does not make the underlying data models interchangeable. A frequency weight says that a record represents replicated observations; a variance weight says that one observation has a specified relative precision. The distinction determines what random data the standard error describes.

Write the linear predictor as \(\eta_i=x_i^{\mathsf T}\beta+o_i\), where \(x_i\) is the covariate vector, \(\beta\) is the coefficient vector, and \(o_i\) is a fixed offset. Let \(\mu_i=E[Y_i]\), and let \(V(\mu_i)\) be the GLM variance function, so that an unweighted response has variance \(\phi V(\mu_i)\). Here \(\phi\) is the dispersion parameter. We focus on model-based covariance estimates, treating \(\phi\) as known unless stated otherwise.

For compactness, define the information contribution from one unweighted observation at row \(i\) by

The derivative is evaluated at the model parameters. We assume it exists and that the relevant information matrix is finite and nonsingular. For frequency weights, \(m_i\) identical covariate rows contribute \(m_iD_ix_ix_i^{\mathsf T}\). For variance weights, \(w_iD_ix_ix_i^{\mathsf T}\) is the model-based information contribution under the stated variance model.

Frequency Weights: Replicated Observations

Suppose each row represents \(m_i\) independent responses with the same covariates and mean. Their log-likelihood contributions add. Consequently, the weighted log-likelihood is the ordinary per-observation log-likelihood multiplied by \(m_i\) and summed over rows. Its expected information is the sum of the information from all represented observations.

To justify the formula, let \(I_i(\beta)\) denote the expected Fisher information from one observation at row \(i\). Under the regular GLM likelihood assumptions—differentiable likelihood, parameter-independent support, and valid differentiation under the integral—the information identity gives \(I_i(\beta)=-E[\partial^2\ell_i/\partial\beta\partial\beta^{\mathsf T}]\). The GLM mean and variance specification yields \(I_i(\beta)=D_ix_ix_i^{\mathsf T}/\phi\). For \(m_i\) independent observations, log-likelihoods and expected information add, giving

The inverse expected information is the model-based covariance used for the usual likelihood-based standard errors (as a large-sample approximation to the estimator’s covariance). Thus the covariance estimate is \(\phi\left(\sum_i m_iD_ix_ix_i^{\mathsf T}\right)^{-1}\), as claimed. This derivation depends on independent replicated observations; a weight that merely happens to be an integer does not establish that assumption. \(\square\)

Variance Weights: One Observation at Each Precision

A variance weight does not assert that a row represents several responses. It specifies how variable the single response is relative to the others. If responses are independent and \(\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i\), the quasi-score for \(\beta\) is

The weights appear because a response with smaller variance should contribute more to estimating the mean relationship. The covariance result follows from the variability and sensitivity of this estimating equation.

For a complete derivation, set \(g_i=(d\mu_i/d\eta_i)/V(\mu_i)\) and \(A=\sum_i w_iD_ix_ix_i^{\mathsf T}\). Independence and the specified response variances give

The expected negative derivative of the score is \(A\): when differentiating, the term containing \(Y_i-\mu_i\) has expectation zero, while differentiating the mean contributes \((d\mu_i/d\eta_i)^2/V(\mu_i)\). A first-order expansion of the estimating equation around the true coefficient therefore gives covariance \(A^{-1}(\phi A)A^{-1}=\phi A^{-1}\). This is the model-based covariance, or the large-sample covariance from the linearization of the score root. It requires independent responses, the stated mean and variance structure, and a nonsingular \(A\). \(\square\)

Same Formula, Different Sampling Story

Both derivations lead to a weighted information matrix, but the interpretation of its weight factor differs. Under frequency weighting, the factor counts independent copies of an observation. Under variance weighting, it expresses relative precision for a single observation. Equal numerical values can yield equal model-based covariance estimates when the mean model, dispersion, and covariate rows also agree; that equality alone does not make the data-generating assumptions equivalent.

Reading and Reporting the Standard Error

For a coefficient \(\beta_j\), the model-based standard error is the square root of the \(j\)th diagonal element of the estimated covariance matrix. Off-diagonal elements describe estimated coefficient covariances; they should not be discarded when forming standard errors for linear combinations. For a vector \(c\), the estimated variance of \(c^{\mathsf T}\hat\beta\) is \(c^{\mathsf T}\widehat{\operatorname{Cov}}(\hat\beta)c\).

These formulas rely on the chosen model for variation. If responses are correlated, if the variance-weight model is wrong, or if the frequency weights do not truly represent independent replication, the model-based covariance may not describe actual sampling uncertainty. In such cases, an appropriate robust or cluster-based covariance analysis may be needed; changing the weight label alone cannot repair a mismatched model.

Common Mistake

Wrong reasoning: “The software multiplies each row’s information by its weight, so a weight of 5 always means five independent observations and gives the same kind of standard error.”

Correction: Multiplication by 5 can arise from five independent replicated records, or from one response assigned five times the relative precision. Those are different assumptions about the random data. The covariance calculation is justified only after identifying which model the weights represent and checking its independence, mean, and variance assumptions.