Variance Weights as Relative Precision

Why Variance Weights Need Their Own Interpretation

A weight in a generalized linear model can have different meanings depending on how it enters the model. Here the weight describes the response’s variance: holding the mean and dispersion structure fixed, a larger weight means a more precise response. That interpretation is useful when observations have different known levels of precision, but it does not by itself say that an observation represents several independent data points.

The distinction matters because the variance model determines how much each response contributes to a mean-parameter score, while literal replication changes the number of observations and the likelihood’s sampling structure. The resulting score equations can sometimes look alike, but the models need not be equivalent.

Definition: Let $Y_i$ be independent responses with $E[Y_i]=\mu_i$. A variance-weighted GLM specifies $$ \operatorname{Var}(Y_i)=\frac{\phi V(\mu_i)}{w_i}, $$ where $V(\mu_i)>0$ is the variance function, $\phi>0$ is a common dispersion parameter, and $w_i>0$ is a known variance weight. For fixed $\mu_i$, $V$, and $\phi$, the weight $w_i$ is a relative precision: multiplying it by $c>0$ divides the response variance by $c$.

The weights are assumed known and fixed for the calculation below. A weight of 4 therefore says that, at the same mean and variance function, the response variance is one quarter of the variance at weight 1. It does not say that four response values were observed. Nor does the variance model alone specify a full probability distribution: a likelihood interpretation requires additional distributional assumptions.

How a Variance Weight Changes the Score

Suppose the mean is linked to a linear predictor by $\eta_i=x_i^{\mathsf T}\beta$ and $\mu_i=g^{-1}(\eta_i)$. Write $d_i=d\mu_i/d\eta_i$. Under the variance model above, the standard quasi-score contribution for $\beta$ from observation $i$ is $$ U_i(\beta)=\frac{w_i}{\phi}\frac{y_i-\mu_i}{V(\mu_i)}d_i x_i. $$ This formula shows the two roles of the weight: it divides the modeled variance by $w_i$, and it multiplies the response’s score contribution by $w_i$. The latter formula is a quasi-score implication of the mean–variance specification; a variance specification by itself does not determine every feature of a likelihood.

For a likelihood derivation, assume more specifically that each response belongs to an exponential-dispersion model admitting the weighted form $$ f(y_i;\theta_i,\phi,w_i) =\exp\left\{\frac{w_i}{\phi}\bigl[y_i\theta_i-b(\theta_i)\bigr]+c_i(y_i,\phi,w_i)\right\}, $$ where $b$ is twice differentiable and $c_i$ does not depend on $\theta_i$. In this parameterization, $\mu_i=b'(\theta_i)$ and $V(\mu_i)=b''(\theta_i)$.

Theorem: Under the weighted exponential-dispersion model just stated, $E[Y_i]=\mu_i$, $\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i$, and the contribution of observation $i$ to the score for $\beta$ is $U_i(\beta)=\dfrac{w_i}{\phi}\dfrac{y_i-\mu_i}{V(\mu_i)}d_i x_i$, provided $\eta_i=x_i^{\mathsf T}\beta$, the link is differentiable, and the required derivatives exist.

Proof. Differentiate the log density with respect to $\theta_i$. Its derivative is $w_i(y_i-b'(\theta_i))/\phi$. Since $\theta_i$ is an interior natural parameter and differentiation under the normalizing integral is valid for this model, the expected score is zero; hence $E[Y_i]=b'(\theta_i)=\mu_i$. Differentiating the expected score with respect to $\theta_i$, or equivalently using the information identity for this regular exponential-dispersion model, gives $\operatorname{Var}(Y_i)=\phi b''(\theta_i)/w_i$. Thus the variance is $\phi V(\mu_i)/w_i$.

For the score in $\beta$, the chain rule gives $$ \frac{\partial \ell_i}{\partial\beta} =\frac{w_i}{\phi}(y_i-\mu_i)\frac{\partial\theta_i}{\partial\beta}. $$ Because $d\mu_i/d\theta_i=b''(\theta_i)=V(\mu_i)$ and $d\mu_i/d\eta_i=d_i$, another application of the chain rule yields $\partial\theta_i/\partial\beta=(d_i/V(\mu_i))x_i$. Substitution proves the stated score contribution. $\square$

The quasi-score has the same form without requiring that particular density. To see its relationship to the variance model, a quasi-likelihood contribution $Q_i$ is defined through $\partial Q_i/\partial\mu_i=(y_i-\mu_i)/\{\phi V(\mu_i)/w_i\}$. Differentiating with respect to $\beta$ multiplies this derivative by $d_i x_i$, giving the displayed $U_i(\beta)$. This is an estimating-score construction; it does not claim that every chosen variance function and weight produce a valid probability density.

Worked Example: Normal Responses With Different Precision

Suppose two independent normal responses share an unknown mean $\mu$, with $Y_i\sim N(\mu,\phi/w_i)$, and let $\phi=4$. The observed values are $y_1=10$, $y_2=14$, with $w_1=1$ and $w_2=4$. Calculate their variances and their score contributions at $\mu=12$, then solve the score equation for the fitted common mean.

For a normal identity-link model, $V(\mu)=1$, $d_i=1$, and $x_i=1$. Therefore $\operatorname{Var}(Y_1)=4/1=4$, whereas $\operatorname{Var}(Y_2)=4/4=1$. The second observation has four times the precision of the first under this model.

At $\mu=12$, the first score contribution is $U_1=(1/4)(10-12)=-0.5$. The second is $U_2=(4/4)(14-12)=2$. Their sum is $1.5$, so the score is positive at 12.

For a general $\mu$, setting the total score to zero gives $$ \frac{1}{4}(10-\mu)+\frac{4}{4}(14-\mu)=0. $$ Multiplying by 4 gives $10-\mu+4(14-\mu)=0$, or $66-5\mu=0$. Thus the fitted common mean is $\hat{\mu}=13.2$. This is the precision-weighted mean: $(1\cdot10+4\cdot14)/(1+4)=66/5=13.2$.

Using the Score With a Noncanonical Link

The weight multiplies the score contribution regardless of whether the link is canonical. The link still determines $d_i=d\mu_i/d\eta_i$, so it affects the contribution through the sensitivity of the mean to the linear predictor.

Worked Example: Gamma Variance and a Log Link

Consider a Gamma mean–variance model with $V(\mu)=\mu^2$, dispersion $\phi=2$, and log link $\eta_i=\log\mu_i=x_i^{\mathsf T}\beta$. For two observations, suppose $(x_1,y_1,\mu_1,w_1)=((1,0)^{\mathsf T},6,4,3)$ and $(x_2,y_2,\mu_2,w_2)=((1,1)^{\mathsf T},3,5,0.5)$. Calculate the score vector at these specified fitted means.

With the log link, $d_i=d\mu_i/d\eta_i=\mu_i$. Thus $$ U_i(\beta)=\frac{w_i}{\phi}\frac{y_i-\mu_i}{\mu_i^2}\mu_i x_i =\frac{w_i(y_i-\mu_i)}{\phi\mu_i}x_i. $$ For observation 1, the scalar multiplying $x_1$ is $3(6-4)/(2\cdot4)=6/8=0.75$, so $U_1=(0.75,0)^{\mathsf T}$.

For observation 2, that scalar is $0.5(3-5)/(2\cdot5)=-1/10=-0.1$, so $U_2=(-0.1,-0.1)^{\mathsf T}$. Adding the contributions gives the total score $(0.65,-0.1)^{\mathsf T}$. In particular, the second observation contributes to both the intercept and slope coordinates because its predictor vector is $(1,1)^{\mathsf T}$.

Variance Weights Are Not Automatically Replication Counts

Replication means that several independent response values are observed under the model. A variance weight instead specifies the variance of the response that is actually in the data. In some special models, a weighted score can coincide with the score from a replicated data set for a particular mean parameter. That algebraic coincidence does not establish that the data sets have the same likelihood, dispersion information, or sampling units.

Worked Example: Same Mean Score, Different Dispersion Estimate

Suppose the observed responses are $0$ and $2$, with weights $1$ and $3$, under a common-mean normal model with $\operatorname{Var}(Y_i)=\sigma^2/w_i$. Compare this with a data set containing one response equal to $0$ and three independent responses equal to $2$, all with variance $\sigma^2$.

In the variance-weighted model, the score equation for the common mean is proportional to $(0-\mu)+3(2-\mu)=0$, so $\hat{\mu}=6/4=1.5$. The weighted residual sum of squares is $$ 1(0-1.5)^2+3(2-1.5)^2=2.25+0.75=3. $$ There are two observed responses. Maximizing the normal likelihood over $\sigma^2$ therefore gives $\hat{\sigma}^2=3/2=1.5$.

With literal replication, there are four independent responses and the ordinary residual sum of squares is $ (0-1.5)^2+3(2-1.5)^2=3$. The normal maximum-likelihood estimate is now $\hat{\sigma}^2=3/4=0.75$. The mean score yields the same fitted mean in this example, but the dispersion estimates differ because the two models have different numbers of observed responses.

The point is not that these dispersion estimates should always be compared in this way; it is that a matching mean score does not turn a variance-weighted response into multiple independent observations. The sampling units and likelihood must come from the data-generating model.

Common Mistakes and Practical Checks

Reading a weight as a count. A weight of 5 means fivefold relative precision under the stated variance model, not automatically five observed individuals.
Ignoring the rest of the variance model. The absolute variance is $\phi V(\mu_i)/w_i$, not simply $1/w_i$. The comparison “four times the precision” holds with the same $\phi$ and $V(\mu_i)$.
Assuming a variance formula uniquely defines a likelihood. It does not. A quasi-score follows from the mean–variance specification; a likelihood score requires a compatible distributional model and its regularity conditions.
Forgetting the link derivative. The score includes $d_i=d\mu_i/d\eta_i$. The weight multiplies the contribution, but it does not replace the effect of the chosen link.
Concluding that equal coefficient estimates mean equivalent data. Different data structures can give the same estimating equation for a mean parameter and still differ in likelihood or dispersion information.

Key takeaway: A variance weight is a known relative-precision factor in $\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i$. It scales the corresponding score contribution by $w_i$, but it represents literal replication only when a separate sampling model justifies that interpretation.

Check Your Understanding

Use the variance model and assumptions stated in each question.

If two responses have the same mean, variance function, and dispersion, but their weights are 2 and 8, what is the ratio of their variances?
State the variance-weighted GLM model and identify the role of each term in it.
Why does a variance model alone not guarantee a likelihood score of the exponential-dispersion form?
For a log link, explain where the factor $d_i$ in the score contribution comes from.
Give one reason that a variance-weighted observation is not automatically equivalent to several independent replicated observations.