Variance Weights And Observation Precision

Why Observation Precision Matters

A generalized linear model (GLM) relates a response mean to predictors through a link function. But observations need not be equally precise. For example, a reported average based on many measurements may be more stable than one based on only a few. Prior weights let a model represent this difference: a larger weight means lower conditional variance, all else equal.

Here, “prior” means supplied to the model before fitting; it does not mean a Bayesian prior distribution. The weights are not predictor variables and do not multiply the observed responses. Their role is to describe relative precision or, in some settings, the number of observations summarized by a row.

Definition: Suppose the responses $Y_1,\ldots,Y_n$ are conditionally independent given their predictors, with means $\mu_i=E[Y_i\mid x_i]$. A GLM with prior weights $w_i>0$ has link $g(\mu_i)=x_i^\mathsf{T}\beta$ and conditional variances $$ \operatorname{Var}(Y_i\mid x_i)=\frac{\phi V(\mu_i)}{w_i}, $$ where $V(\mu_i)$ is the variance function and $\phi$ is a dispersion parameter. Thus, relative to an observation with weight 1, weight $w_i$ gives $1/w_i$ times the variance, for the same mean and dispersion.

The weight affects both the variance assigned to an observation and its contribution to fitting. In the exponential-dispersion representation, the part of observation $i$'s log likelihood that depends on its natural parameter $\theta_i$ is proportional to

\frac{w_i}{\phi}\{y_i\theta_i-b(\theta_i)\}.

Terms that do not depend on the fitted mean may also appear in the full likelihood. In particular, the exact normalizing terms depend on the distribution and how weights arise. For a fixed dispersion, the parameter-dependent likelihood kernel and the unit deviance are weighted by $w_i$. A high-weight observation therefore has a greater effect on the fit, but its effect is not simply to multiply its response by $w_i$.

How Weights Enter Variance and Likelihood

For a given mean $\mu_i$, the variance function describes the response’s baseline variability. Dividing by $w_i$ expresses the added precision. For example, if $Y_i$ is Gaussian with identity link, then $V(\mu_i)=1$, so its variance is $\phi/w_i$. If a binomial proportion is based on $m_i$ trials, its variance is $\mu_i(1-\mu_i)/m_i$; the trial count supplies the precision weight.

The likelihood interpretation can be seen directly from the exponential-family form. If a single observation has parameter-dependent log-likelihood kernel $\{y_i\theta_i-b(\theta_i)\}/\phi$, assigning it precision $w_i$ multiplies that kernel by $w_i$. The same scaling appears in the unit deviance: observation $i$'s contribution to the model deviance is $w_i d(y_i,\mu_i)$, where $d(y_i,\mu_i)$ is the unit deviance for one unit of information. For grouped binomial data, the trial count is the multiplier.

Interpretation: A weight of 4 means four times the precision of a weight of 1 under the stated variance model, not four times the response. It gives one-quarter the conditional variance when $\mu_i$, $V$, and $\phi$ are held fixed.

A brief derivation explains the familiar binomial case. If $K_i$ is the number of successes in $m_i$ independent Bernoulli trials with success probability $p_i$, then $E[K_i]=m_ip_i$ and $\operatorname{Var}(K_i)=m_ip_i(1-p_i)$. The observed proportion $Y_i=K_i/m_i$ therefore has

E[Y_i]=p_i,\qquad \operatorname{Var}(Y_i)=\frac{\operatorname{Var}(K_i)}{m_i^2} =\frac{p_i(1-p_i)}{m_i}.

Thus, when the response is a proportion, the number of trials $m_i$ acts as its precision weight. The corresponding binomial log-likelihood kernel is $m_i\{y_i\log p_i+(1-y_i)\log(1-p_i)\}$, so the same trial count scales the likelihood contribution.

Worked Examples

Worked Example: Gaussian Likelihood at Two Precision Levels

A hypothetical portfolio study models a claim-cost summary $Y$ with a Gaussian distribution and identity link. For this observation, $Y=12$, the fitted mean is $\mu=10$, the dispersion is $\phi=9$, and its prior weight is $w=4$. Calculate its conditional variance, standardized residual, and log-likelihood contribution. Compare the variance with weight 1.

Method. For a Gaussian identity-link model, $V(\mu)=1$, so $\operatorname{Var}(Y)=\phi/w$. The normal log density is $-\tfrac12\{\log(2\pi\operatorname{Var}(Y))+(Y-\mu)^2/\operatorname{Var}(Y)\}$.

Variance and residual. With $w=4$, the variance is $9/4=2.25$, and the standard deviation is $\sqrt{2.25}=1.5$. The residual is $12-10=2$, so the standardized residual is $2/1.5=1.3333$ to four decimals. Its squared value is $1.7778$.

Likelihood contribution. Substitution into the normal log density gives

\ell=-\frac12\left\{\log(2\pi\cdot2.25)+\frac{2^2}{2.25}\right\} \approx-\frac12(2.6488+1.7778)=-2.2133.

The log-likelihood is approximately $-2.2133$ to four decimals. With weight 1, the variance would be $9$, four times as large. The standardized residual would be $2/3=0.6667$. Its log-density contribution would be $-\tfrac12\{\log(2\pi\cdot9)+4/9\}\approx-2.2395$. The normalizing term matters: a higher precision does not imply every individual log-density value must be smaller.

Check and trap avoided. The variance ratio is $9/2.25=4$, as required by the weights’ inverse relationship with variance. Using $w\phi=36$ for the variance reverses the intended effect and is incorrect.

Worked Example: Grouped Binomial Data and Precision

Two hypothetical groups each have an observed success proportion of 0.60. Group A has 3 successes in 5 trials; Group B has 12 successes in 20 trials. Under a binomial model with success probability $p$, compare the variance of each observed proportion at $p=0.50$, and compare the likelihood ratios of $p=0.60$ to $p=0.50$.

Variance. For a proportion based on $m$ trials, the variance at $p=0.50$ is $p(1-p)/m=0.25/m$. For Group A it is $0.25/5=0.05$; for Group B it is $0.25/20=0.0125$. Group B’s variance is one-quarter of Group A’s.

Likelihood ratio. For $k$ successes in $m$ trials, the binomial likelihood is $\binom{m}{k}p^k(1-p)^{m-k}$. When taking a ratio at two values of $p$, the binomial coefficient cancels. For Group A, the ratio is

\frac{L(0.60)}{L(0.50)} =\left(\frac{0.60}{0.50}\right)^3 \left(\frac{0.40}{0.50}\right)^2 =1.2^3(0.8)^2=1.10592.

For Group B, the ratio is

\frac{L(0.60)}{L(0.50)} =1.2^{12}(0.8)^8 =\exp\{12\log(1.2)+8\log(0.8)\} \approx1.4959.

The larger group gives stronger relative likelihood support for $p=0.60$ over $p=0.50$, even though its observed proportion is identical. The calculations use exact grouped-binomial likelihoods, not a claim that the two groups’ proportions are independent single Bernoulli outcomes.

Check and trap avoided. Group B has four times as many trials, one-quarter the proportion variance, and a likelihood ratio farther from 1. Treating the two proportions as equally precise would ignore the different trial counts.

Worked Example: A Weighted Gaussian Mean

A hypothetical analyst observes two independent Gaussian summaries, $Y_1=4$ and $Y_2=10$, with common mean $\mu$, known dispersion $\phi=4$, and precision weights $w_1=1$ and $w_2=3$. Find the maximum-likelihood estimate of $\mu$, the weighted residual sum of squares at that estimate, and the variance of the estimated mean.

Method. The variances are $\phi/w_i$, so they are 4 and $4/3$. Ignoring terms not depending on $\mu$, the log likelihood is proportional to $-\sum_i w_i(Y_i-\mu)^2/(2\phi)$. Maximizing it is equivalent to minimizing the weighted sum of squared residuals. Differentiating that sum with respect to $\mu$ and setting the result to zero gives $\sum_i w_i(Y_i-\mu)=0$, hence $\hat\mu=\sum_i w_iY_i/\sum_i w_i$.

Estimate. The total weight is $1+3=4$, and the weighted response sum is $1(4)+3(10)=34$. Therefore, $\hat\mu=34/4=8.5$.

Weighted residual sum of squares. The residuals are $4-8.5=-4.5$ and $10-8.5=1.5$. Thus,

\sum_i w_i(Y_i-\hat\mu)^2 =1(-4.5)^2+3(1.5)^2 =20.25+6.75=27.

Variance and check. The variance of the weighted mean is $\phi/\sum_i w_i=4/4=1$, so its standard error is 1. The estimate, 8.5, is closer to the more precise observation, 10, than the unweighted mean, 7. The weighted residual calculation and the weighted-mean formula are consistent: the weighted residuals sum to $(-4.5)+3(1.5)=0$.

Trap avoided. Using the ordinary mean would treat the observations as equally precise. The weight is applied to each squared residual in the likelihood, which leads to the weighted mean rather than the ordinary mean.

What a Weight Does—and Does Not—Mean

The same numerical weight can arise in different ways, and the interpretation should follow the data structure. In grouped binomial data, a row with $m_i$ trials records a proportion based on $m_i$ Bernoulli outcomes. Its likelihood and precision follow from those trials. In a Gaussian model, a precision weight instead specifies an inverse-variance relationship. These interpretations are related through precision, but they are not interchangeable descriptions of every data set.

In particular, a general precision weight need not mean that the row represents exactly $w_i$ independent, identical observations. That literal replication interpretation is most direct for integer frequency weights and grouped counts. Noninteger precision weights can still specify relative variance or weight a likelihood contribution, but they do not describe a literal fractional number of replicated observations.

A further qualification concerns dispersion. If $\phi$ is fixed, multiplying all weights by a constant increases the likelihood’s parameter-dependent curvature and reduces variances implied by the specified model. If $\phi$ is estimated, its scale and the overall scale of the weights can interact. Consequently, the relative weights may determine relative precision while an overall rescaling may also change the fitted dispersion or uncertainty calculation. State what is held fixed before interpreting a change in all weights.

Setting	Meaning of weight	Variance implication
Gaussian response with precision weights	Relative inverse variance	$\phi/w_i$
Binomial proportion from $m_i$ trials	Number of trials summarized in the row	$\mu_i(1-\mu_i)/m_i$
Frequency-weighted observations	Number of times a record is represented	Depends on the repeated-observation model

Common Mistakes

Multiplying the response by the weight. A weight changes the observation’s precision or likelihood contribution; it does not replace $y_i$ with $w_i y_i$.
Using the weight in the wrong direction. Under the stated model, larger $w_i$ means smaller variance: $\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)/w_i$.
Calling every weight a trial count. A binomial denominator has a precise sampling interpretation. An arbitrary precision weight need not be an integer or represent that many Bernoulli trials.
Dropping the distribution’s normalizing terms when calculating an exact likelihood. Weighted likelihood kernels are useful for fitting, but a full probability or density also includes its appropriate normalizing terms.
Assuming a higher weight always produces a smaller individual log density. A density includes both a residual penalty and a normalizing term, as the Gaussian example shows.

Key takeaway: A GLM prior weight expresses relative information under a specified model. It divides the conditional variance and scales the parameter-dependent likelihood contribution. Correct interpretation requires identifying whether the weight represents precision, grouped observations, or frequency.

Check Your Understanding

For each problem, use the stated weight interpretation and keep the response distribution and dispersion fixed unless the problem says otherwise.

A Gaussian identity-link observation has $y=10$, fitted mean $\mu=7$, dispersion $\phi=6$, and weight $w=3$. Find its variance and the weighted quadratic term $w(y-\mu)^2/\phi$. (Answer: variance $2$; weighted quadratic term $4.5$.)
Two binomial groups have success proportions $8/10$ and $32/40$. At $p=0.50$, find the variance and standard error of each observed proportion. (Answer: for $m=10$, variance $0.025$, standard error $0.1581$; for $m=40$, variance $0.00625$, standard error $0.0791$, to four decimals.)
Independent Gaussian observations are $Y_1=5$ and $Y_2=11$, with common mean, known dispersion $\phi=3$, and weights $w_1=2,w_2=1$. Find the weighted mean estimate and its variance. (Answer: $\hat\mu=7$; variance $1$.)
A grouped binomial row has $m=12$ trials and observed proportion $y=0.25$. At fitted probability $\mu=0.50$, calculate its weighted unit deviance $2m\{y\log(y/\mu)+(1-y)\log[(1-y)/(1-\mu)]\}$. (Answer: approximately $3.1395$, to four decimals.)
In the Gaussian mean problem with $\phi=4$ and weights $1$ and $3$, all weights are multiplied by 9 while $\phi$ remains fixed. What is the variance of the weighted mean under the revised weights? (Answer: $4/[9(1+3)]=1/9$.)

Setting	Meaning of weight	Variance implication
Gaussian response with precision weights	Relative inverse variance	\(\phi/w_i\)
Binomial proportion from \(m_i\) trials	Number of trials summarized in the row	\(\mu_i(1-\mu_i)/m_i\)
Frequency-weighted observations	Number of times a record is represented	Depends on the repeated-observation model

What You'll Learn