From a Variance Assumption to an Analysis
A quasi-likelihood analysis can be carried out without specifying a full probability distribution. What it does require is a mean model and a variance function. Together with a link, these determine an estimating equation; the data then determine its solution and estimates of uncertainty. The calculations below bring these pieces together for counts and grouped proportions.
For independent observations \(Y_i\), let \(\mu_i=E[Y_i]\), let \(g(\mu_i)=x_i^{\mathsf T}\beta\), and suppose \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\). Here \(x_i\) is a known covariate vector, \(\beta\) is the regression parameter, and \(\phi>0\) is a dispersion parameter. The quasi-score for \(\beta\) is
An estimate \(\hat\beta\) solves \(U(\beta)=0\). Multiplying the equation by the common factor \(\phi\) does not change its root. In the examples, this convenient rescaling removes \(\phi\) from the estimating equation. A common Pearson estimate is
where \(p\) is the number of fitted regression parameters. This is a residual-degrees-of-freedom estimate, generally used as a moment-based estimate rather than as an exact finite-sample identity.
Sandwich Uncertainty for an Estimating Equation
For a vector estimating equation \(U_n(\beta)=\sum_{i=1}^n u_i(\beta)=0\), define the empirical bread and meat by
The resulting estimated covariance is \(n^{-1}\hat A^{-1}\hat B\hat A^{-\mathsf T}\). The raw meat in this expression needs a centering condition. A standard set of sufficient conditions is that the observations are independent, \(E[u_i(\beta_0)]=0\) for every \(i\), the centered contributions satisfy a central limit theorem, their covariance average converges to a finite matrix \(B\), and the derivative average converges to a nonsingular matrix \(A\). Also assume a consistent root \(\hat\beta\), differentiability near \(\beta_0\), and a Taylor remainder negligible at the \(n^{-1/2}\) scale. Then
To see why, expand the estimating equation around \(\beta_0\). The stated derivative and remainder conditions give \(0=U_n(\beta_0)-nA(\hat\beta-\beta_0)+o_p(\sqrt n)\). Dividing by \(\sqrt n\) and rearranging yields \(\sqrt n(\hat\beta-\beta_0)=A^{-1}n^{-1/2}U_n(\beta_0)+o_p(1)\). The centered central limit theorem gives \(n^{-1/2}U_n(\beta_0)\xrightarrow{d}N(0,B)\); the result follows by the continuous mapping theorem. Consistency and the corresponding regularity for the empirical derivative and meat allow \(A\) and \(B\) to be replaced by \(\hat A\) and \(\hat B\). In particular, centering matters: without \(E[u_i(\beta_0)]=0\), raw outer products can include a contribution from nonzero individual means and need not estimate the average covariance.
For a contrast \(c^{\mathsf T}\beta\), the sandwich standard error is the square root of \(c^{\mathsf T}\widehat{\operatorname{Cov}}(\hat\beta)c\). A large-sample robust Wald statistic for \(H_0:c^{\mathsf T}\beta=r\) is
Under the null and the stated asymptotic conditions, this is approximately standard normal. With few independent observations, this calibration and the estimated sandwich covariance can both be unreliable.
Worked Problems
Worked Example: Two-Group Quasi-Poisson Counts
Suppose six independent counts are observed, with three in each of two groups. Use a log link and \(V(\mu)=\mu\), as in a quasi-Poisson variance specification. The counts are \(1,2,4\) in group 0 and \(3,5,8\) in group 1. Fit a separate mean for each group, estimate dispersion, and use a robust standard error to test equality of the log means.
Let the parameter vector be \(\beta=(\beta_0,\beta_1)^{\mathsf T}\), where \(\log\mu_g=\beta_g\). With the log link, \(d\mu_i/d\eta_i=\mu_i\). Thus the quasi-score, after dropping the common factor \(1/\phi\), is \(x_i(Y_i-\mu_i)\), where \(x_i\) selects the observation’s group. Setting each group score to zero gives \(\sum_{i\in g}(Y_i-\mu_g)=0\), so \(\hat\mu_g=\bar Y_g\). The estimates are
For group 0, the residuals are \(-4/3,-1/3,5/3\), whose squared values sum to \(42/9=14/3\). For group 1, the residuals are \(-7/3,-1/3,8/3\), whose squares sum to \(114/9=38/3\). Since \(V(\hat\mu_g)=\hat\mu_g\), the Pearson sum is
For the robust covariance, use individual score contributions \(u_i=x_i(Y_i-\mu_i)\). The negative derivative for group \(g\) sums to \(3\hat\mu_g\); its empirical meat is the sum of squared residuals in that group. The two group-specific sandwich variances are therefore
There are no cross-group terms because the two parameter coordinates use disjoint observations. To test \(H_0:\beta_1-\beta_0=0\), the estimated contrast is \(\log(16/7)\approx0.827\), and its variance is \(2/21+19/384\approx0.14472\). Thus its standard error is about \(0.380\), and \(Z\approx0.827/0.380=2.17\). The large-sample two-sided normal \(p\)-value is about \(0.030\). This is an asymptotic result based on only six observations, so it should not be treated as precise small-sample calibration.
Worked Example: Grouped Proportions With a Logit Link
Consider six independent groups of binomial observations. Each group has a number of successes \(Y_i\) out of \(m_i\) trials. Fit separate logits for groups A and B, assuming \(\operatorname{Var}(Y_i)=\phi m_i p_i(1-p_i)\). The data are A: \(2/5,5/10,8/15\), and B: \(3/5,7/10,12/15\). Estimate the group proportions and dispersion, then test equality of the logits robustly.
Write \(\operatorname{logit}(p_g)=\beta_g\). The mean is \(m_i p_i\), and its derivative with respect to the linear predictor is \(m_i p_i(1-p_i)\). After dividing by the variance function, the score contribution for the group parameter is \(Y_i-m_i p_i\), apart from the common factor \(1/\phi\). Setting its group sum to zero gives the total successes equal to the fitted total trials times \(p_g\). Hence
The Pearson sum for group A is \(0.5^2/(5/4)+0^2/(10/4)+0.5^2/(15/4)=4/15\). For group B the residuals are \(-2/3,-1/3,1\), and the fitted variance factors are \(5(44/225),10(44/225),15(44/225)\). Its Pearson contribution is
The total is \(4/15+75/88=1477/1320\), so with \(6-2=4\) residual degrees of freedom, \(\hat\phi_{\mathrm P}=1477/5280\approx0.280\). This low value reflects these particular grouped residuals; it is not evidence by itself that the binomial model is generally underdispersed.
For robust uncertainty, the group score contributions are the observed residuals \(Y_i-m_i\hat p_g\). The bread for group A is \(30(1/2)(1/2)=7.5\); its meat is \(0.5^2+0^2+0.5^2=0.5\). For group B the bread is \(30(11/15)(4/15)=176/30\), and its meat is \(({-2/3})^2+({-1/3})^2+1^2=14/9\). Therefore
For the contrast \(\beta_B-\beta_A\), the estimate is \(\log(11/4)\approx1.012\). Its estimated variance is \(2/225+350/30976\approx0.0541\), giving standard error about \(0.233\) and \(Z\approx4.35\). The large-sample Wald test rejects equal logits at conventional levels. The six groups, rather than the total number of individual trials, are the independent score contributions in this grouped sandwich calculation. With so few independent groups, its asymptotic uncertainty estimate should be interpreted cautiously.
Worked Example: A Different Variance Function for Counts
For four independent positive counts \(1,2,4,8\), fit a common mean \(\mu\) with a log link and variance specification \(V(\mu)=\mu^2\). Find the quasi-score estimate, Pearson dispersion estimate, and robust standard error for \(\log\mu\).
The derivative of the mean with respect to the log mean is \(\mu\). Thus, after dropping \(1/\phi\), each score contribution is \((Y_i-\mu)/\mu\). The estimating equation gives \(\sum_i(Y_i-\mu)/\mu=0\), or \(\sum_iY_i=4\mu\). Therefore \(\hat\mu=15/4=3.75\) and \(\widehat{\log\mu}=\log(15/4)\).
The residuals are \(-11/4,-7/4,1/4,17/4\), with squared sum \((121+49+1+289)/16=460/16\). Because \(V(\hat\mu)=225/16\), the Pearson sum is \((460/16)/(225/16)=92/45\). There are three residual degrees of freedom, so \(\hat\phi_{\mathrm P}=92/135\approx0.6815\).
The score contribution is \(u_i=(Y_i-\mu)/\mu\). Its negative derivative with respect to \(\log\mu\) is \(Y_i/\mu\), which sums at the estimate to \(4\). The empirical meat is the sum of the squared score contributions, equal to the Pearson sum \(92/45\). Thus
This variance function changes the score contributions and the uncertainty calculation. The example illustrates why a variance assumption must be carried through the estimating equation and variance estimate, rather than chosen after fitting the mean.
How to Read the Results
These problems separate three tasks that are easy to conflate. The mean model and variance function determine the estimating equation. The Pearson estimate summarizes residual variation relative to the specified variance function. The sandwich estimate instead uses the observed variability of the estimating-equation contributions together with their sensitivity to the parameter. When the variance function is correct, a model-based covariance can be more efficient; the sandwich is designed to remain valid under variance misspecification when its regularity and centering conditions hold.
For grouped proportions, a row may represent several trials, but the independent contribution in a grouped sandwich is the row’s score, not a separate contribution for each trial treated as if it were individually observed. If observations are dependent within clusters, treating the rows as independent is also inappropriate: the score contributions must reflect the independent sampling units. These choices affect the meat and therefore the reported uncertainty.
Common Mistakes
- Using the wrong variance factor. For grouped binomial counts, the variance factor is \(m_i p_i(1-p_i)\), not just \(p_i(1-p_i)\). Omitting \(m_i\) changes both Pearson residuals and the score.
- Confusing dispersion with robust uncertainty. The Pearson estimate is a scalar summary relative to a specified \(V(\mu)\); the sandwich uses the empirical covariance of score contributions. They answer related but different questions.
- Using uncentered contributions without justification. The raw empirical meat estimates the intended covariance under suitable centering and convergence conditions. If individual score means are not zero, the raw outer products can also capture variation in those means.
- Overstating a small-sample Wald result. Sandwich covariance and normal calibration are asymptotic. A large nominal \(Z\)-statistic does not remove the limitations of having very few independent observations.
Check Your Understanding
Check that you can connect the variance assumption to estimation and inference.
- For a log-link quasi-Poisson model with \(V(\mu)=\mu\), why does the quasi-score contribution simplify to a covariate vector multiplied by \(Y_i-\mu_i\), apart from dispersion?
- In the two-group count example, what are the bread and meat contributions for a group-specific log mean?
- Why does the grouped-proportion Pearson variance factor include the number of trials \(m_i\)?
- What centering condition makes the raw empirical sandwich meat estimate the covariance limit described in the asymptotic argument?
- When grouped observations are independent rows, what determines the number of independent contributions in the grouped sandwich calculation?