Building a Quasi-Likelihood Analysis

From Assumptions to an Analysis

A quasi-likelihood analysis is useful when the mean relationship is the primary scientific target but a full probability distribution for the response is not justified. The analysis still requires explicit assumptions: a model for each conditional mean, a variance function describing how variability changes with that mean, and a method for estimating uncertainty. The variance function guides the estimating equation; it does not, by itself, specify a likelihood.

This tutorial builds those pieces into one regression workflow. We use independent positive responses with a log-linear mean and a quadratic variance function. The example is deliberately a quasi-likelihood analysis: no particular response distribution is asserted.

For the worked analysis, take \(g(\mu)=\log(\mu)\) and \(V(\mu)=\mu^2\). Thus the conditional mean is positive, and the assumed variance is proportional to the square of that mean. This choice is a convenient general variance structure for illustrating the calculations; it should not be mistaken for a claim that the responses follow a particular named distribution.

Deriving the Estimating Equation

Let \(D_i=\partial\mu_i/\partial\beta\), written as a column vector. A quasi-score contribution, with the common dispersion factor omitted, is \(\psi_i(\beta)=D_i\{Y_i-\mu_i\}/V(\mu_i)\). Omitting \(\phi\) does not change the solution of the estimating equation when \(\phi\) is a common positive scalar. This form follows directly by differentiating the quasi-likelihood integrand.

To verify the score formula, consider the quasi-likelihood contribution defined, up to a constant independent of \(\mu\), by integrating \((y_i-t)/V(t)\) with respect to \(t\). Its derivative with respect to \(\mu_i\) is \((y_i-\mu_i)/V(\mu_i)\). By the chain rule, its derivative with respect to \(\beta\) is \(D_i(y_i-\mu_i)/V(\mu_i)\). Summing these derivatives gives \(U(\beta)\). At the true parameter, \(D_i\) and \(V(\mu_i)\) are fixed given the covariates, so

For the stated log link, write \(\eta_i=z_i^T\beta\), so \(\mu_i=\exp(\eta_i)\) and \(D_i=\mu_i z_i\). Substituting \(D_i\) and \(V(\mu_i)=\mu_i^2\) gives

Consequently the estimating equation is

This is an estimating equation, not a likelihood score derived from a specified response distribution. Its validity as a mean-zero equation comes from the stated mean assumption. Consistency and asymptotic inference also require appropriate regularity conditions, including an identifiable mean model and sufficiently well-behaved independent observations.

Solving the Equation in a Small Data Set

With a binary covariate, the estimating equation separates into one equation for each exposure group. The intercept-only part and exposure part together imply that the sum of \(Y_i/\mu_i-1\) is zero in each group. Therefore the fitted mean in each group is its arithmetic sample mean. The following example shows the calculations rather than relying on that shortcut.

Estimating Dispersion

The mean estimates do not determine the scale of the variability. A standard moment-based estimate uses Pearson residuals, which divide each observed residual by the square root of its variance function. With \(p\) fitted regression parameters, the usual estimate is

The divisor \(n-p\) accounts approximately for the degrees of freedom used in fitting the mean. This is not generally an exact unbiasedness identity with fitted means. To see the moment calculation behind it, suppose first that the true means are known and the variance function is correct. Then

Estimating the regression parameters uses information from the same observations, so the residual sum is typically smaller; dividing by \(n-p\) is a conventional adjustment, not a universal exact correction. The estimate also depends on the chosen variance function and may be unstable in small samples.

Sandwich-Based Inference

A model-based covariance calculation relies on the stated variance function being correct. Sandwich covariance instead estimates variability of the empirical estimating-equation contributions. Define \(\hat\psi_i=\psi_i(\hat\beta)\), \(\hat A=-\partial U(\beta)/\partial\beta^T\) evaluated at \(\hat\beta\), and \(\hat B=\sum_i\hat\psi_i\hat\psi_i^T\). When the observations are independent and the estimating equation has a well-behaved, nonsingular derivative, the sandwich estimate is

This form follows from linearizing the estimating equation around the true parameter: the estimation error is approximated by the inverse derivative times the sum of the individual mean-zero contributions. The variance of that sum is estimated by adding their outer products, which yields the displayed sandwich. The approximation requires regularity conditions for the estimator and the empirical covariance; it is not a small-sample exact result. If the equation is written with every contribution divided by \(\phi\), both derivative and meat scale accordingly and the common factors cancel in the sandwich.

Reading the Analysis Responsibly

The example separates three tasks that should remain distinct in an applied report: specifying the mean model, specifying a variance function for the estimating equation, and estimating uncertainty. The coefficient estimate describes the fitted conditional mean relationship under the mean model. The Pearson estimate summarizes residual scale relative to the chosen variance function. The sandwich estimates coefficient uncertainty from observed variation in the estimating contributions.

Robust covariance does not make the entire analysis robust to every modeling error. In particular, it does not correct a misspecified mean relationship, dependence between observations when independence was assumed, or influential observations. The variance function still affects the estimating equation and hence can affect the coefficient estimate. Sandwich inference protects against some variance misspecification under suitable conditions; it does not replace careful modeling or diagnostics.