Why Consistency Is a Mean-Model Question
A quasi-likelihood analysis specifies how the response mean depends on covariates and uses a variance function to construct an estimating equation. It does not, by itself, specify a full probability distribution for each response. This leaves an important question: can solving the quasi-score equation still recover the true regression parameter as the sample grows?
The answer is yes under suitable conditions, but unbiasedness alone is not enough. The estimating equation must distinguish the true parameter from alternatives, and its sample version must approach its population version uniformly over the parameter space. These conditions establish consistency without requiring the responses to follow a particular likelihood distribution.
Unbiasedness at the True Parameter
Consider independent responses \(Y_1,\ldots,Y_n\), with fixed covariate vectors \(x_i\). Suppose their means follow a regression model \(E[Y_i]=\mu_i(\beta_0)=h(x_i^\mathsf{T}\beta_0)\), where \(h\) is differentiable. Let \(V(\mu)>0\) be the working variance function, and let \(\phi>0\) be a common dispersion parameter. The quasi-score contribution for \(\beta\) is \(\psi_i(\beta)=x_i\mu_i'(\eta_i)\{Y_i-\mu_i(\beta)\}/\{\phi V(\mu_i(\beta))\}\), where \(\eta_i=x_i^\mathsf{T}\beta\) and \(\mu_i'(\eta_i)\) is the derivative of the mean with respect to its linear predictor.
The dispersion factor may be omitted when it is common to all observations: multiplying the full equation by the same positive constant does not change its roots. The following fact explains the equation’s central property.
Indeed, at \(\beta_0\), every factor in \(\psi_i\) other than \(Y_i-\mu_i(\beta_0)\) is fixed, and \(E[Y_i-\mu_i(\beta_0)]=E[Y_i]-\mu_i(\beta_0)=0\). Multiplying this zero expectation by the fixed factors gives \(E[\psi_i(\beta_0)]=0\). Integrability ensures that the expectation exists.
Worked Example: A Quasi-Poisson Mean With Non-Poisson Responses
Let \(Y_1,\ldots,Y_n\) be independent with \(P(Y_i=0)=P(Y_i=2)=1/2\). Then \(E[Y_i]=1\) and \(\operatorname{Var}(Y_i)=1\). Fit the intercept-only log-mean model \(\mu(\beta)=e^\beta\), using the Poisson variance function \(V(\mu)=\mu\).
The quasi-score contribution, with \(\phi=1\), is \(\psi_i(\beta)=\{d\mu/d\beta\}\{Y_i-\mu\}/V(\mu)=e^\beta(Y_i-e^\beta)/e^\beta=Y_i-e^\beta\). Consequently, the estimating equation is \(\sum_i(Y_i-e^\beta)=0\), which gives \(e^{\hat{\beta}_n}=\bar{Y}\) and \(\hat{\beta}_n=\log\bar{Y}\) whenever \(\bar{Y}>0\).
For the observations \(0,2,2,0,2,2\), the sample mean is \(8/6=4/3\), so the fitted mean is \(4/3\) and \(\hat{\beta}_6=\log(4/3)\approx0.288\). The true mean is \(1\), so \(\beta_0=0\).
The responses are not Poisson: they take only the values 0 and 2, whereas a Poisson variable can take every nonnegative integer value. Nevertheless, the weak law of large numbers gives \(\bar{Y}\xrightarrow{p}1\). Since the logarithm is continuous at 1, \(\log\bar{Y}\xrightarrow{p}0\) wherever \(\bar{Y}>0\). The exceptional event \(\bar{Y}=0\) has probability \(2^{-n}\to0\); assigning any fixed finite value to the estimator on that event leaves its limit in probability unchanged. Thus the quasi-score estimator is consistent without a Poisson response distribution.
A Consistency Theorem for Estimating Equations
Write the normalized estimating equation as \(\Psi_n(\beta)=n^{-1}\sum_{i=1}^n\psi_i(\beta)\), and its expectation as \(m_n(\beta)=E[\Psi_n(\beta)]\). The observations, and hence the contributions, may have different distributions across \(i\). Let \(\Theta\) be a compact subset of a finite-dimensional parameter space containing \(\beta_0\).
- \(\sup_{\beta\in\Theta}\|\Psi_n(\beta)-m_n(\beta)\|\xrightarrow{p}0\); this is a uniform law of large numbers for the estimating equation.
- \(m_n\) converges uniformly on \(\Theta\) to a continuous function \(m\).
- \(m(\beta_0)=0\), and \(\beta_0\) is the unique zero of \(m\) in \(\Theta\).
- The estimator \(\hat{\beta}_n\) takes values in \(\Theta\) and satisfies \(\|\Psi_n(\hat{\beta}_n)\|\xrightarrow{p}0\).
The condition \(m(\beta_0)=0\) follows, for example, when the estimating equation is unbiased at \(\beta_0\) for every \(n\): then \(m_n(\beta_0)=0\), and uniform convergence implies \(m(\beta_0)=0\). The theorem also permits approximate roots, provided their equation residual converges to zero in probability.
Continuity, compactness, and the unique-zero condition give a positive lower bound for \(\|m(\beta)\|\) away from \(\beta_0\).
Uniform convergence makes \(\Psi_n(\beta)\) uniformly close to \(m(\beta)\) with probability tending to one.
A point away from \(\beta_0\) cannot have an estimating-equation residual close to zero when the population equation is bounded away from zero there.
Proof. Fix \(\varepsilon>0\). If the set \(A_\varepsilon=\{\beta\in\Theta:\|\beta-\beta_0\|\ge\varepsilon\}\) is empty, then every estimator taking values in \(\Theta\) is within \(\varepsilon\) of \(\beta_0\). Otherwise, \(A_\varepsilon\) is compact. Since \(m\) is continuous and has no zero on \(A_\varepsilon\), the minimum \(c_\varepsilon=\inf_{\beta\in A_\varepsilon}\|m(\beta)\|\) exists and is strictly positive.
Uniform convergence of \(m_n\) to \(m\), together with the uniform law of large numbers, gives \(\sup_{\beta\in\Theta}\|\Psi_n(\beta)-m(\beta)\|\xrightarrow{p}0\). On the event that this supremum is less than \(c_\varepsilon/3\), every \(\beta\in A_\varepsilon\) satisfies \(\|\Psi_n(\beta)\|\ge\|m(\beta)\|-\|\Psi_n(\beta)-m(\beta)\|>2c_\varepsilon/3\). But \(P(\|\Psi_n(\hat{\beta}_n)\|>c_\varepsilon/3)\to0\) by the approximate-root assumption. Thus the probability that \(\hat{\beta}_n\in A_\varepsilon\) tends to zero. This holds for every \(\varepsilon>0\), proving \(\hat{\beta}_n\xrightarrow{p}\beta_0\). \(\square\)
What Supports the Uniform Law?
The theorem makes uniform convergence an explicit requirement rather than hiding it inside the word “regularity.” One convenient sufficient condition illustrates how it can be checked. Suppose \(\Theta\) is compact, the contributions are independent, and there are finite constants \(M\) and \(C\) such that, for every \(i\) and \(\beta,\gamma\in\Theta\), \(\|\psi_i(\beta)\|\le M\) and \(\|\psi_i(\beta)-\psi_i(\gamma)\|\le C\|\beta-\gamma\|\) almost surely. Then the uniform law of large numbers above holds.
To see why, fix a grid spacing \(\delta>0\). Compactness gives a finite \(\delta\)-net \(G_\delta\subset\Theta\): every point of \(\Theta\) lies within distance \(\delta\) of a grid point. For each fixed grid point \(g\) and each coordinate, independence and boundedness imply that the variance of the average \(\Psi_n(g)\) is at most \(M^2/n\). Chebyshev’s inequality therefore gives convergence in probability of each coordinate to its expectation. There are finitely many grid points and coordinates, so the largest grid-point deviation also converges to zero in probability.
For any \(\beta\in\Theta\), choose \(g\in G_\delta\) within distance \(\delta\). The Lipschitz bound applies to both \(\Psi_n\) and \(m_n\), since taking expectations preserves that bound. Hence \(\|\Psi_n(\beta)-m_n(\beta)\|\le\|\Psi_n(g)-m_n(g)\|+2C\delta\). First choose \(\delta\) small enough that \(2C\delta\) is below any prescribed tolerance, then let \(n\) increase so that the finite-grid maximum is small with probability tending to one. This proves uniform convergence. In applications, bounded responses and smooth, bounded quasi-score coefficients on a compact parameter set can help verify such conditions. For unbounded responses, other uniform laws may apply, but their moment and smoothness requirements must be checked rather than presumed.
Worked Example: Consistency With a Working Gaussian Variance
Suppose there are two fixed design values, \(x=0\) and \(x=1\), with \(n_0\) and \(n_1\) independent observations in the respective groups. Let the model be \(E[Y\mid x]=\beta_0+\beta_1x\), with working variance \(V(\mu)=1\). In the \(x=0\) group, let each response be \(-1\) or \(1\), equally likely. In the \(x=1\) group, let each response be \(0\) or \(2\), equally likely. The true parameter is \((\beta_0,\beta_1)=(0,1)\).
With the identity link, the quasi-score equation is the ordinary least-squares equation \(X^\mathsf{T}(Y-X\beta)=0\). In this two-group design, its fitted values are the two group means: \(\hat{\beta}_0=\bar{Y}_0\) and \(\hat{\beta}_0+\hat{\beta}_1=\bar{Y}_1\). Therefore \(\hat{\beta}_1=\bar{Y}_1-\bar{Y}_0\).
For four observations in each group, take realized values \((-1,1,-1,1)\) at \(x=0\) and \((0,2,0,2)\) at \(x=1\). Then \(\bar{Y}_0=0\), \(\bar{Y}_1=1\), and the fitted coefficients are exactly \((0,1)\). More generally, as both group sizes tend to infinity, the weak law of large numbers gives \(\bar{Y}_0\xrightarrow{p}0\) and \(\bar{Y}_1\xrightarrow{p}1\). The displayed formulas then give \((\hat{\beta}_0,\hat{\beta}_1)\xrightarrow{p}(0,1)\).
The two groups have different response distributions, and neither group is Gaussian. Consistency follows from the correct conditional means, an identifying design, and convergence of the group averages—not from Gaussian likelihood theory.
Identification Is Essential
For a regression parameter to be consistently estimated as a vector, distinct parameter values must produce distinct population estimating equations, at least near the truth. The unique-zero condition in the theorem is one precise version of this requirement. A quasi-score can be unbiased at the truth and still fail to identify the parameter.
Worked Example: An Unbiased Equation That Does Not Identify Two Coefficients
Let \(Y_1,\ldots,Y_n\) be independent, each equal to 0 or 2 with probability \(1/2\). Fit the log-mean model \(\mu_i=\exp(\beta_1+\beta_2)\), with the Poisson variance function. The true mean is 1, and one valid true parameter is \((\beta_1,\beta_2)=(0,0)\).
Writing \(s=\beta_1+\beta_2\), the quasi-score is proportional to \((\sum_i(Y_i-e^s),\sum_i(Y_i-e^s))^\mathsf{T}\). At the truth its expectation is the zero vector, so the equation is unbiased. But both components depend only on \(s\), not on the two coefficients separately. Whenever \(\bar{Y}>0\), the roots satisfy \(s=\log\bar{Y}\), leaving infinitely many possibilities: \((\log\bar{Y},0)\), \((\log\bar{Y}+1,-1)\), and so on.
For the valid choice \(\hat{\beta}_n=(\log\bar{Y}+1,-1)\), with any fixed definition on the event \(\bar{Y}=0\), the fitted sum converges in probability to 0, but the estimator converges to \((1,-1)\), not to the designated true parameter \((0,0)\). The mean is consistently recovered, while the coefficient vector is not identified. An unbiased equation alone therefore cannot guarantee consistency of every root.
Common Mistake and Practical Implications
Common mistake: “The quasi-score has expectation zero at the true parameter, so any solution of its sample equation must be consistent.” The first clause establishes unbiasedness, not identification or convergence. If the population equation has multiple roots, the sample equation may have roots far from the chosen true parameter. Even with a unique population root, a sample equation that does not converge uniformly can behave poorly away from any fixed parameter value.
The correct argument keeps three questions separate: is the equation unbiased at the true value; does its population version have that value as its unique zero; and does the sample equation converge uniformly enough for its roots to track the population root? A correct full response distribution is not among these requirements. The mean model, the working variance used to construct the equation, the design, and the relevant convergence conditions are what matter for this consistency result.
This distinction is useful when the response distribution is complicated, overdispersed, or otherwise not credibly described by a standard likelihood family. Quasi-likelihood consistency can still be justified if the mean specification and identification are sound and the required regularity conditions hold. Conversely, a familiar variance function or an unbiased score does not repair a misspecified mean or an unidentified design.
Check Your Understanding
Use the definitions and conditions in this tutorial to answer the following questions.
- Which feature of the quasi-score makes its expectation zero at the true parameter?
- Why does the consistency theorem require a unique zero of the limiting population equation?
- In the two-group example, which sample quantities converge to the true regression coefficients’ information?
- What fails when a regression design has identical columns for two coefficients?
- Give one reason why a quasi-likelihood estimator can be consistent even when a Poisson or Gaussian response distribution is false.