Why the Estimating Equation Determines the Variance
A g-estimate is obtained by choosing a blip parameter so that empirical moment conditions are approximately satisfied. Its asymptotic variance has two distinct ingredients: how rapidly the population moment changes when the parameter moves, and how much the empirical moment fluctuates from sample to sample. The first ingredient is a derivative; the second is a covariance. Their combination is the sandwich variance.
The derivation applies to moment equations arising from g-estimation, including equations built from treatment-residual instruments and blipped-down outcomes. For interpretation, let \(U_i(\theta)\) denote the outcome after applying the candidate blip-down transformation for subject \(i\), and let \(R_i\) be a vector of instruments. A typical contribution is \(R_i U_i(\theta)\). Under the identification conditions for the structural nested mean model, its population expectation is zero at the true blip parameter. The variance calculation below concerns the sampling behavior of that estimating equation; it does not replace the identification argument.
The conditions behind this definition matter. The observations must be independent and identically distributed for the stated covariance and central limit theorem; the moment must have a finite second moment; the estimator must be consistent for an interior point \(\theta_0\); and the moment must be differentiable near \(\theta_0\). In addition, a uniform law of large numbers for its derivative and a Taylor remainder that is \(o_p(n^{-1/2})\) are needed. Finally, \(A\) must be nonsingular. In g-estimation, these regularity conditions are in addition to the substantive identification assumptions, such as sequential exchangeability, positivity, and correct specification of the blip model and moment conditions.
Deriving the Just-Identified Sandwich
The derivation makes the two components transparent. Expand the empirical equation around the true parameter. Under the differentiability and uniform convergence conditions just stated,
Solving this expansion gives the estimator’s first-order representation. Since \(E[\psi(O;\theta_0)]=0\), the multivariate central limit theorem applies to the empirical moments.
This proves the sandwich result: multiplication by \(A^{-1}\) converts moment fluctuations into parameter fluctuations, while \(\Omega\) describes the covariance of those moment fluctuations. The minus sign in the linear representation disappears from the covariance. An empirical sandwich replaces \(A\) and \(\Omega\) by consistent sample estimates, for example the sample average derivative and the sample covariance matrix of the fitted contributions.
Worked Example: One Treatment Decision and a Scalar Blip
Suppose \(A\) is a binary treatment with \(P(A=1)=1/2\), and the outcome follows \(Y=2A+3+\varepsilon\), where \(\varepsilon\) is independent of \(A\), has mean zero, and has variance \(4\). The constant 3 is a prognostic component; the blip parameter of interest is the treatment effect \(\theta_0=2\). With the known propensity \(e=1/2\), consider the g-estimation contribution \(\psi(A,Y;\theta)=(A-e)(Y-\theta A)\).
At \(\theta_0=2\), the blipped-down outcome is \(Y-2A=3+\varepsilon\). Independence and \(E[A-e]=0\) imply \(E[\psi(\theta_0)]=0\). The derivative is \(\partial\psi/\partial\theta=-(A-e)A\), so \(A_{\!0}=E[-(A-1/2)A]=-1/4\).
Because \(A-1/2\) is equally likely to be \(1/2\) or \(-1/2\), its squared value is always \(1/4\). Also \(E[(3+\varepsilon)^2]=9+4=13\), since \(E[\varepsilon]=0\). Thus \(\Omega=E[\psi(\theta_0)^2]=(1/4)(13)=13/4\). The asymptotic variance is \[ A_{\!0}^{-2}\Omega=16(13/4)=52. \] Consequently \(\operatorname{Var}(\hat\theta)\) is approximately \(52/n\). The large variance here reflects the variability in the prognostic component left in the outcome moment, not a change in the blip parameter.
Overidentified Moments
With more moment conditions than blip parameters, the estimator generally cannot set every empirical moment to zero. Let \(m(O;\theta)\) be a \(q\)-vector of moment contributions, with \(q>p\), \(E[m(O;\theta_0)]=0\), derivative \(D=E[\partial m(O;\theta_0)/\partial\theta']\), and covariance \(\Omega=\operatorname{Var}(m(O;\theta_0))\). A GMM estimator minimizes \(\bar m_n(\theta)'W_n\bar m_n(\theta)\), where \(W_n\) converges in probability to a symmetric positive definite matrix \(W\). Assume \(D\) has full column rank, the same differentiability and moment conditions needed for a central limit theorem and Taylor expansion hold, and the estimator is consistent.
At an interior minimizer, the first-order condition is \(D'W\bar m_n(\hat\theta)\) equal to zero up to terms negligible at the \(n^{-1/2}\) scale. Expanding \(\bar m_n(\hat\theta)\) around \(\theta_0\) gives \((D'WD)\sqrt n(\hat\theta-\theta_0)=-D'W\sqrt n\,\bar m_n(\theta_0)+o_p(1)\). Full column rank of \(D\) and positive definiteness of \(W\) make \(D'WD\) nonsingular. Applying the central limit theorem therefore yields the following result.
This is the overidentified sandwich: the derivative maps parameter changes into moment changes, \(W\) determines how the moments are combined, and \(\Omega\) measures their joint variability. If \(q=p\), the equation is exactly identified and any nonsingular weighting matrix gives the same first-order variance. With \(q>p\), different weights can produce different variances; the optimal-instrument and optimal-weighting discussion establishes how to choose among them. The formula here is the variance for the weight actually used.
Worked Example: Two Moments for One Blip Parameter
Suppose a scalar parameter is estimated from two valid moment conditions. At the truth, let the derivative vector be \(D=(1,2)'\), the moment covariance be \(\Omega=\begin{pmatrix}4&1\\1&9\end{pmatrix}\), and use \(W=I\), the two-dimensional identity matrix. These numbers define a valid covariance matrix: its diagonal entries are positive and its determinant is \(4(9)-1=35>0\).
First, \(D'WD=1^2+2^2=5\). Next, \(\Omega D=(6,19)'\), so \(D'W\Omega WD=D'\Omega D=1(6)+2(19)=44\). Substitution into the overidentified formula gives \[ V_W=5^{-1}(44)5^{-1}=\frac{44}{25}=1.76. \] Thus the estimator’s asymptotic variance is \(1.76/n\). The covariance between the two moment contributions is included: omitting the off-diagonal entries of \(\Omega\) would instead give \(D'\operatorname{diag}(4,9)D=40\), a different and incorrect variance for the specified joint moments.
When the Propensity Is Estimated
A g-estimation moment may depend on a propensity parameter \(\eta\), for example through a treatment-residual instrument. If \(\eta\) is estimated from the same data, treating it as fixed can give the wrong variance. Let \(\hat\eta\) solve \(n^{-1}\sum_i g(O_i;\eta)=0\), where \(E[g(O;\eta_0)]=0\) and \(H=E[\partial g(O;\eta_0)/\partial\eta']\) is nonsingular. Let \(C=E[\partial\psi(O;\theta_0,\eta_0)/\partial\eta']\). Under joint regularity conditions, a Taylor expansion of the nuisance equation gives \(\sqrt n(\hat\eta-\eta_0)=-H^{-1}n^{-1/2}\sum_i g(O_i;\eta_0)+o_p(1)\).
Substitute this expansion into the g-estimation equation. Its first-order empirical contribution is not just \(\psi\), but \(\psi-C H^{-1}g\). Therefore, when \(\theta\) is just identified, its asymptotic covariance is \(A^{-1}\operatorname{Var}(\psi-C H^{-1}g)A^{-T}\), where \(A=E[\partial\psi/\partial\theta']\). This formula includes the covariance between the original moment and the nuisance-estimation contribution. If the derivative \(C\) is zero, estimating \(\eta\) has no first-order effect on this equation; that special case should be established, not assumed.
Worked Example: Fitting the Marginal Treatment Probability
Use the same data-generating model \(Y=2A+3+\varepsilon\), with \(P(A=1)=1/2\), \(\operatorname{Var}(\varepsilon)=1\), and \(\varepsilon\) independent of \(A\). Now estimate the propensity by the sample mean: \(\hat\eta=\bar A\), whose estimating contribution is \(g(A;\eta)=A-\eta\). The outcome moment is \(\psi=(A-\eta)(Y-\theta A)\).
At \((\theta_0,\eta_0)=(2,1/2)\), \(A=E[\partial\psi/\partial\theta]=-1/4\), as before. The nuisance derivative is \(C=E[-(Y-2A)]=-E[3+\varepsilon]=-3\). For \(g=A-\eta\), \(H=-1\). Hence the corrected empirical contribution is \(\psi-C H^{-1}g=\psi-3g\).
At the truth this simplifies exactly: \[ \psi-3g=(A-1/2)(3+\varepsilon)-3(A-1/2)=(A-1/2)\varepsilon. \] Its variance is \(E[(A-1/2)^2]E[\varepsilon^2]=(1/4)(1)=1/4\). The parameter variance is therefore \((-1/4)^{-2}(1/4)=4\), or \(4/n\) asymptotically. If one incorrectly treated the propensity as known, the moment variance would be \((1/4)E[(3+\varepsilon)^2]=(1/4)(10)=2.5\), giving \(2.5/(1/16)=40\), or \(40/n\). The fitted propensity removes the sample-constant prognostic component in this example. This reduction is a consequence of the specified equations and model; it is not a general promise that nuisance fitting always reduces variance.
Diagnostics, Alternatives, and Limitations
A sandwich estimate is only as credible as its moments and regularity conditions. Useful checks include whether the empirical derivative is nearly singular, whether treatment residuals have very little variation, whether fitted moment contributions have extreme values, and whether a few observations dominate the covariance estimate. In an overidentified analysis, inspect the scale and dependence of all moment contributions; a numerically stable optimizer does not establish that the moments are informative or correctly specified.
- Weak identification: if the derivative matrix is close to rank deficient, its inverse can magnify small moment fluctuations. A finite-looking estimate does not guarantee a stable variance.
- Incorrect moment covariance: calculating separate variances but omitting cross-covariances is wrong when the moment contributions are correlated.
- Ignoring fitted nuisance parameters: using the covariance of \(\psi\) alone is generally incorrect unless the nuisance derivative is zero or the relevant influence contribution is otherwise accounted for.
- Model-based variance: a variance formula derived from a fully specified outcome distribution may be more efficient if that distribution is correct, but can be misleading under misspecification. The sandwich instead uses the empirical moment covariance and does not require that outcome distribution to be correct; it still requires valid moments and a correctly specified estimating target.
- Resampling: a bootstrap can approximate sampling variability when the full estimation procedure, including nuisance fitting, is repeated in each resample. It does not repair nonidentification, invalid moments, or poor finite-sample behavior automatically.
The usual sandwich is a first-order, large-sample result. It does not account for bias from a misspecified blip function or failed exchangeability, and it does not guarantee reliable finite-sample approximations with sparse treatment patterns or heavy-tailed contributions. With clustered, longitudinally dependent, or otherwise non-independent observations, the independent-observation covariance above must be replaced by one appropriate to the sampling design. The essential comparison is not “sandwich versus no assumptions”: it is which assumptions justify the estimating equation and which covariance matches the actual sampling structure.
Check Your Understanding
Answer each question, then compare with the solutions.
- For a just-identified estimating equation, name the derivative and variability components of the sandwich covariance.
- In an overidentified problem with \(q>p\), why is the covariance of each moment separately insufficient to calculate the estimator variance?
- For an estimated nuisance parameter, what determines whether its estimation affects the g-estimator at first order?
- Suppose the derivative matrix is nearly singular. What does that imply for the variance calculation and its interpretation?
Solutions. (1) They are \(A=E[\partial\psi/\partial\theta']\) and \(\Omega=\operatorname{Var}(\psi)\); they combine as \(A^{-1}\Omega A^{-T}\). (2) The moments can be correlated, and the GMM variance uses the full covariance matrix, including off-diagonal entries. (3) The nuisance derivative \(C\) and its influence contribution determine the correction; if \(C=0\), the nuisance has no first-order effect in this expansion. (4) The inverse derivative can amplify sampling noise substantially, so the estimate may be weakly identified and the asymptotic approximation unstable.