Worked GMM For An Exponential Mean Model

Why use multiple moments for an exponential mean?

A conditional mean that changes exponentially with covariates is useful when the outcome is positive and a multiplicative change is more natural than a constant additive change. Its parameters enter through the exponential function, so the moments used to estimate them are nonlinear. Generalized method of moments (GMM) lets us combine several restrictions on those moments, including when there are more restrictions than parameters.

The purpose here is to carry out that calculation for a specified model. The moment conditions must be substantively justified: GMM does not make an invalid restriction valid. We will treat their validity as an assumption, estimate the parameters, quantify uncertainty, and assess the overidentifying restrictions as a group.

Model and moments: For observation $i$, let $Y_i$ be an outcome, $X_i$ a $k$-vector of regressors, and $Z_i$ a $q$-vector of observed variables used to form moments. The exponential conditional-mean model is $$ E[Y_i\mid X_i]=\mu_i(\beta)=\exp(X_i'\beta). $$ If the specified moment restrictions are valid, then $$ E[g_i(\beta_0)]=0,\qquad g_i(\beta)=Z_i\{Y_i-\exp(X_i'\beta)\}. $$ Here $\beta_0$ is the parameter vector satisfying the restrictions. The sample moment vector is $\bar g_n(\beta)=n^{-1}\sum_{i=1}^n g_i(\beta)$. The parameter dimension is $k$, and the number of moments is $q$.

The model describes the conditional mean, not the entire distribution of $Y_i$. In particular, it does not by itself specify a variance or require a particular probability distribution for the outcome. The restrictions $E[g_i(\beta_0)]=0$ follow from the model and the chosen moment variables only when the relevant assumptions hold. They are not guaranteed merely because the conditional mean has an exponential form.

For local identification, the derivative matrix $$ D=E\left[\frac{\partial g_i(\beta_0)}{\partial\beta'}\right] $$ must have full column rank $k$. In this model its sample analogue has rows informed by derivatives of the exponential mean: $$ \frac{\partial g_i(\beta)}{\partial\beta'}=-Z_i\exp(X_i'\beta)X_i'. $$ There must be enough independent information in the moments to distinguish changes in the $k$ parameters. When $q>k$, the model is overidentified: there are more moment restrictions than parameters, so the restrictions generally cannot all be set to zero in a finite sample.

Worked Example: Calculating a Nonlinear Moment Contribution

Suppose a hypothetical observation has outcome $Y_i=1.70$, regressors $X_i=(1,2)'$, and moment variables $Z_i=(1,0.5,2)'$. At the trial parameter $\beta=(0.20,0.10)'$, calculate its contribution to the three sample moments.

The fitted conditional mean is $$ \mu_i(\beta)=\exp(0.20+0.10(2))=\exp(0.40)=1.4918. $$ The residual is $1.70-1.4918=0.2082$, so $$ g_i(\beta)= \begin{pmatrix}1\\0.5\\2\end{pmatrix}(0.2082) = \begin{pmatrix}0.2082\\0.1041\\0.4164\end{pmatrix}. $$ These are one observation's moment contributions, not estimates of the population moments. The GMM calculation averages such contributions over all $n$ observations.

Estimation and the efficient weight

For a positive definite $q\times q$ weighting matrix $W$, the GMM estimate minimizes the quadratic form in the sample moments:

\hat\beta(W)=\arg\min_{\beta}\ Q_n(\beta;W), \qquad Q_n(\beta;W)=n\,\bar g_n(\beta)'W\bar g_n(\beta).

With more moments than parameters, a single parameter vector will not usually make every sample moment exactly zero. The weight determines how the estimator balances the discrepancies. A moment with high sampling variability should generally receive less weight than a more precise moment, while correlations between moments also matter.

Under independent, identically distributed observations, define the moment covariance matrix at the true parameter by $$ \Omega=E[g_i(\beta_0)g_i(\beta_0)']. $$ The efficient GMM weight is $W=\Omega^{-1}$, provided $\Omega$ is nonsingular. In practice, $\Omega$ is unknown. Two-step feasible GMM first estimates $\beta$ using an initial positive definite weight, commonly the identity matrix. It then estimates $\Omega$ using the preliminary residual-based moment vectors and minimizes the criterion again using the inverse of that estimate. If observations have serial dependence or clustering, the covariance estimate must account for that dependence; the independent-observation formula is not then appropriate.

Two-step feasible GMM: Obtain a preliminary estimate $\tilde\beta$. Form $\hat g_i=g_i(\tilde\beta)$ and estimate $\Omega$, for example by $\hat\Omega=n^{-1}\sum_i\hat g_i\hat g_i'$ under independent sampling. Then set $\hat W=\hat\Omega^{-1}$ and minimize $Q_n(\beta;\hat W)$. The second minimization gives the two-step estimate. The weight is held fixed during this minimization.

Worked Example: A Two-Step Estimate with Three Moments

Consider a hypothetical sample of $n=100$ units, with an outcome measured in units and regressors $X_i=(1,x_i)'$. Three moment variables produce $q=3$ moments, while the model has $k=2$ parameters. Suppose an identity-weight preliminary fit gives $\tilde\beta=(0.18,0.12)'$. The resulting covariance estimate is $$ \hat\Omega= \begin{pmatrix} 1&0&0\\ 0&4&0\\ 0&0&1 \end{pmatrix}. $$ Thus the feasible efficient weight for the second minimization is $$ \hat W=\hat\Omega^{-1}= \begin{pmatrix} 1&0&0\\ 0&0.25&0\\ 0&0&1 \end{pmatrix}. $$

Suppose numerical minimization of the nonlinear second-step criterion yields $\hat\beta=(0.20,0.10)'$. At that estimate, the sample moments and derivative matrix are $$ \bar g_n(\hat\beta)= \begin{pmatrix}0.05\\0.10\\-0.05\end{pmatrix}, \qquad \hat D= \begin{pmatrix} -1&0\\ 0&-2\\ -1&-1 \end{pmatrix}. $$ This is an overidentified fit: the three moments are not all zero, but the weighted criterion is minimized at the reported parameter values. For example, the first-order condition is satisfied because $$ \hat D'\hat W\bar g_n(\hat\beta)= \begin{pmatrix}-1&0&-1\\0&-2&-1\end{pmatrix} \begin{pmatrix}0.05\\0.025\\-0.05\end{pmatrix} = \begin{pmatrix}0\\0\end{pmatrix}. $$ A numerical optimizer uses the full nonlinear moment functions; the displayed summaries show the resulting fit and its local derivative information.

Sandwich uncertainty

The asymptotic covariance matrix of a GMM estimate accounts both for the sensitivity of the moments to the parameters and for variation in the moments themselves. For a fixed weight $W$, its sample analogue is the sandwich $$ \widehat{\operatorname{Var}}(\hat\beta)= \frac{1}{n} (\hat D'W\hat D)^{-1} \hat D'W\hat\Omega W\hat D (\hat D'W\hat D)^{-1}. $$ This expression requires a full-rank derivative matrix and a suitable estimate of the moment covariance. With the efficient weight $W=\hat\Omega^{-1}$, it simplifies to $$ \widehat{\operatorname{Var}}(\hat\beta) =\frac{1}{n}(\hat D'\hat\Omega^{-1}\hat D)^{-1}. $$ The factor $1/n$ is essential: the matrix without it estimates the asymptotic covariance of $\sqrt n(\hat\beta-\beta_0)$, not the covariance of $\hat\beta$ itself.

Worked Example: Standard Errors and Parameter Intervals

Continue with the hypothetical two-step fit above. Using the displayed derivative and efficient weight gives $$ \hat D'\hat W\hat D= \begin{pmatrix}2&1\\1&2\end{pmatrix}, \qquad (\hat D'\hat W\hat D)^{-1} =\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}. $$ Therefore, with $n=100$, $$ \widehat{\operatorname{Var}}(\hat\beta) =\frac{1}{100}\frac{1}{3} \begin{pmatrix}2&-1\\-1&2\end{pmatrix} = \begin{pmatrix} 0.006667&-0.003333\\ -0.003333&0.006667 \end{pmatrix}. $$ Each standard error is $\sqrt{0.006667}=0.08165$. This matrix is based on the stated $\hat D$ and $\hat\Omega$; the standard errors inherit their units from the corresponding parameters.

For example, a large-sample 95% Wald interval for $\beta_1$ is $$ 0.10\ \mathbin{\pm}\ 1.96(0.08165) =(-0.0600,\ 0.2600). $$ Under the model, a one-unit increase in $x$ multiplies the conditional mean by $\exp(\beta_1)$. The fitted multiplier is $\exp(0.10)=1.105$, or about a 10.5% increase. Transforming the interval endpoints gives a multiplier interval of approximately $(0.942,\ 1.297)$. These are large-sample confidence intervals for the parameter and its implied multiplier, not probabilities that fixed parameter values lie in the calculated intervals.

Overidentification and the Hansen J test

When the moment restrictions are valid and the model is identified, the efficient-weight GMM criterion evaluated at the estimate has an asymptotic chi-square distribution under the null that all specified moment conditions hold. Its degrees of freedom are the number of moments minus the number of estimated parameters, $q-k$. This is the Hansen J test. It assesses the restrictions jointly; it does not identify which particular moment is problematic if the test rejects.

J=n\,\bar g_n(\hat\beta)'\hat\Omega^{-1}\bar g_n(\hat\beta) \ \overset{a}{\sim}\ \chi^2_{q-k}.

The reference distribution requires valid moments, appropriate large-sample regularity conditions, and a consistent covariance estimate. The test is not available as an overidentification test when $q=k$, because then there are no excess restrictions to test. A failure to reject is not proof that every moment is correct; limited sample size or weak information can leave meaningful departures undetected.

Worked Example: Computing and Interpreting the J Statistic

For the three-moment, two-parameter fit, calculate the J statistic using the reported sample moments and efficient weight: $$ J=100\left[ (0.05)^2+0.25(0.10)^2+(-0.05)^2 \right] =100(0.0025+0.0025+0.0025) =0.750. $$ There are $q-k=3-2=1$ degrees of freedom. A $\chi^2_1$ statistic of $0.750$ has a p-value of approximately $0.3865$.

At a 5% significance level, this result does not provide evidence to reject the joint moment restrictions. It does not establish that the conditional-mean model or every moment is correct, and it does not assign a probability to the null hypothesis. The conclusion is limited to the evidence in this sample against the specified set of restrictions.

Common mistake: treating weighting as a cure for invalid moments

A frequent error is to assume that using $\hat\Omega^{-1}$ makes the moments valid. It does not. The efficient weight accounts for how variable and correlated valid moments are; it cannot repair a moment whose population expectation is nonzero at the parameter of interest. Another error is to read a non-rejection of the J test as confirmation that the model is true. The test can fail to detect misspecification, and it evaluates the restrictions jointly rather than certifying them one by one.

Key takeaway: Exponential GMM combines the nonlinear conditional mean $\exp(X_i'\beta)$ with explicitly justified moment restrictions. Efficient weighting and sandwich uncertainty depend on the covariance of those moments, while the J test evaluates the excess restrictions as a group.

Check Your Understanding

Answer these questions using the definitions and calculations above.

For the exponential conditional-mean model, write the moment vector when there are three moment variables.
If there are four moments and two parameters, how many degrees of freedom does the overidentification test have?
Why does the efficient weight use the inverse of the moment covariance matrix?
In the numerical example, why is it possible for the GMM estimate to be a minimizer even though none of the three sample moments is zero?
What does a J-test p-value of 0.3865 imply at the 5% level, and what does it not establish?