Why the Weighting Matrix Matters
Minimum-distance estimation compares sample restrictions with the values implied by a candidate parameter. When there are more restrictions than parameters, the restrictions may not all be matched exactly in a finite sample. The weighting matrix determines how discrepancies across restrictions are combined. A restriction that is noisy should generally count less than a precise one, and correlations between restrictions also matter.
The key object is the asymptotic covariance matrix of the sample restrictions. Its inverse gives the asymptotically efficient weighting matrix under regularity conditions. This choice is not merely a convenient rescaling: among minimum-distance estimators formed from the same restrictions, it yields the smallest first-order asymptotic covariance matrix.
Setup and Assumptions
Let \(\theta_0\) be a \(p\)-dimensional parameter and let \(r_n(\theta)\) be a \(q\)-dimensional vector of sample restrictions, with \(q\geq p\). The population restrictions satisfy \(r(\theta_0)=0\). The minimum-distance estimator minimizes the weighted quadratic discrepancy:
Here \(W_n\) is a symmetric positive-definite weighting matrix. Multiplying all its entries by the same positive constant does not change the minimizer. For the efficiency result below, assume that the restrictions are differentiable near \(\theta_0\), that their Jacobian has full column rank, and that the estimator is consistent and an interior solution. Also assume the sample restrictions have the asymptotic distribution
where \(\Omega\) is positive definite and \(G\) is a \(q\times p\) matrix of rank \(p\). Finally, suppose \(W_n\) converges in probability to a positive-definite matrix \(W\). These conditions make the local curvature of the objective informative about all \(p\) parameters and ensure that the covariance matrix used to construct the optimal weight can be inverted.
How the Efficient Weight Arises
A first-order expansion around \(\theta_0\) gives \(r_n(\theta)\) approximately equal to \(r_n(\theta_0)+G(\theta-\theta_0)\). Minimizing the resulting quadratic approximation yields the limiting distribution of the estimator. For any fixed positive-definite \(W\), its asymptotic covariance is
This expression is a sandwich: the middle matrix \(\Omega\) accounts for sampling variability, while \(W\) determines how the restrictions are combined. Substituting \(W=\Omega^{-1}\) simplifies the expression to
The simplification is not, by itself, a proof that this choice is best. To justify the efficiency claim, view the local estimator as a linear combination of the sample restrictions. Let \(L\) be any \(p\times q\) matrix satisfying \(LG=I_p\). Then the first-order estimation error has the form \(-Lr_n(\theta_0)\), and its asymptotic covariance is \(L\Omega L'\). The condition \(LG=I_p\) ensures that this combination responds correctly to local changes in the parameter.
Set \(L_*=(G'\Omega^{-1}G)^{-1}G'\Omega^{-1}\). Direct multiplication gives \(L_*G=I_p\).
For any \(L\) with \(LG=I_p\), write \(L=L_*+C\). Then \(CG=0\).
Because \(CG=0\), the cross terms vanish: \(L_*\Omega C'=0\) and \(C\Omega L_*'=0\). Thus \(L\Omega L'=L_*\Omega L_*'+C\Omega C'\), where \(C\Omega C'\) is positive semidefinite.
Therefore no other linear first-order combination satisfying \(LG=I_p\) has a smaller asymptotic covariance matrix than \(L_*\). Minimizing the distance with \(W=\Omega^{-1}\) produces exactly this combination. For a general \(W\), the corresponding matrix is \((G'WG)^{-1}G'W\). This establishes why inverse-covariance weighting is asymptotically efficient within the class of minimum-distance estimators based on these restrictions.
Worked Examples
Worked Example: Two Independent Estimates of One Parameter
Suppose two sample restrictions provide estimates \(m_1\) and \(m_2\) of a scalar parameter \(\theta_0\). Their restriction vector is \(r_n(\theta)=(m_1-\theta,\ m_2-\theta)'\). Assume the covariance of \(\sqrt{n}(m_1-\theta_0,\ m_2-\theta_0)'\) is \(\Omega=\begin{pmatrix}1&0\\0&4\end{pmatrix}\). In a hypothetical sample, let \(m_1=10\), \(m_2=13\), and \(n=100\). Compare efficient weighting with equal weighting.
The first estimate has asymptotic variance 1 and the second has asymptotic variance 4. The efficient weight is \(\Omega^{-1}=\begin{pmatrix}1&0\\0&1/4\end{pmatrix}\). Minimizing the weighted squared discrepancies gives
Its asymptotic variance for \(\sqrt{n}(\hat{\theta}_{\mathrm{eff}}-\theta_0)\) is \((1+1/4)^{-1}=0.8000\). Thus its approximate standard error at \(n=100\) is \(\sqrt{0.8000/100}=0.08944\), in the units of \(\theta\). With equal weights, the estimate is \((10+13)/2=11.50\), and its asymptotic variance is \((1+4)/4=1.250\), giving an approximate standard error of \(\sqrt{1.250/100}=0.1118\). The efficient estimate places more weight on the more precise restriction; these calculations describe sampling precision, not which estimate is closer to the unknown parameter in this particular sample.
Worked Example: Accounting for Correlated Restrictions
Now suppose two restrictions again estimate the same scalar parameter, but their asymptotic covariance is \(\Omega=\begin{pmatrix}4&1\\1&9\end{pmatrix}\). For hypothetical sample values \(m_1=2.4\) and \(m_2=3.0\), calculate the efficient estimate and compare it with the equal-weight estimate.
The determinant of \(\Omega\) is \(4(9)-1(1)=35\), so
The restriction vector is \((m_1-\theta,\ m_2-\theta)'\). The coefficient vector for the efficient average is proportional to \(\Omega^{-1}(1,1)'=(8/35,3/35)'\). Normalizing the coefficients to sum to one gives \(8/11\) and \(3/11\). Therefore
The equal-weight estimate is \((2.4+3.0)/2=2.700\). The efficient estimate uses the covariance between restrictions as well as their separate variances: inverse-covariance weighting is not generally the same as assigning weights based only on the diagonal variances. For this example, the efficient asymptotic variance is \(( (1,1)\Omega^{-1}(1,1)' )^{-1}=35/11=3.182\), compared with \((4+9+2(1))/4=3.750\) for the equal-weight average.
Worked Example: Three Restrictions for Two Parameters
Consider the hypothetical linear restriction system \(m=A\theta+u\), where \(A=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}\), the covariance of \(\sqrt{n}u\) is \(\Omega=\operatorname{diag}(1,4,1)\), and the observed restriction targets are \(m=(2,5,8)'\). Compare efficient weighting with the identity weight.
With \(W=\Omega^{-1}=\operatorname{diag}(1,1/4,1)\), the weighted least-squares estimate is \((A'WA)^{-1}A'Wm\). The required products are
The determinant of \(A'WA\) is \(2(1.25)-1=1.5\), so its inverse is \(\frac{1}{1.5}\begin{pmatrix}1.25&-1\\-1&2\end{pmatrix}\). Consequently,
The efficient asymptotic covariance is \((A'\Omega^{-1}A)^{-1}=\begin{pmatrix}0.8333&-0.6667\\-0.6667&1.333\end{pmatrix}\). With \(W=I_3\), the estimate is \((A'A)^{-1}A'm=\frac{1}{3}\begin{pmatrix}2&-1\\-1&2\end{pmatrix}\begin{pmatrix}10\\13\end{pmatrix}=(2.333,\ 5.333)'\). Its sandwich covariance is \(\begin{pmatrix}1&-1\\-1&2\end{pmatrix}\). Both procedures use all three restrictions, but identity weighting gives the noisier second restriction too much influence and has larger marginal asymptotic variances for both parameters.
Feasible Weighting and Practical Interpretation
In applications, \(\Omega\) is usually unknown. A feasible procedure estimates it, often using preliminary consistent parameter estimates and an estimator of the covariance of the sample restrictions. If \(\hat{\Omega}\) is consistent for a positive-definite \(\Omega\), use \(\hat{\Omega}^{-1}\) as the weight. Under the stated regularity conditions, estimating the weight this way preserves the efficient first-order asymptotic distribution. The covariance estimator must reflect the sampling design and dependence in the restrictions; a poorly matched covariance estimate can produce an inappropriate weight.
There are two important qualifications. First, when \(q=p\) and the restrictions can be solved exactly, a nonsingular weight does not change that exact solution; weighting matters most when there are more restrictions than parameters. Second, if \(\Omega\) is singular or nearly singular, the ordinary inverse may not exist or may be numerically unstable. The positive-definite assumption used in the derivation then fails, and the analyst must address the redundant or nearly redundant restrictions rather than blindly invert the estimated covariance matrix.
Common Mistake
- Wrong reasoning: “The efficient weight is the covariance matrix itself, because restrictions with high variance should count more.”
- Correction: The efficient weight is the inverse covariance matrix. A restriction with greater variance is downweighted, and off-diagonal covariance affects how restrictions are combined. The covariance matrix still appears in the sandwich formula for a general weight; it is not itself the optimal weight.
Check Your Understanding
Use the assumptions and formulas in this tutorial to answer the following questions.
- What does the matrix \(\Omega\) describe, and why must it be positive definite for the inverse-weight formula used here?
- If one restriction has four times the asymptotic variance of another and the restrictions are uncorrelated, how does inverse-covariance weighting treat them relative to equal weighting?
- Why does the covariance formula for a nonoptimal weight contain \(\Omega\) between two factors involving \(W\)?
- Under what condition does the weight not alter an exactly solved minimum-distance estimate?
- Explain why an estimated covariance matrix used for weighting must account for dependence among the sample restrictions.