Why Compare Feasible and Infeasible Estimators?
An estimator may depend on a population quantity that is unknown in practice. Replacing that quantity with an estimate makes the procedure feasible, but the substitution can also change its sampling error. The key question is not simply whether the replacement is consistent. It is whether the resulting difference is small compared with the scale at which the estimator’s limiting distribution is studied.
For example, if an estimator has error of order \(n^{-1/2}\), an added error that is merely \(o_p(1)\) may still be large enough to change its root-\(n\) limit. By contrast, an added error of order \(o_p(n^{-1/2})\) is negligible at that scale. The distinction determines whether the infeasible estimator can be used to describe the feasible estimator’s first-order behavior.
The notation \(\xrightarrow{p}\) denotes convergence in probability. The usual rate for a root-\(n\) estimator is \(r_n=\sqrt n\). Equivalence at this rate allows the feasible and infeasible estimators to share the same first-order limit, provided the infeasible estimator has a limit at that rate. It does not, by itself, establish a limit distribution: that still requires the relevant limit result for the infeasible estimator.
A Rate Condition for Equivalence
Write an estimator as a function of the nuisance quantity, \(\theta_n(\eta)\). The infeasible version is \(\theta_n(\eta_0)\); the feasible version is \(\theta_n(\hat\eta_n)\). Suppose the function is differentiable in \(\eta\) near \(\eta_0\). For a scalar nuisance quantity, a first-order expansion gives
where \(R_n\) is a remainder term. For a vector nuisance quantity, the derivative is a row vector and its product with \(\hat\eta_n-\eta_0\) replaces the scalar product. The expansion is useful only if the derivative and remainder are controlled at the scale of interest.
The implication follows by multiplying the expansion by \(r_n\). A derivative bounded in probability times a term converging to zero in probability converges to zero in probability; the scaled remainder does too. For root-\(n\) equivalence, this sufficient condition is \(\sqrt n(\hat\eta_n-\eta_0)\xrightarrow{p}0\). A nuisance estimate that is only root-\(n\) consistent generally does not meet it.
Worked Example: A Faster Nuisance Estimate Gives Equivalence
Suppose the infeasible estimator is \(\tilde\theta_n=\bar Y_n-2\eta_0\), and the feasible estimator is \(\hat\theta_n=\bar Y_n-2\hat\eta_n\). Here \(\bar Y_n\) is the sample mean of an outcome measured in units, and \(\eta_0\) and \(\hat\eta_n\) are measured in the same units. Assume \(\sqrt n(\tilde\theta_n-\theta_0)=O_p(1)\). To make the nuisance rate explicit, suppose \(\hat\eta_n\) is the mean of an independent calibration sample of size \(m=n^2\), with mean \(\eta_0\) and finite variance \(\tau^2\).
The two estimators differ by $$ \hat\theta_n-\tilde\theta_n=-2(\hat\eta_n-\eta_0). $$ The calibration-sample mean has variance \(\tau^2/m=\tau^2/n^2\). Therefore \(\hat\eta_n-\eta_0=O_p(n^{-1})\), and $$ \sqrt n(\hat\theta_n-\tilde\theta_n) =-2\sqrt n(\hat\eta_n-\eta_0) =O_p(n^{-1/2})\xrightarrow{p}0. $$ So the feasible estimator is root-\(n\) equivalent to the infeasible estimator. This conclusion is about their difference; the assumed root-\(n\) behavior of the infeasible estimator supplies the separate condition needed to describe their common first-order limit.
For a concrete hypothetical realization, let \(n=400\) and suppose \(\hat\eta_n-\eta_0=0.0015\) units. Then \(\hat\theta_n-\tilde\theta_n=-2(0.0015)=-0.0030\) units, and the root-\(n\)-scaled difference is \(20(-0.0030)=-0.0600\) units on the scaled estimator scale. This finite-sample value illustrates the calculation; the equivalence result follows from the rate condition, not from this one realization.
When the Plug-In Error Changes the Limit
If the nuisance estimate converges at the same rate as the target estimator, its effect may remain visible after normalization. A first-order expansion can make that effect explicit. But a limiting distribution for the difference must be derived from joint behavior: marginal convergence of two random quantities alone does not determine the distribution of their difference.
Worked Example: Root-\(n\) Nuisance Error Is Not Negligible
Consider the scalar estimators $$ \tilde\theta_n=\theta_0+n^{-1/2}W_n, \qquad \hat\eta_n=\eta_0+n^{-1/2}Z_n, \qquad \hat\theta_n=\tilde\theta_n-2(\hat\eta_n-\eta_0). $$ Assume explicitly that the pair converges jointly: $$ \big(\sqrt n(\tilde\theta_n-\theta_0),\sqrt n(\hat\eta_n-\eta_0)\big) =(W_n,Z_n)\Rightarrow(W,Z), $$ where \(W\) and \(Z\) are independent standard normal random variables. Joint convergence and independence are assumptions here, not consequences of the two marginal limits.
The scaled feasible estimator satisfies $$ \sqrt n(\hat\theta_n-\theta_0)=W_n-2Z_n. $$ By joint convergence and continuity of the subtraction map, its limit is \(W-2Z\). Independence gives $$ \operatorname{Var}(W-2Z)=\operatorname{Var}(W)+4\operatorname{Var}(Z) =1+4=5, $$ so \(W-2Z\sim N(0,5)\). The infeasible estimator instead has root-\(n\) limit \(W\sim N(0,1)\). Also, $$ \sqrt n(\hat\theta_n-\tilde\theta_n)=-2Z_n\Rightarrow -2Z, $$ which does not converge in probability to zero. The estimators are therefore not root-\(n\) equivalent.
For a hypothetical realization with \(n=400\), \(W_n=0.6\), and \(Z_n=-0.3\), the oracle estimator’s scaled error is \(0.6\), while the feasible estimator’s scaled error is \(0.6-2(-0.3)=1.2\). This is consistent with the formula: the plug-in adjustment is not negligible on the root-\(n\) scale.
The joint-convergence condition matters even when both marginal limits are normal. Marginal convergence of \(W_n\) and \(Z_n\) does not imply that their difference converges to the difference of independent normal variables. If dependence persists in the limit, the variance of the difference includes a covariance term: $$ \operatorname{Var}(W-2Z) =\operatorname{Var}(W)+4\operatorname{Var}(Z)-4\operatorname{Cov}(W,Z). $$ Thus a claimed limiting distribution for a feasible estimator must reflect the actual joint limit, not an assumed independence that has not been established.
Zero First Derivatives and Second-Order Error
The rate condition based on the first derivative is sufficient, but it need not be necessary. If the derivative with respect to the nuisance quantity is zero at its true value, the leading plug-in effect may be second order. In that case, a nuisance estimate with the same rate as the target estimator can still yield equivalence.
Worked Example: A Zero First Derivative Makes the Error Second Order
Suppose the feasible estimator has the exact form \(\hat\theta_n=\tilde\theta_n+(\hat\eta_n-\eta_0)^2\). This adjustment is a function of the nuisance error whose first derivative at \(\eta_0\) is zero. Assume \(\sqrt n(\hat\eta_n-\eta_0)=O_p(1)\). Then $$ \sqrt n(\hat\theta_n-\tilde\theta_n) =\sqrt n(\hat\eta_n-\eta_0)^2 =\frac{\{\sqrt n(\hat\eta_n-\eta_0)\}^2}{\sqrt n} \xrightarrow{p}0. $$ The last step holds because the squared quantity in braces is \(O_p(1)\), while \(1/\sqrt n\) tends to zero. Hence the estimators are root-\(n\) equivalent even though the nuisance estimate is only root-\(n\) consistent.
For example, at \(n=400\), if the hypothetical scaled nuisance error is \(\sqrt n(\hat\eta_n-\eta_0)=1.5\), then \(\hat\eta_n-\eta_0=1.5/20=0.075\) units. The estimator difference is \(0.075^2=0.005625\) squared units, and its root-\(n\)-scaled value is \(20(0.005625)=0.1125\) in the correspondingly scaled units. The asymptotic argument says this scaled difference tends to zero as \(n\) grows; it does not say it is exactly zero at \(n=400\).
More generally, a second-order expansion may give a difference bounded in probability by a constant times \(\|\hat\eta_n-\eta_0\|^2\). For root-\(n\) equivalence, it is then sufficient that \(\sqrt n\|\hat\eta_n-\eta_0\|^2\xrightarrow{p}0\). For example, a nuisance error of order \(n^{-1/2}\) meets this condition because \(\sqrt n(n^{-1/2})^2=n^{-1/2}\to0\). A zero derivative alone is not enough: the remainder must also be controlled at the required scale.
How to Check Equivalence
A useful analysis separates three questions: what is the target estimator’s normalization, how sensitively does it depend on the nuisance quantity, and what rate does the nuisance estimate attain? The following steps keep these questions distinct.
Identify \(r_n\) from the estimator’s convergence rate, such as \(r_n=\sqrt n\) for a root-\(n\) estimator.
Write \(\hat\theta_n-\tilde\theta_n\) directly so that the source of the feasible estimator’s extra error is visible.
Determine whether the leading derivative term and the remainder both vanish after multiplication by \(r_n\). If deriving a limit rather than equivalence, state the joint convergence needed for the terms in that limit.
A nuisance estimate that is consistent, meaning \(\hat\eta_n-\eta_0\xrightarrow{p}0\), need not satisfy the rate condition for first-order equivalence. If the target is root-\(n\), an error of order \(n^{-1/4}\) is consistent but has root-\(n\)-scaled magnitude of order \(n^{1/4}\), which does not vanish. When the first derivative is nonzero, that plug-in error can dominate the target’s own root-\(n\) error. Conversely, if the first derivative is zero, a slower nuisance rate may suffice, but it must be checked against the actual order of the remainder.
Why the Distinction Matters
If feasible and infeasible estimators are equivalent at the target scale, a valid first-order approximation for the infeasible estimator also describes the feasible one. This can simplify analysis when the infeasible estimator is easier to expand, even though it cannot be computed because it uses an unknown population quantity. If equivalence fails, ignoring the plug-in term can give the wrong limiting variance or distribution, and therefore misleading uncertainty statements.
The appropriate conclusion is always tied to a specified rate. Two estimators can both converge in probability to the same parameter while failing to be equivalent at rate \(\sqrt n\). They can also have the same marginal limiting distributions without their difference converging to zero in probability. Equivalence requires control of the difference itself at the chosen scale.
Common Mistake
- Wrong reasoning: “The nuisance estimate is consistent, so replacing the unknown quantity cannot affect the estimator’s limiting distribution.”
- Correction: Consistency only says the unscaled nuisance error vanishes. For root-\(n\) equivalence with a nonzero first derivative, one typically needs \(\sqrt n(\hat\eta_n-\eta_0)\xrightarrow{p}0\), along with a negligible scaled remainder. If the nuisance error is root-\(n\), its contribution may remain in the limit.
- A second related error: Marginal limits for two terms do not justify calculating the distribution of their difference as if the limits were independent. Establish joint convergence and use the dependence in the joint limit.
Check Your Understanding
For each question, state the relevant scale or condition rather than relying only on consistency.
- For root-\(n\) equivalence and a nonzero bounded-in-probability derivative, what rate condition on \(\hat\eta_n-\eta_0\) is sufficient?
- If both \(\sqrt n(\tilde\theta_n-\theta_0)\) and \(\sqrt n(\hat\eta_n-\eta_0)\) converge marginally, what additional information is needed to derive a limit for their difference?
- A plug-in error is \(O_p(n^{-1/4})\) and enters linearly. Is it negligible at the root-\(n\) scale? Show the rate comparison.
- Why can a root-\(n\) nuisance estimate still produce root-\(n\) equivalence when the first derivative of the estimator with respect to the nuisance quantity is zero?
- Does consistency of both estimators for the same parameter imply their root-\(n\) equivalence? Explain using the definition.