Asymptotic Normality of Debiased Lasso

Why Debiasing Can Support Inference

In a high-dimensional regression, a Lasso estimate is regularized to control estimation error. That regularization generally introduces bias, so the usual standard error for an ordinary least-squares coefficient cannot simply be attached to the selected Lasso coefficient. The debiased Lasso adds a correction based on the regression residuals. Its key inferential property is that the corrected estimate can be written as a leading score term plus a remainder.

The score term behaves like an average of mean-zero errors and can satisfy a central limit theorem. The remainder captures imperfections in the initial Lasso estimate and in the approximate inverse of the design covariance matrix. Asymptotic normality follows when that remainder is negligible relative to the score's \(n^{-1/2}\) scale.

Model and Decomposition

Consider the linear model with \(n\) observations and \(p\) regressors, where \(p\) may exceed \(n\):

Here \(y\) is the \(n\)-vector of outcomes, \(X\) is the \(n\times p\) design matrix, \(\beta^0\) is the target coefficient vector, and \(\varepsilon\) is the error vector. Let \(\widehat\beta\) be a Lasso estimate and let \(\widehat\Theta\) be an approximate inverse of \(\widehat\Sigma\), such as one constructed using nodewise Lasso. The debiased estimate is

For coefficient \(j\), write \(\widehat\theta_j'\) for row \(j\) of \(\widehat\Theta\), and \(e_j\) for the \(j\)th coordinate vector. Substituting \(y=X\beta^0+\varepsilon\) into the estimator and collecting terms gives the exact identity

The first term is a weighted sum of errors. The second term is a product: the approximate-inverse error, measured coordinatewise, multiplied by the Lasso estimation error. This distinction matters. Debiasing does not make the remainder exactly zero in general; it makes it small under suitable conditions.

Conditions for a Negligible Remainder

A useful sufficient set of conditions makes explicit the roles of sparsity and approximate inversion. Suppose the true coefficient vector has at most \(s\) nonzero entries, and the Lasso estimation error obeys

Suppose also that the approximate inverse satisfies

These are high-dimensional estimation rates, not automatic properties of every design or tuning choice. They typically rely on suitable design conditions and penalty levels. Applying the inequality \(|a'b|\leq\|a\|_\infty\|b\|_1\) gives

Consequently, the sufficient sparsity-rate condition \(s\log p/\sqrt n\to0\) implies \(\sqrt n R_j=o_p(1)\). In applications with different estimation rates, the same product argument applies: it is the product of the two relevant errors, not sparsity in isolation, that must be negligible at the \(n^{-1/2}\) scale.

Normal Approximation and Standard Error

For a precise statement, treat \(X\) as fixed (or condition on \(X\)). Assume the errors are independent, have conditional mean zero and common conditional variance \(\sigma^2\), and have sufficiently controlled tails for a central limit theorem. Let \(a_i=x_i'\widehat\theta_j\), where \(x_i'\) is row \(i\) of \(X\). The scaled leading score is

Its conditional variance is \(\sigma^2\widehat\theta_j'\widehat\Sigma\widehat\theta_j\). A sufficient central limit condition is that this variance stays bounded away from zero and infinity and that no single weighted error dominates the sum. For example, for some \(\delta>0\), one may require the conditional Lyapunov ratio

Under these conditions the normalized leading score converges to \(N(0,1)\). If, in addition, \(\sqrt nR_j=o_p(1)\), then adding the remainder does not change that limiting distribution. This last step follows from the fact that adding a term converging in probability to zero leaves a weak limit unchanged.

The estimated standard error for the coefficient itself is therefore

The variance estimator must match the error assumptions. The formula shown uses homoskedastic errors. If errors are heteroskedastic, the leading score variance must instead account for observation-specific error variances, and the standard error must be constructed accordingly; the homoskedastic formula should not be used without justification.

Constructing a Coefficient Confidence Interval

When the normal approximation applies, an approximate two-sided \(100(1-\alpha)\%\) confidence interval for \(\beta^0_j\) is the estimate plus and minus the standard normal critical value times its estimated standard error:

The interval has repeated-sampling coverage approximately \(1-\alpha\) under the model and asymptotic conditions. It is not a statement that the fixed coefficient has a \(1-\alpha\) probability of lying in this particular realized interval.

Common Pitfall and Practical Checks

A common mistake is to see a bias-corrected estimate and conclude that the Lasso remainder must be zero, so any normal-theory standard error is automatically valid. The correction does not guarantee exact inversion: the remainder is controlled only when the product of the approximate-inverse error and coefficient-estimation error is small enough. Nor does a small remainder alone establish normality; the weighted errors must also satisfy a central limit condition, and their variance must be consistently estimated.

Before reporting an interval, check that the model and design justify the required Lasso rates, that the approximate inverse has the needed coordinatewise accuracy, and that the target coordinate's leading-score variance is nondegenerate. Also consider whether the error variance assumptions match the standard error formula. A reported numerical interval cannot repair a failure of these underlying conditions.