Double Selection for Treatment Effects

Why Select Controls Twice?

In a treatment-effect regression with many potential controls, choosing controls only because they predict the outcome can miss a variable that predicts treatment strongly but has a modest outcome association. That variable may still confound the treatment comparison. Conversely, selecting controls only because they predict treatment can omit outcome predictors that improve adjustment. Double selection addresses both risks by combining controls selected from two separate lasso regressions, then estimating the treatment effect in an ordinary least-squares regression using their union.

The method is designed for settings with many candidate controls, where the number of controls may be large relative to the sample size, but where the relevant relationships are sparse or approximately sparse. The lasso steps are used to screen controls; the final treatment coefficient is estimated without an \(\ell_1\) penalty. This distinction matters: the final coefficient is not simply one of the lasso coefficients.

Model and Identifying Assumptions

Let \(Y_i\) be an outcome, \(D_i\) a treatment, and \(X_i=(X_{i1},\ldots,X_{ip})'\) a vector of observed controls for unit \(i\), with \(i=1,\ldots,n\). A standard partially linear model is

Here \(\alpha\) is the treatment effect of interest, \(\beta\) and \(\gamma\) describe control relationships, \(\varepsilon_i\) is an outcome disturbance, and \(v_i\) is the part of treatment not explained by the controls. The treatment effect is constant in this model. The second equation can be understood as a linear projection of treatment on controls, with \(E[v_i\mid X_i]=0\) under the exact specification.

For a causal interpretation, a typical sufficient identification framework includes consistency of the observed outcome with the outcome under the received treatment, no interference between units, and conditional exchangeability: conditional on \(X_i\), treatment assignment is independent of the potential outcomes. There must also be treatment variation at relevant values of the controls (positivity). In the partially linear formulation, these ideas are represented by a correctly specified constant-effect model and an outcome disturbance satisfying \(E[\varepsilon_i\mid D_i,X_i]=0\). Without assumptions that make treatment conditionally exogenous, the procedure estimates an adjusted association, not necessarily a causal effect.

High-dimensional validity requires more than identification. A common set of sufficient conditions includes independent sampling (or suitable weak dependence conditions), approximate sparsity of both the outcome and treatment relationships, adequate design conditions such as restricted eigenvalue or compatibility bounds, suitable moment and tail conditions, and nondegenerate residual treatment variation. Informally, approximate sparsity means that each relationship can be represented well using relatively few controls, even if \(p\) is large. One familiar scaling condition for exact sparsity is \(s\log(p)/\sqrt{n}\to0\), where \(s\) bounds the relevant sparsity; precise results impose additional regularity conditions. These conditions support asymptotic inference, not a guarantee that every selected set is exactly the true support.

The Two Lasso Steps and the Final Regression

Let \(\widehat S_Y\) be the controls selected by the outcome lasso, and \(\widehat S_D\) those selected by the treatment lasso. Each lasso includes a freely estimated, unpenalized intercept. For example, the outcome fit solves

The treatment fit is defined analogously, replacing \(Y_i,c,\lambda_Y\) with \(D_i,c,\lambda_D\). The intercept is not penalized, while the coefficients on controls are. Writing the intercept as a free parameter makes the objective valid whether or not the controls have been centered. Penalty choices may be based on theory or cross-validation; selection depends on both the penalty and the data.

The selected sets consist of variables with nonzero lasso coefficients: \( \widehat S_Y=\{j:\widehat\beta^Y_j\ne0\}\) and \( \widehat S_D=\{j:\widehat\beta^D_j\ne0\}\). Set \( \widehat S=\widehat S_Y\cup\widehat S_D\), removing duplicates. The final regression is

where the intercept and all final-regression coefficients are estimated by ordinary least squares. The controls in the union are not penalized in this last step. For a conventional final OLS fit, the selected design matrix must have full column rank and have fewer columns than observations; high-dimensional theory also requires that its effective size and estimation error remain controlled.

Why the Union Helps

The outcome lasso looks for controls useful for predicting \(Y\), while the treatment lasso looks for controls useful for predicting \(D\). Their targets need not coincide. In the partially linear model, the linear projection of \(Y\) on \(X\) has coefficient \(\beta+\alpha\gamma\), when the displayed equations hold and the projection conditions are satisfied. A control can therefore be relevant to the outcome projection, the treatment projection, or both. Taking the union is a safeguard against relying on just one of these routes to discover an important adjustment variable.

The basic omitted-variable calculation explains why a treatment predictor deserves attention. Suppose a control \(Z\) is left out, while other included regressors have already been accounted for. If the true relation is \(Y=\alpha D+\delta Z+\varepsilon\), and the residualized variables satisfy \(E[D\varepsilon]=0\), the coefficient on \(D\) from omitting \(Z\) is

Thus, omission creates bias when \(Z\) both affects the outcome (\(\delta\ne0\)) and is associated with treatment. Selection in the treatment regression can bring such a control into the final adjustment set even if the outcome lasso does not select it. This is the key intuition, not a claim that lasso always selects every confounder: the sparsity, signal, and design assumptions underpinning the method remain essential.

Worked Numerical Treatment-Effect Example

Consider the following hypothetical eight-unit data set. The controls \(X_1\) and \(X_2\) are candidate adjustment variables. To make both selection steps transparent, suppose the lasso penalty is \(\lambda_Y=\lambda_D=0.5\), both controls are centered and have squared length 8, and their cross-product is zero. The treatment values are generated as \(D=X_1+v\), and the outcomes as \(Y=2D+3X_2+0.5q\), where \(v\) and \(q\) are centered vectors orthogonal to \(X_1\) and \(X_2\). All entries below are hypothetical.

For centered, orthogonal controls with \(\sum_i X_{ij}^2/n=1\), a lasso coefficient is the soft-thresholded sample cross-product \(n^{-1}\sum_i X_{ij}R_i\): it is set to zero when the absolute cross-product is no larger than \(\lambda\), and otherwise reduced in magnitude by \(\lambda\). This follows by minimizing the one-coordinate quadratic loss plus the absolute-value penalty. The free intercept in each fit is zero here because the response and both controls have mean zero.

Outcome selection. The cross-products are \(n^{-1}\sum_iX_{i1}Y_i=2\) and \(n^{-1}\sum_iX_{i2}Y_i=3\). With \(\lambda_Y=0.5\), the fitted lasso coefficients are \(2-0.5=1.5\) and \(3-0.5=2.5\). Both are nonzero, so \(\widehat S_Y=\{X_1,X_2\}\).

Treatment selection. The cross-products are \(n^{-1}\sum_iX_{i1}D_i=1\) and \(n^{-1}\sum_iX_{i2}D_i=0\). With \(\lambda_D=0.5\), the fitted coefficients are \(1-0.5=0.5\) and 0. Thus \(\widehat S_D=\{X_1\}\), and the union is \(\widehat S=\{X_1,X_2\}\).

Final regression. The data were constructed so that \(Y_i=2D_i+3X_{i2}+0.5q_i\), with \(q\) orthogonal to the intercept, \(D\), \(X_1\), and \(X_2\). Therefore OLS of \(Y\) on an intercept, \(D\), \(X_1\), and \(X_2\) gives \(\widehat\alpha_{\mathrm{DS}}=2\), coefficient 0 on \(X_1\), and coefficient 3 on \(X_2\). The residual sum of squares is \(8(0.5)^2=2\). There are \(8-4=4\) residual degrees of freedom, so the residual variance estimate is \(s^2=2/4=0.5\).

After partialling out the intercept, \(X_1\), and \(X_2\), the residualized treatment is \(v\), whose squared length is 8. Hence the illustrative OLS standard error for the treatment coefficient is

The estimate is 2 outcome units per treatment unit, with an illustrative standard error of 0.25. The ratio is \(2/0.25=8\). A \(t_4\) reference calculation would give a two-sided p-value of about 0.0013. This tiny, constructed sample is for arithmetic practice, not a setting in which that p-value should be treated as exact post-selection inference. The method's inferential justification is asymptotic and depends on the high-dimensional conditions stated above; ordinary OLS calculations in a selected model do not automatically provide exact finite-sample inference.

Inference, Limitations, and a Common Pitfall

Under suitable sparsity, design, moment, and treatment-variation conditions, the double-selection coefficient is asymptotically centered at the target at the usual root-\(n\) scale, and a suitable standard error can support confidence intervals and tests. In practice, inference must be tied to a result whose assumptions match the sampling design and implementation. Double selection is not a license to use an unadjusted post-lasso OLS standard error in every finite sample. In particular, a selected set that is too large, severe collinearity, weak residual treatment variation, or poor approximation by sparse nuisance relationships can undermine the approximation.

Common mistake: “The outcome lasso did not select this control, so it cannot confound the treatment effect.” This reasoning ignores the treatment regression. A control may be important because it predicts treatment, even if it adds little to outcome prediction by itself. The correction is to estimate both lasso regressions and use \(\widehat S_Y\cup\widehat S_D\) in the final regression. A related implementation error is to fix the lasso intercept at the response mean while leaving controls uncentered. The correct objective estimates an unpenalized intercept jointly with the penalized slopes, as written above.