Why an Interaction Can Survive Without Its Main Effects
An interaction allows the association between one predictor and the response to depend on another predictor. In a lasso model, each coefficient is penalized separately. Consequently, a fitted model can retain an interaction while setting one or both component main-effect coefficients to zero. This is allowed by the ordinary lasso, although it may impose a restrictive pattern on the fitted relationship.
For example, suppose a hypothetical insurer models a claim-related outcome using a continuous risk measure and a binary indicator. The risk measure’s slope could differ between the two indicator groups. A lasso fit might keep the product of those predictors while dropping their individual terms. The resulting interpretation depends on how the predictors and their product were constructed.
For a squared-error lasso with an unpenalized intercept, one common objective is
Here \(n\) is the number of observations, \(y_i\) is the response, \(x_{ij}\) is predictor \(j\) for observation \(i\), and \(\lambda\) controls the penalty. The intercept is not included in the penalty. If the interaction is one of the predictors, its coefficient is penalized just like any other slope coefficient. The objective does not require the component main effects to remain in the model.
What an Interaction-Only Selection Implies
With an uncentered continuous predictor \(x\) and a binary indicator \(A\), the model
has slope \(\widehat{\beta}_x\) when \(A=0\) and slope \(\widehat{\beta}_x+\widehat{\beta}_{xA}\) when \(A=1\). The coefficient \(\widehat{\beta}_A\) is the difference between the groups at \(x=0\), not an overall, constant group difference at every value of \(x\). If the interaction is retained but \(\widehat{\beta}_A=0\), the fitted group difference is zero at \(x=0\) and changes with \(x\).
Centering changes the reference point. If \(x_c=x-c\), then the coefficient of \(A\) in a model using \(x_c\) is the group difference at \(x=c\). Thus, “the main effect is zero” has a precise meaning only after specifying the basis used to construct the model matrix.
Worked Example: Interpreting a Selected Interaction After Centering
A hypothetical model uses \(x_c=x-5\) and a binary indicator \(A\). At one chosen lasso penalty, the fitted model is
Calculate predictions for \(x=5\) and \(x=7\) in both groups. Then interpret the coefficients and expand the equation in terms of \(x\) and \(A\).
Step 1: Calculate the centered predictor. When \(x=5\), \(x_c=5-5=0\). When \(x=7\), \(x_c=7-5=2\).
Step 2: Calculate the predictions. For \(A=0\), the interaction and \(A\) terms are zero. The predictions are \(10+2(0)=10\) at \(x=5\), and \(10+2(2)=14\) at \(x=7\). For \(A=1\), the predictions are \(10+2(0)+1.5(1)(0)=10\) at \(x=5\), and \(10+2(2)+1.5(1)(2)=10+4+3=17\) at \(x=7\).
Step 3: Interpret the slopes and group differences. The slope for \(A=0\) is \(2\). For \(A=1\), it is \(2+1.5=3.5\). The group difference is \(0\) at \(x=5\), then \(17-14=3\) at \(x=7\). This agrees with the interaction coefficient: the difference changes by \(1.5\) for each one-unit increase in \(x\).
Step 4: Expand carefully. Substituting \(x_c=x-5\) gives
The centered-model coefficient of \(A\) is zero, but the equivalent uncentered expression contains \(-7.5A\). This is not a contradiction: the coefficient of \(A\) in the centered basis describes the group difference at \(x=5\), whereas the coefficient in the uncentered basis describes it at \(x=0\). Method and mistake avoided: compute predictions in the fitted basis first, then expand algebraically. Do not treat a zero centered main-effect coefficient as a zero group difference at every \(x\).
Why Lasso Can Set a Main Effect to Zero
The lasso trades fit against the sum of absolute coefficient values. A coefficient is set to zero when the improvement in fit from retaining it does not justify its contribution to the penalty. There is no special exception for a main effect whose product term is retained.
A short derivation makes the selection rule concrete. Suppose the predictors are centered, mutually orthogonal, and scaled so that each has mean squared value one. For a single coefficient \(b\), after holding the other coefficients fixed, the part of the objective that depends on \(b\) can be written, up to an additive constant, as
where \(r_i\) is the response after removing the intercept and the contributions of the other predictors. For \(b>0\), differentiating gives \(b-c+\lambda=0\), so a positive solution is \(b=c-\lambda\), which is possible only when \(c>\lambda\). For \(b<0\), the derivative condition gives \(b-c-\lambda=0\), so a negative solution is \(b=c+\lambda\), possible only when \(c<-\lambda\). At \(b=0\), the lasso optimality condition is \(|c|\le\lambda\). Together these cases give the soft-thresholding rule
This derivation applies to the stated orthogonal, scaled least-squares setting; in a general design, the coefficients are coupled through the predictors’ relationships, so this formula cannot be applied independently to each coefficient. It nevertheless shows why an interaction can survive: its association with the response may exceed the penalty threshold while the associations for its component main effects do not.
Worked Example: Selecting Only the Product in an Orthogonal Design
Consider four hypothetical observations with centered predictors \(x,z\), each taking values \(-1\) or \(1\), and an interaction \(w=xz\). Fit a lasso model with an unpenalized intercept and the three predictors \(x,z,w\). The response values are given below, and use \(\lambda=0.5\).
| \(x\) | \(z\) | \(w=xz\) | \(y\) |
|---|---|---|---|
| 1 | 1 | 1 | 1.6 |
| 1 | -1 | -1 | -0.8 |
| -1 | 1 | -1 | -1.0 |
| -1 | -1 | 1 | 0.2 |
Step 1: Check the design. Each predictor has mean zero and mean squared value one. The products across rows sum to zero for each pair: \(xz\) sums to \(1-1-1+1=0\), \(xw\) sums to \(1-1+1-1=0\), and \(zw\) sums to \(1+1-1-1=0\). Thus these columns are mutually orthogonal. The response mean is \((1.6-0.8-1.0+0.2)/4=0\), so the fitted intercept is zero.
Step 2: Calculate the correlations used in the soft-threshold rule. For \(x\), the products \(xy\) are \(1.6,-0.8,1.0,-0.2\), giving \(c_x=(1.6-0.8+1.0-0.2)/4=0.4\). For \(z\), the products \(zy\) are \(1.6,0.8,-1.0,-0.2\), giving \(c_z=(1.6+0.8-1.0-0.2)/4=0.3\). For \(w\), the products \(wy\) are \(1.6,0.8,1.0,0.2\), giving \(c_w=(1.6+0.8+1.0+0.2)/4=0.9\).
Step 3: Apply soft-thresholding. With \(\lambda=0.5\), \(\widehat{\beta}_x=0\) because \(0.4\le0.5\), and \(\widehat{\beta}_z=0\) because \(0.3\le0.5\). The interaction coefficient is \(\widehat{\beta}_w=0.9-0.5=0.4\). The fitted model is therefore \(\widehat{y}=0.4xz\).
Step 4: Check the fit and the selection conditions. The fitted values in row order are \(0.4,-0.4,-0.4,0.4\). The residuals are \(1.2,-0.4,-0.6,-0.2\), whose mean is zero. For the zero coefficients, the correlations satisfy \(|c_x|=0.4\le0.5\) and \(|c_z|=0.3\le0.5\). For the nonzero interaction, \(c_w-\widehat{\beta}_w=0.9-0.4=0.5=\lambda\), as required for a positive coefficient. The interaction is selected while both main effects are zero. Method and mistake avoided: use the threshold rule only after checking orthogonality and scaling. Applying it coefficient by coefficient to a nonorthogonal design would not generally give the lasso solution.
Binary Predictors and Hierarchy
When both predictors are binary indicators, the product identifies observations in which both indicators equal one. Under reference-cell coding, an interaction-only term can therefore express a departure specific to the joint group. It does not automatically imply separate shifts for either indicator alone.
Worked Example: A Joint-Group Departure With No Main Effects
Suppose a hypothetical lasso fit predicts an outcome measured in dollars using two binary indicators \(X\) and \(Z\). The fitted equation is
Calculate the fitted value in each of the four groups and interpret the interaction.
Step 1: Evaluate the reference group. When \(X=0,Z=0\), the product is zero, so \(\widehat{y}=40\).
Step 2: Evaluate the groups with only one indicator equal to one. When \(X=1,Z=0\), \(XZ=0\), giving \(40+0+0+0=40\). When \(X=0,Z=1\), \(XZ=0\), also giving \(40\).
Step 3: Evaluate the joint group. When \(X=1,Z=1\), \(XZ=1\), so \(\widehat{y}=40+12=52\).
| \(X\) | \(Z\) | Fitted value |
|---|---|---|
| 0 | 0 | 40 |
| 1 | 0 | 40 |
| 0 | 1 | 40 |
| 1 | 1 | 52 |
The difference from the reference group is \(12\) only when both indicators equal one. The difference-in-differences is \((52-40)-(40-40)=12\). Method and mistake avoided: evaluate all four indicator combinations. Do not describe the interaction as a constant effect of \(X\) or \(Z\) on its own.
A common modeling principle called hierarchy recommends retaining lower-order terms when an interaction is included. Strong hierarchy requires both component main effects to be present; weak hierarchy requires at least one. These principles can make interpretation more stable, but ordinary lasso does not enforce either one. A lasso model that violates hierarchy is not automatically an invalid computation; it is a model whose constraints and interpretation should be stated clearly.
The selected terms can also depend on choices made before fitting. Centering a continuous predictor changes the reference value for its main effect. Changing factor coding changes the meanings of coefficients. Scaling affects the penalty applied relative to a coefficient’s units, so interaction terms should be treated consistently with the chosen scaling procedure. Although centering and coding changes can represent the same set of fitted relationships when all terms are present, the lasso penalty is not generally unchanged by such reparameterizations. A different representation can therefore produce a different sparse selection.
Interpreting and Reporting the Fit
When reporting a lasso model with an interaction, identify the response, the predictors used to construct the product, the coding or centering convention, and the penalty at which the terms were selected. Then describe the fitted relationship conditionally. For a continuous-by-binary interaction, report the slope in each group and the group difference at a stated reference value. For two binary indicators, report fitted values for the four combinations or state the joint-group departure relative to the reference coding.
A zero coefficient means the fitted model has set that coefficient to zero under the specified penalized objective; it does not prove that the corresponding relationship is absent in the population. Likewise, retaining an interaction is a predictive model choice, not evidence by itself of a causal mechanism. These distinctions matter when communicating results to decision-makers. This treatment of interaction selection is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each problem, interpret the fitted terms in the specified coding.
- A model is \(\widehat{y}=20+3x+0z+2xz\), where \(z\) is an indicator. Calculate predictions at \(x=2\) for \(z=0\) and \(z=1\), and give the slope in each group. (Answer: Predictions \(26\) for \(z=0\) and \(30\) for \(z=1\); slopes \(3\) and \(5\), respectively.)
- A model uses \(t=x-10\) and is \(\widehat{y}=25+1.2t+0A-0.4At\). Calculate predictions for \(x=15\) in both \(A\) groups and state where the zero \(A\) coefficient applies. (Answer: \(31\) for \(A=0\), \(29\) for \(A=1\); the zero group difference applies at \(x=10\).)
- In a centered, standardized orthogonal design, the correlations for the \(x\), \(z\), and \(xz\) columns are \(-0.2, 0.55,\) and \(0.8\). With \(\lambda=0.6\), give the lasso coefficients. (Answer: \(\widehat{\beta}_x=0,\ \widehat{\beta}_z=0,\ \widehat{\beta}_{xz}=0.2\).)
- With binary indicators, a fitted model is \(\widehat{y}=70+0X+0Z-9XZ\). Give all four fitted values and the difference-in-differences. (Answer: \(70,70,70,61\) for \((X,Z)=(0,0),(1,0),(0,1),(1,1)\); difference-in-differences \(=-9\).)
- A model uses \(x_c=x-4\) and is \(\widehat{y}=8+2x_c+0A+3Ax_c\). What are the slopes for \(A=0\) and \(A=1\), and at what value of \(x\) does the model impose no group difference? (Answer: Slopes \(2\) and \(5\), respectively; no group difference at \(x=4\).)