Two Rules for Choosing a Penalty
A cross-validation curve may have a clear minimum, or it may be nearly flat across a range of penalty values. Choosing only the minimum can favor a more complicated fit for a very small apparent reduction in error. The one-standard-error rule offers a different choice: it accepts penalties whose estimated error is close enough to the minimum, then selects the largest such penalty. In many penalized fits, that choice produces more shrinkage and a more compact model.
In R, a fitted object from cv.glmnet contains both choices as lambda.min and lambda.1se. These are two selection rules applied to the same cross-validation results; neither is a universal guarantee of best future performance. The appropriate choice depends on the modeling objective and whether a small estimated error difference justifies a less parsimonious fit.
The word “largest” refers to the numerical value of lambda, not to the error. A larger lambda applies a stronger penalty. The rule therefore finds the strongest penalty that remains within the specified error allowance. Its use presumes that the errors on the curve are comparable: they must come from the same outcome, loss measure, and cross-validation setup. It does not require that the curve be smooth or that its minimum be unique.
How to Read the One-Standard-Error Cutoff
For cv.glmnet, use the error estimate and estimated standard error at the minimum-error penalty. Add that standard error to the minimum mean error to obtain a cutoff. Then compare each candidate penalty’s mean error with the cutoff. Do not add each candidate’s own standard error, and do not use the standard error at the largest penalty.
This follows directly from the rule’s two decisions. First, the minimum observed mean error is the reference value. The rule allows a candidate to be higher than that reference by at most one estimated standard error, so its permissible mean error is the reference plus that allowance. Second, among candidates meeting the allowance, the rule chooses the largest lambda. The formula expresses those two steps; it does not assert that every qualifying model has the same true prediction error.
The cutoff is a practical selection threshold, not a confidence bound on the true prediction error. Cross-validation errors at different lambda values are calculated using related folds and are not independent estimates. Consequently, “within one standard error” should not be read as a formal hypothesis test that the candidate and minimum-error models are statistically indistinguishable.
Worked Example: Calculating Both Choices
A hypothetical insurer is fitting a penalized regression model to predict a continuous claim-cost measure. The analyst uses one fixed cross-validation run and records mean squared error. The relevant portion of the curve is:
| Lambda | Mean cross-validation error |
|---|---|
| 0.80 | 0.449 |
| 0.50 | 0.426 |
| 0.30 | 0.421 |
| 0.20 | 0.412 |
| 0.10 | 0.415 |
| 0.05 | 0.423 |
At lambda \(=0.20\), the minimum mean error is \(0.412\), and cv.glmnet reports an estimated standard error of \(0.018\) at that minimum. Find both choices.
Step 1: Find lambda.min. The smallest listed mean error is \(0.412\), at lambda \(=0.20\). Thus lambda.min is \(0.20\).
Step 2: Calculate the cutoff. Add the standard error at the minimum: \(0.412 + 0.018 = 0.430\).
Step 3: Check eligibility and choose the largest eligible penalty. At lambda \(=0.80\), the mean error \(0.449\) exceeds \(0.430\). At \(0.50\), \(0.426 \leq 0.430\); at \(0.30\), \(0.421 \leq 0.430\); at \(0.20\), \(0.412 \leq 0.430\); and at \(0.10\) and \(0.05\), the errors \(0.415\) and \(0.423\) also meet the cutoff. The largest qualifying lambda is \(0.50\).
Therefore lambda.1se is \(0.50\), while lambda.min is \(0.20\). The rule chooses the stronger penalty even though its estimated error is \(0.426\), rather than the minimum \(0.412\). The calculation checks: \(0.426 - 0.412 = 0.014\), which is no greater than the allowance \(0.018\).
Method and mistake avoided. The method is to compute one cutoff from the minimum and its standard error, then screen the mean errors. A common mistake would be to use a candidate’s own standard error to create a different cutoff for each row; that is not the lambda.1se rule.
Why Parsimony Can Be a Reasonable Choice
The minimum-error rule chooses the best point estimate on the observed cross-validation curve. But the minimum can be only slightly better than nearby, more strongly penalized fits, and fold-to-fold variation makes small differences uncertain. If a stronger penalty gives nearly the same estimated error while shrinking more coefficients, the simpler fit may be preferable for practical reasons such as easier review or fewer retained predictors.
This is a preference, not a theorem that the one-standard-error model will predict better. The number of nonzero coefficients often decreases as lambda increases in a lasso fit, but lambda.1se does not directly minimize the number of predictors. For other penalized fits, or depending on the predictor relationships and path, coefficient patterns need not translate into a simple, guaranteed reduction in model complexity. Inspect the fitted model if parsimony is important.
Worked Example: Applying the Cutoff Carefully
For a hypothetical model of monthly service costs, suppose the minimum mean cross-validation error is \(0.310\) at lambda \(=0.20\), with an estimated standard error of \(0.015\). The following candidates have these mean errors: lambda \(=0.90\), \(0.350\); \(0.70\), \(0.326\); \(0.50\), \(0.324\); \(0.30\), \(0.319\); \(0.20\), \(0.310\); and \(0.10\), \(0.314\). Find lambda.1se.
Step 1: Calculate the one-standard-error cutoff. \(0.310 + 0.015 = 0.325\).
Step 2: Compare each candidate’s mean error to \(0.325\). The errors at \(0.90\) and \(0.70\) are \(0.350\) and \(0.326\), both greater than the cutoff. The errors at \(0.50\), \(0.30\), \(0.20\), and \(0.10\) are \(0.324\), \(0.319\), \(0.310\), and \(0.314\), all no greater than the cutoff.
Step 3: Select the largest qualifying lambda. Among the qualifying values, \(0.50\) is largest, so lambda.1se is \(0.50\). Lambda \(=0.70\) does not qualify because \(0.326 > 0.325\), while lambda \(=0.50\) does because \(0.324 \leq 0.325\).
The minimum-error choice remains \(0.20\). The one-standard-error choice is \(0.50\), whose mean error is \(0.014\) above the minimum, within the \(0.015\) allowance. The boundary comparison is important: a value just above the cutoff is not eligible, even if it is close.
Method and mistake avoided. Use the exact cutoff and include candidates equal to it. A frequent error is to say that neither \(0.70\) nor \(0.50\) qualifies because both are near \(0.325\). In fact, \(0.324 < 0.325\), so \(0.50\) qualifies; only \(0.326\) exceeds the cutoff.
Using the Choices in R
With a fitted cv.glmnet object named cvfit, the selected penalties are available as cvfit$lambda.min and cvfit$lambda.1se. To obtain fitted values or coefficients at one of them, specify the choice explicitly, for example predict(cvfit, newx = xnew, s = "lambda.1se"). The lambda value is selected from the fitted cross-validation results; the prediction call uses that choice to produce predictions for the supplied matrix.
A sound comparison keeps the underlying cross-validation results fixed. If the response, folds, loss measure, or model specification changes, the curve and its choices may change too. When comparing the two rules, report which one was used and, where useful, show both the estimated error and the model’s practical complexity. That makes the trade-off visible rather than presenting lambda.1se as automatically superior.
Worked Example: Relating the Rule to Model Size
A hypothetical provider predicts a binary renewal outcome using penalized logistic regression. In one cross-validation run, the minimum mean error is \(0.084\) at lambda \(=0.12\), and the estimated standard error at that minimum is \(0.006\). The fitted model at lambda \(=0.12\) has 18 nonzero predictor coefficients; the model at \(0.35\) has 9; and the model at \(0.60\) has 5. Their mean errors are \(0.084\), \(0.089\), and \(0.093\), respectively. Determine lambda.1se and interpret the available size information.
Step 1: Calculate the cutoff. The cutoff is \(0.084 + 0.006 = 0.090\).
Step 2: Check the candidate errors. Lambda \(=0.35\) qualifies because \(0.089 \leq 0.090\). Lambda \(=0.60\) does not qualify because \(0.093 > 0.090\). Lambda \(=0.12\) qualifies because it is the minimum-error choice.
Step 3: Choose the largest qualifying penalty. The eligible penalties among those given are \(0.35\) and \(0.12\); therefore lambda.1se is \(0.35\). Its estimated error is \(0.089 - 0.084 = 0.005\) above the minimum, which is within the \(0.006\) allowance.
At these stated penalties, the \(0.35\) fit has 9 nonzero coefficients, compared with 18 at lambda.min. Thus it is a plausible parsimony choice if the small difference in estimated error is acceptable for the application. The \(0.60\) fit is smaller, but its mean error is outside the rule’s cutoff. These counts describe the supplied fits; the cutoff itself does not select a target number of predictors.
Method and mistake avoided. Apply the error rule first, then use model size to explain the practical trade-off. Do not choose the smallest model just because it is smallest; lambda.1se is defined by the error threshold, not by minimizing coefficient count.
Common Traps and a Practical Decision
- Using the wrong standard error. The allowance is the estimated standard error at lambda.min, not the standard error at a candidate lambda.
- Choosing the smallest qualifying lambda. Lambda.1se is the largest numerical lambda that qualifies, because the aim is to accept the strongest penalty within the allowance.
- Treating “within one SE” as a significance result. It is a tuning heuristic, not a confidence interval, p-value, or guarantee of equal future risk.
- Assuming the more complex fit must be better. Lambda.min has the smallest observed mean error on this curve, but a small difference may be less compelling when compared with the reported uncertainty.
- Assuming lambda.1se always yields a particular model size. It selects a penalty by cross-validation error. The resulting number of nonzero coefficients should be checked separately.
A practical choice starts by identifying the purpose of the fit. If minimizing the observed cross-validation error is the priority, lambda.min is a direct choice. If a compact, more strongly penalized fit is valuable and its mean error is within one estimated standard error of the minimum, lambda.1se is a defensible alternative. State the rule used, quantify the error difference, and avoid claiming that the rule proves one model is best.
Check Your Understanding
For each problem, calculate the one-standard-error cutoff first, then apply the largest-eligible-lambda rule. Assume all listed candidates come from the same cross-validation fit and that lower mean error is better.
- A minimum mean error of \(0.274\) occurs at lambda \(=0.16\), with estimated standard error \(0.012\). Candidate errors are \(0.284\) at lambda \(=0.40\) and \(0.287\) at lambda \(=0.60\). Which candidate is lambda.1se? (Answer: The cutoff is \(0.286\); lambda.1se is \(0.40\), since \(0.284 \leq 0.286\) and \(0.287 > 0.286\).)
- The minimum error is \(0.205\) at lambda \(=0.08\), with estimated standard error \(0.009\). Errors are \(0.213\) at lambda \(=0.30\), \(0.215\) at \(0.45\), and \(0.220\) at \(0.60\). Find lambda.1se. (Answer: The cutoff is \(0.214\); lambda.1se is \(0.30\).)
- The minimum error is \(0.091\) at lambda \(=0.25\), with estimated standard error \(0.004\). Errors are \(0.094\) at lambda \(=0.40\), \(0.095\) at \(0.55\), and \(0.097\) at \(0.80\). Which penalty is selected? (Answer: The cutoff is \(0.095\); lambda.1se is \(0.55\), which is eligible at equality and is the largest qualifying value.)
- The minimum error is \(0.500\) at lambda \(=0.10\), with estimated standard error \(0.020\). At lambda \(=0.70\), the mean error is \(0.519\) and its own reported standard error is \(0.003\); at lambda \(=1.00\), mean error is \(0.521\). Find lambda.1se and identify the relevant standard error. (Answer: Use the standard error at the minimum, \(0.020\), giving a cutoff of \(0.520\); lambda.1se is \(0.70\), since \(0.519 \leq 0.520\) and \(0.521 > 0.520\).)
- The minimum error is \(0.132\) at lambda \(=0.12\), with estimated standard error \(0.010\). A model at lambda \(=0.40\) has error \(0.140\) and 7 nonzero coefficients; a model at \(0.65\) has error \(0.143\) and 4 nonzero coefficients. What is lambda.1se, and why is the smaller model not selected by this rule? (Answer: The cutoff is \(0.142\), so lambda.1se is \(0.40\); the \(0.65\) model is more compact but its error \(0.143\) exceeds the cutoff.)