Why Search Learning Rate and Depth Together?
A boosted-tree model is controlled by several choices. The learning rate scales each tree's contribution to the current model, while tree depth limits how complex an individual tree can be. A small learning rate makes each update more gradual; a greater depth permits a tree to represent more complex predictor relationships. Neither setting is automatically best: the useful combination depends on the data and the prediction objective.
A grid search evaluates a prespecified set of combinations. Its value is not merely that it produces a winning pair. It makes the comparison structured: every candidate is fitted under the same data partition and other specified settings, then scored with the same validation metric. The result is evidence for choosing among the candidates in that grid, not proof that the selected values are universally optimal.
Depth conventions can differ across software. In this tutorial, \(d\) denotes the maximum number of edges from a tree's root to a leaf: depth 1 permits a stump, while depth 2 permits another level of splits. Always check the software's definition before interpreting a depth value. The learning rate is denoted by \(\eta\), and \(M\) denotes the fixed number of boosting iterations in the grid comparison.
Fixing \(M\) makes this a controlled comparison of learning rate and depth at that iteration count. It does not establish that the same pair remains best at every possible iteration count. Other controls—such as the loss function, minimum observations per leaf, row-sampling setting, and feature set—should also be held fixed unless they are explicitly part of the search.
Constructing and Comparing the Grid
Choose a modest set of plausible learning rates and depths based on the problem and the computational budget. A grid with three learning rates and two depths contains \(3\times2=6\) candidates. Increasing the number of values can make the search more detailed, but also increases fitting time and the number of comparisons made against the validation data.
For a quantitative response, one common metric is root mean squared error (RMSE). If the validation set contains \(n\) observations, actual responses \(y_i\), and candidate predictions \(\hat y_i\), then:
The response and predictions must be on the same scale. For example, if the model was trained to predict a transformed response, compute the metric on the intended evaluation scale only after converting predictions appropriately. This tutorial's examples use an untransformed quantitative response.
Why a Shared Validation Set Gives a Fair Comparison
Suppose two candidates are evaluated on the same \(n\) validation observations and RMSE is the selected metric. Because the square-root function is increasing, the candidate with the smaller sum of squared errors also has the smaller RMSE:
This equivalence holds because both candidates use the same positive denominator \(n\), and taking a square root preserves order. If candidates are scored on different observations, the comparison no longer isolates model performance: a candidate may appear better simply because it faced easier cases. Using the same validation observations, metric, and response scale makes the comparison interpretable.
Worked Example: Select a Candidate From Validation Errors
A hypothetical insurer is developing a model to predict a claim's paid amount in hundreds of dollars. Four candidate boosted models are evaluated on the same four validation claims. Their signed residuals, defined as actual amount minus predicted amount, are shown below. Select the candidate with the smallest validation RMSE.
| Candidate | Learning rate | Depth | Validation residuals |
|---|---|---|---|
| A | 0.03 | 1 | 2, -2, 1, -1 |
| B | 0.03 | 2 | 1, -1, 1, -1 |
| C | 0.10 | 1 | 0.5, -1.5, 1, -1 |
| D | 0.10 | 2 | 0, -1, 1, -1 |
Method. Square each residual, add the four squared errors, divide by \(n=4\), and take the square root. The common mistake to avoid is averaging signed residuals: positive and negative errors can cancel even when predictions are inaccurate.
For A, the squared residuals are \(2^2=4\), \((-2)^2=4\), \(1^2=1\), and \((-1)^2=1\). Their sum is \(10\), so its RMSE is \(\sqrt{10/4}=\sqrt{2.5}\approx1.5811\).
For B, the squared residuals are \(1,1,1,1\). Their sum is \(4\), so its RMSE is \(\sqrt{4/4}=1.0000\).
For C, the squared residuals are \(0.25,2.25,1,1\). Their sum is \(4.5\), so its RMSE is \(\sqrt{4.5/4}=\sqrt{1.125}\approx1.0607\).
For D, the squared residuals are \(0,1,1,1\). Their sum is \(3\), so its RMSE is \(\sqrt{3/4}\approx0.8660\). Thus D, with learning rate 0.10 and depth 2, has the lowest validation RMSE among these four candidates. The ranking is checked against the sums of squares: \(3<4<4.5<10\), which gives the same ordering.
The RMSE is in hundreds of dollars, so D's validation RMSE is approximately \(0.8660\times100=\$86.60\). This conversion is only a change of units; it does not change which candidate ranks best.
Fitting the Candidates in R
The following R pattern uses the gbm package for a Gaussian-response example. It illustrates the grid-search structure rather than the full range of package-specific fitting options. Here, train and valid are already separated data frames with a numeric response named loss and numeric predictors named exposure and score. The number of iterations, loss, leaf-size setting, and sampling control are fixed for all candidates.
First, specify the grid and a fixed iteration count: grid <- expand.grid(eta = c(0.03, 0.10), depth = c(1, 2)); M <- 150. The code below fits each pair on the same training data and calculates validation RMSE on the same validation rows. A fixed seed is included for reproducibility if the fitting procedure uses randomness.
R fitting loop: results <- data.frame(eta = grid$eta, depth = grid$depth, rmse = NA_real_); set.seed(412); for (j in seq_len(nrow(grid))) { fit <- gbm::gbm(loss ~ exposure + score, data = train, distribution = "gaussian", n.trees = M, interaction.depth = grid$depth[j], shrinkage = grid$eta[j], n.minobsinnode = 5, bag.fraction = 1, train.fraction = 1, verbose = FALSE); pred <- predict(fit, newdata = valid, n.trees = M, type = "response"); results$rmse[j] <- sqrt(mean((valid$loss - pred)^2)) }.
Inspect the results with results[order(results$rmse), ]. The first row has the smallest validation RMSE. Check that the response and predictions contain no missing values before interpreting the ranking; otherwise, missing values can make the metric undefined or change which observations are compared.
A grid result is only reproducible if the data split and relevant fitting controls are reproducible too. Keep a record of the candidate values, fixed settings, split, metric, and random seed. If any candidate is fitted on a different split, or receives a different number of iterations, record that change and do not describe the result as a comparison of learning rate and depth alone.
Worked Example: Aggregate Evidence Across Three Folds
A hypothetical portfolio team compares three learning-rate-and-depth pairs using three validation folds of equal size. Each entry below is a fold RMSE in the same units. The team has decided in advance to aggregate squared errors across folds and then take one square root. Which configuration has the lowest pooled RMSE?
| Candidate | Fold 1 RMSE | Fold 2 RMSE | Fold 3 RMSE |
|---|---|---|---|
| A: \(\eta=0.03,d=1\) | 3.0 | 4.0 | 5.0 |
| B: \(\eta=0.03,d=2\) | 2.8 | 4.2 | 4.0 |
| C: \(\eta=0.10,d=2\) | 3.1 | 3.3 | 3.5 |
Method. Because all folds have the same size, the pooled RMSE is the square root of the mean of the three fold mean squared errors. Each fold MSE is the square of its RMSE.
For A, the fold MSEs are \(3^2=9\), \(4^2=16\), and \(5^2=25\). The pooled RMSE is \(\sqrt{(9+16+25)/3}=\sqrt{50/3}\approx4.0825\).
For B, the fold MSEs are \(2.8^2=7.84\), \(4.2^2=17.64\), and \(4^2=16\). The pooled RMSE is \(\sqrt{(7.84+17.64+16)/3}=\sqrt{41.48/3}\approx3.7184\).
For C, the fold MSEs are \(3.1^2=9.61\), \(3.3^2=10.89\), and \(3.5^2=12.25\). The pooled RMSE is \(\sqrt{(9.61+10.89+12.25)/3}=\sqrt{32.75/3}\approx3.3030\). Candidate C has the smallest pooled RMSE and is selected under the team's stated rule.
The common mistake is to average fold RMSEs and call that the pooled RMSE. For C, the simple mean of fold RMSEs is \((3.1+3.3+3.5)/3=3.3000\), which is close here but is not the pooled value \(3.3030\). With unequal fold sizes, the pooled calculation must also account for the number of observations in each fold. Define the aggregation rule before comparing candidates.
Interpreting the Selected Configuration
The minimum validation score identifies a candidate for further consideration, not a guarantee of future performance. A search over many candidates can favor one that happens to fit quirks of the validation sample. The validation set has been used to choose among candidates, so its minimum score should not be reported as an unbiased final estimate of performance.
Keep a separate test set untouched during grid selection. After choosing the configuration and completing any permitted model-development decisions, evaluate the final procedure on that test set once for the final performance assessment. If the test results prompt another search or model change, the test set has become part of development rather than an independent final check.
Also distinguish a close ranking from a decisive one. If the top two candidates have nearly identical validation scores, the grid does not establish that their underlying predictive performance differs meaningfully. Consider whether the simpler or less costly configuration is adequate, and report the small score difference without overstating it. If fold rankings change substantially, that is evidence that the choice may depend on the particular partition.
Check Your Understanding
For each problem, use the stated comparison rule and check units and denominators before selecting a candidate.
- A hypothetical repair-cost model has validation residuals, in thousands of dollars, of \(1,-1,2\) for candidate A and \(0,2,-2\) for candidate B. Calculate both RMSEs to four decimal places and identify the smaller one. (Answer: A has RMSE \(\sqrt{2}=1.4142\); B has RMSE \(\sqrt{8/3}=1.6330\), so A is smaller.)
- A grid has four learning rates and three depths. All other settings are fixed. How many candidates are fitted, and how many validation RMSE values are needed if each candidate is scored once? (Answer: 12 candidates and 12 validation RMSE values.)
- Two candidates have validation SSE values of 90 and 100 on the same 25 observations. Calculate their RMSEs to four decimal places and state which is selected by RMSE. (Answer: \(\sqrt{90/25}=1.8974\) and \(\sqrt{100/25}=2.0000\); select the candidate with SSE 90.)
- Three equal-sized validation folds give one candidate RMSEs of 2, 3, and 4. Using pooled RMSE, calculate the score. (Answer: \(\sqrt{(2^2+3^2+4^2)/3}=\sqrt{29/3}\approx3.1091\).)
- A team tries 30 grid candidates, selects the smallest validation RMSE, then reports that same minimum as its final test performance. Identify the problem and the appropriate correction. (Answer: the validation data were used for selection, so the minimum is a selection-optimistic estimate; retain an untouched test set for a final evaluation after the configuration is fixed.)