Why Preprocessing Must Be Reusable
A preprocessing step may look like a simple calculation: replace missing ages with a median, convert a factor to indicator columns, or standardize a numeric variable. But some of these operations learn settings from data. The median, the category frequencies, the factor levels, and the mean and standard deviation used for standardization are all estimated quantities. If they are estimated separately in each data partition, the model is not receiving predictors prepared by one consistent process.
A reusable preprocessing pipeline separates two operations. First, it estimates the required settings using the training data. Second, it applies those stored settings to training, validation, or test data. The second operation must not estimate new medians, means, standard deviations, or category rules.
This separation matters because evaluation is intended to represent predictions from a model and preprocessing procedure built using training information. A validation or test partition is held out for evaluation, not for estimating how predictors should be transformed. The earlier material on data leakage explains why information crossing a partition boundary can make performance estimates misleading.
Define, Prepare, and Apply
The R package recipes provides a direct way to implement this pattern. A recipe records the outcome and predictor roles, followed by ordered preprocessing steps. The function prep() estimates settings from the designated training data. The function bake() applies the prepared recipe to data with the same underlying predictor structure.
In the code below, assume the data have already been divided into training, validation, and test partitions. Each partition contains a binary outcome named claim_paid, a numeric age, annual mileage, and a categorical region. The split itself is not part of preprocessing.
A recipe for imputing missing numeric predictors, handling categorical values, creating indicator columns, removing constant predictors, and standardizing numeric predictors can be written as follows. The package functions and selectors are used in their usual order; the outcome is excluded from the predictor transformations because it has the outcome role in the formula.
R: library(recipes); library(dplyr); rec <- recipe(claim_paid ~ ., data = train) %>% step_impute_median(all_numeric_predictors()) %>% step_unknown(all_nominal_predictors()) %>% step_novel(all_nominal_predictors()) %>% step_dummy(all_nominal_predictors()) %>% step_zv(all_predictors()) %>% step_normalize(all_numeric_predictors())
The recipe is a specification, not yet an estimated transformation. Estimate its settings using only the training partition, then apply the prepared object to each partition as needed.
R: rec_prep <- prep(rec, training = train); train_processed <- bake(rec_prep, new_data = train); validation_processed <- bake(rec_prep, new_data = validation); test_processed <- bake(rec_prep, new_data = test)
Write down the predictor transformations and their order. Identify the outcome separately so it is not imputed, normalized, or dummy-encoded as if it were a predictor.
Call prep() with the training partition. The prepared object stores learned values, such as medians and normalization settings.
Call bake() on training, validation, and test data using the same prepared object. Do not call prep() again on held-out data.
Confirm that the predictor columns are compatible across partitions and that the outcome remains available for evaluation.
Worked Examples
Worked Example: Imputation and Scaling From Training Data
A hypothetical insurer has four training records. Their ages are 20, 30, 40, and missing; their annual mileages, in thousands, are 4, 6, 8, and 10. A separate validation record has age 50 and annual mileage 12. Find the training-estimated imputations and standardized validation values. Use the sample standard deviation, as in the usual R standardization calculation.
Method. Estimate the median, mean, and standard deviation from training data only. Impute the validation age using the training median, then standardize using the training mean and sample standard deviation.
Age imputation. The observed training ages in order are 20, 30, 40. Their median is 30, so the missing training age is replaced by 30. The completed training ages are 20, 30, 40, 30, with mean \((20+30+40+30)/4=30\). Their squared deviations from 30 sum to \(100+0+100+0=200\), so the sample variance is \(200/(4-1)=66.6667\), and the sample standard deviation is approximately 8.1650.
The validation age is 50, so its standardized value is \((50-30)/8.1650 \approx 2.4495\). No validation value was used to calculate the imputation or scaling settings.
Mileage scaling. The training mean is \((4+6+8+10)/4=7\). The squared deviations sum to \(9+1+1+9=20\), so the sample standard deviation is \(\sqrt{20/3}\approx2.5820\). The validation value 12 therefore becomes \((12-7)/2.5820\approx1.9365\).
Check. The training ages have mean 30 and sample standard deviation approximately 8.1650; the mileage values have mean 7 and sample standard deviation approximately 2.5820. The validation values are measured relative to those training quantities. Values are rounded to four decimal places.
Common mistake avoided: calculating a new mean or standard deviation after adding the validation record would make the transformation depend on held-out data.
Worked Example: Rare and Unseen Categories
A hypothetical pricing model uses a categorical territory predictor. In ten training records, the counts are East 6, Central 3, and Coastal 1. A validation partition contains one East record, one Coastal record, one previously unseen Remote record, and one missing territory. Use a training frequency threshold of 0.20 to group rare observed categories, while preserving a separate missing-value treatment and a route for novel levels.
Method. Estimate the rare-category rule from training frequencies, then prepare a recipe that groups rare levels before handling missing and novel values. Do not estimate category frequencies from validation data.
The training proportions are East \(6/10=0.60\), Central \(3/10=0.30\), and Coastal \(1/10=0.10\). Coastal is below the 0.20 threshold and is grouped as Other; East and Central remain distinct. The recipe steps can be specified as follows, assuming territory is a nominal predictor:
R: rec_region <- recipe(claim_paid ~ ., data = train) %>% step_other(territory, threshold = 0.20, other = "Other") %>% step_unknown(territory) %>% step_novel(territory) %>% step_dummy(territory); region_prep <- prep(rec_region, training = train); validation_region <- bake(region_prep, new_data = validation)
After baking, the validation East value remains East, and Coastal maps to Other because Coastal was rare in training. Remote was absent from training and is handled by the novel-level step. The missing value is assigned the missing category. Dummy columns are created using the levels and encoding rules stored in the prepared recipe, not by generating a new set of columns from validation.
Check. The frequency calculation uses exactly the ten training records. The validation records determine neither the 0.20 cutoff nor which training levels are rare. A model cannot learn a separate predictive effect for a category never observed during training; the preprocessing rule only ensures that the new value is handled consistently.
Common mistake avoided: grouping categories after combining training and validation would allow validation frequencies to change which categories are retained.
Why the Settings Must Be Frozen
Let \(D_{\text{train}}\) denote the training data and let \(\theta_{\text{train}}\) collect the settings estimated from it: for example, an imputation median, normalization mean and standard deviation, and the category levels retained by the recipe. The fitted transformation is then \(T_{\theta_{\text{train}}}\). For any held-out predictor record \(x\), preprocessing produces \(T_{\theta_{\text{train}}}(x)\).
The reason for reusing the same settings is direct: a held-out record is meant to be processed by the procedure that could have been constructed from the training data. If instead the validation partition supplies its own settings \(\theta_{\text{validation}}\), the evaluated procedure becomes \(T_{\theta_{\text{validation}}}\), which is not the training procedure. The validation data have now affected the predictor representation before performance is assessed.
This is also why fitting a transformation once on the complete data set, and only then splitting, is not equivalent to preparing it on training data. Even an unsupervised calculation such as a median or principal-component transformation can use information about the held-out predictor distribution. For this tutorial, the practical rule is narrower and operational: make the recipe object using the training data, prepare it on training data, and apply that prepared object to held-out partitions.
Worked Example: Reuse One Prepared Object
A hypothetical portfolio has training ages 25, 35, and 45, a validation age of 55, and a test age that is missing. The intended preprocessing is median imputation followed by standardization. Show how one prepared object handles both held-out records.
Method. The training median is 35. The training mean is \((25+35+45)/3=35\). The squared deviations sum to \(100+0+100=200\), giving sample variance \(200/(3-1)=100\) and sample standard deviation 10. These are the stored settings.
The validation age is standardized as \((55-35)/10=2\). The missing test age is imputed to 35 and then standardized as \((35-35)/10=0\). Both calculations use the same median, mean, and standard deviation.
R: age_recipe <- recipe(claim_paid ~ age, data = train) %>% step_impute_median(age) %>% step_normalize(age); age_prep <- prep(age_recipe, training = train); validation_age <- bake(age_prep, new_data = validation); test_age <- bake(age_prep, new_data = test)
Check. The validation result is 2 and the imputed test result is 0. Preparing the recipe on the validation or test data would produce different settings and would no longer represent the specified training-based procedure.
Common mistake avoided: calling prep() separately on each partition. Prepare once, then call bake() repeatedly.
Implementation Checks and Common Traps
A reusable recipe reduces inconsistency, but it does not remove the need to check the processed data. Verify that the recipe has the intended roles, that transformations are in the intended order, and that each partition can be baked without unexpected missing or novel values. When comparing processed partitions, compare predictor names separately from the outcome column.
For example, after baking validation and test data that both contain the outcome, a basic check is identical(setdiff(names(validation_processed), "claim_paid"), setdiff(names(test_processed), "claim_paid")). The predictor columns should agree in name and order. If the outcome is not available in a prediction data set, keep it separately for evaluation rather than treating it as a predictor.
- Do not refit on validation or test data. Reuse the prepared recipe object.
- Keep the outcome out of predictor steps. A formula such as claim_paid ~ . assigns the outcome its own role; selectors such as all_numeric_predictors() respect that role.
- Do not assume an unseen category has a learnable effect. A novel-level step handles the value; training data still determine what a fitted model can learn about it.
- Keep the recipe with the fitted model. A model without its preprocessing settings cannot reliably transform future records in the same way.
- Record the software environment. Saving the prepared recipe and recording package versions support reproducibility when the workflow is run later.
For example, the prepared object can be saved with saveRDS(rec_prep, "preprocessor.rds") and later restored with rec_prep <- readRDS("preprocessor.rds"). The restored object carries learned preprocessing settings; it should not be re-prepared using the new data being scored.
Check Your Understanding
For each problem, decide which values are estimated from training data and how the prepared preprocessing should be applied.
- A training predictor has observed values 10, 20, 30, and a missing value. A test record has value 50. Use median imputation and training-mean centering only. What value is imputed in training, and what is the centered test value? (Answer: Imputed value 20; centered test value \(50-20=30\).)
- Training category counts are Pine 5, Oak 3, and Elm 2 out of 10. With a rare-category threshold of 0.25, which training level is grouped as Other? (Answer: Elm only, since \(2/10=0.20<0.25\); Pine and Oak are retained.)
- Training numeric values are 2, 4, and 6. A validation value is 8. Using the training mean and sample standard deviation, what is the standardized validation value? (Answer: Mean \(=4\); sample standard deviation \(=\sqrt{[(2-4)^2+(4-4)^2+(6-4)^2]/2}=2\); standardized value \(=(8-4)/2=2\).)
- A factor has training levels North and South. A test record has a previously unseen West level. The prepared recipe includes a novel-level step and dummy encoding. Should you prepare a new recipe on the test data to add West? (Answer: No. Bake with the training-prepared recipe; it handles West using its stored novel-level rule and keeps the trained predictor schema.)
- A team computes a median from training and validation values combined, then uses it to impute missing training values before fitting. What is the problem, and what should be done instead? (Answer: The validation partition influenced preprocessing, creating leakage; estimate the median from training data only, prepare the recipe there, and bake validation with that prepared recipe.)