Refitting The Selected Model For Deployment

From a Selected Model to a Deployment Fit

Cross-validation or another validation procedure can identify a useful modeling specification: for example, a particular penalty, mixing parameter, predictor set, and preprocessing plan. The fitted coefficients used during that selection are not automatically the coefficients to deploy. They may have been estimated on only part of the available development data, or may represent fold-specific fits. The deployment step is to refit the selected specification on the intended development data and make sure future observations pass through compatible preprocessing.

This is a procedural step, not another opportunity to choose whichever model gives the most attractive result. The penalty and other selected settings stay fixed during the refit. The model coefficients are estimated again using the data designated for fitting. This topic is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Definition: A deployment refit is a new model fit using the selected model specification and the intended development data. The specification includes the response and predictor definitions, model family and link when relevant, penalty settings, and preprocessing rules. A prediction pipeline is the ordered set of stored preprocessing and model steps that converts a new observation in its original form into a prediction on the required scale.

What Is Fixed, and What Is Re-estimated?

A refit keeps the chosen method fixed but estimates its coefficients using the specified fitting data. For a Gaussian elastic-net model, one common objective, with the intercept unpenalized, is

\frac{1}{2n}\sum_{i=1}^{n}(y_i-\beta_0-x_i^\mathsf{T}\beta)^2 +\lambda\left[\alpha\sum_{j=1}^{p}|\beta_j|+ \frac{1-\alpha}{2}\sum_{j=1}^{p}\beta_j^2\right].

Here $n$ is the number of fitting observations, $p$ is the number of predictors, $\lambda$ is the penalty strength, and $\alpha$ controls the mix of lasso and ridge penalties. In a deployment refit, the selected $\lambda$ and $\alpha$ remain unchanged, while $\beta_0$ and $\beta$ are re-estimated. A generalized linear model uses its corresponding likelihood-based loss in place of the squared-error term; the same distinction between fixed settings and refitted coefficients applies.

Preprocessing also has two parts. The rule can remain fixed—for example, mean-impute a numeric predictor, standardize it, and use a specified reference category for a factor. A data-derived value in that rule—such as the imputation mean or standardization mean and standard deviation—must be estimated from the data permitted for the final fit. If the development data expand, recompute those values from that fitting data and apply them consistently. Do not silently change the rule itself after evaluation.

Important distinction: Preserve the selected preprocessing procedure; do not necessarily preserve every numeric preprocessing value learned from a smaller fit. Re-estimate data-dependent values on the final fitting data, then store those values for future predictions. If a transformation was fixed in advance rather than learned from data, retain that fixed definition.

The fitting data must respect the evaluation design. If a final test set is still being held out to estimate performance, exclude it from the deployment refit at this stage. Do not use its outcomes to tune the penalty, select predictors, or estimate coefficients while continuing to describe it as an untouched test set. If an evaluation protocol later permits using those observations for a final operational fit, document that this happens after the reported evaluation; the test result remains an assessment of the modeling procedure, not a fresh test of the subsequently refitted coefficients.

Freeze the selected specification.
Record the response, predictors, model family and link, penalty type and values, offset or weights if used, and preprocessing rules.

Define the fitting boundary.
Identify exactly which observations are available for coefficient fitting. Keep any still-reserved test data outside that boundary.

Re-estimate preprocessing and coefficients.
Learn needed imputation, scaling, or encoding values from the permitted fitting data, then refit the model with the selected settings.

Validate the prediction interface.
Check that new data have the expected variables, units, factor levels, missing-value handling, and any required offsets or weights.

Store the complete pipeline.
Save the preprocessing values and rules together with the refitted model, and test the pipeline on known inputs.

Why Preprocessing Must Travel with the Model

Suppose a numeric predictor $x$ is standardized using a training mean $\bar{x}$ and standard deviation $s_x$. The fitted model uses $z=(x-\bar{x})/s_x$, not the raw $x$. A coefficient estimated for $z$ therefore has meaning only when new observations are transformed using the same stored $\bar{x}$ and $s_x$.

The reason follows directly by substituting the transformation into the fitted linear predictor. If the model is $\eta=\beta_0+\beta_1z$, then for a new value $x_{\mathrm{new}}$,

\eta_{\mathrm{new}} =\beta_0+\beta_1\frac{x_{\mathrm{new}}-\bar{x}}{s_x}.

Changing the mean or standard deviation at prediction time changes $z$, and therefore changes the predictor and the resulting prediction. For instance, using a new batch's mean would make the prediction for one person depend on which other people happened to arrive in that batch. The stored training transformation avoids that inconsistency. The same principle applies to imputation values and factor encodings: their definitions and fitted values belong to the pipeline, not to each incoming batch.

Worked Example: Refit a Standardized Ridge Model

A hypothetical portfolio team used cross-validation to select a Gaussian ridge model with $\alpha=0$ and $\lambda=0.08$. There are three development observations, and no separate final test set is being held out. The chosen preprocessing standardizes two numeric predictors using the sample standard deviation. On the full development data, the first predictor has values 1, 2, 3 and the second has values 0, 1, 2. Suppose the refit returns an intercept of 50 and standardized-scale coefficients 4 and $-2$. For a new observation with predictor values 4 and 3, calculate the prediction. Values are in thousands of dollars.

Method. Keep $\alpha=0$, $\lambda=0.08$, and the selected standardization rule. Recalculate its values on all three permitted development observations, then transform the new observation with those same values. The stated coefficients are the hypothetical output of that refit.

For predictor 1, the development mean is $(1+2+3)/3=2$. Its sample variance is $((1-2)^2+(2-2)^2+(3-2)^2)/(3-1)=2/2=1$, so its sample standard deviation is 1. The new standardized value is $(4-2)/1=2$.

For predictor 2, the development mean is $(0+1+2)/3=1$. Its sample variance is $((0-1)^2+(1-1)^2+(2-1)^2)/2=2/2=1$, so its sample standard deviation is 1. Its new standardized value is $(3-1)/1=2$.

The fitted linear prediction is $50+4(2)+(-2)(2)=50+8-4=54$ thousand dollars. Thus the prediction is $54,000.00. The check is that both new values are two training standard deviations above their respective training means, and the coefficient contributions are $+8$ and $-4$ thousand dollars.

Common mistake avoided: applying the coefficients to raw values gives $50+4(4)-2(3)=60$, which is wrong because the coefficients apply to standardized predictors, not raw values.

Factor Levels and the Prediction Contract

A factor must be encoded consistently between fitting and prediction. For a factor with reference level “Online” and indicator columns for “Agent” and “Branch,” a new “Agent” observation receives indicator values 1 and 0, respectively. It does not receive a new coefficient column simply because it is presented in a later data batch. If a new level appears that was not represented in the fitted encoding, the pipeline needs a defined policy: reject the input for correction, or map it to a designated “Other” level that was included in fitting. Assigning arbitrary indicators or silently treating an unknown level as the reference can produce an unintended prediction.

In R, the practical requirement is to rebuild the same design matrix structure used in fitting, not to create a fresh encoding independently for each new data frame. With a `glmnet` fit, `predict()` expects a numeric predictor matrix with compatible columns. If predictors are manually imputed, scaled, and dummy-encoded, apply the stored training values and column order to new data, and fit with `standardize=FALSE` if that is how the model was specified. Alternatively, when using the model's internal standardization, keep the fitted object and use its prediction method rather than applying a second, independently calculated standardization.

Worked Example: Logistic Prediction with a Factor Indicator

A hypothetical service team selected a binomial elastic-net model with $\alpha=0.6$, $\lambda=0.12$, and an “Online” reference level for channel. Age is standardized using a development mean of 40 years and standard deviation of 10 years. After refitting, the intercept is $-1.2$, the age coefficient is $0.05$, and the “Agent” indicator coefficient is $0.3$. Calculate the predicted event probability for a 50-year-old “Agent” customer.

Method. Use the stored age transformation and the fitted reference-level encoding, then convert the logistic linear predictor to a probability. The coefficients are the stipulated output from the refit with the selected settings.

The standardized age is $z=(50-40)/10=1$. “Agent” is not the reference level, so its indicator is 1; the “Branch” indicator, if present, is 0. The linear predictor is $\eta=-1.2+0.05(1)+0.3(1)=-1.2+0.05+0.3=-0.85$.

For the logit link, $p=1/(1+\exp(-\eta))$. Hence $p=1/(1+\exp(0.85))$, where $\exp(0.85)\approx2.3396$. Therefore $p=1/3.3396\approx0.2994$, or 0.2994 to four decimal places. This is a probability of about 29.94%. The check is that a negative linear predictor must give a probability below 0.5.

Common mistake avoided: using the raw age of 50 in place of its standardized value would give the age term $0.05(50)=2.5$, an invalid calculation because the coefficient was fitted for standardized age.

Implementing and Checking the Pipeline

A deployment-ready object should contain more than a coefficient vector. It should also preserve the predictor names and order, factor levels and reference categories, any imputation values, scaling values, transformations, and the model settings that determine how predictions are produced. If the model uses an offset, the prediction process must obtain an offset on the scale required by the fitted model; if it uses observation weights, those weights belong to fitting and should not be confused with predictor columns.

A sound workflow separates tuning from refitting. During cross-validation, preprocessing that learns values from data must be learned within each training fold rather than from the entire data set before the folds are formed. Otherwise, information from a validation fold can influence the transformation used to assess that fold. After the specification is selected, fit the preprocessing anew on the intended development data and then refit the model. Do not reuse one fold's means, factor matrix, or coefficients as if they represented the full development fit.

Worked Example: Imputation and a Stored Prediction Pipeline

A hypothetical model predicts a response measured in thousands of dollars. The selected preprocessing imputes missing values of a numeric predictor with its development-data mean, standardizes that predictor using the development sample standard deviation, and represents factor level “B” with an indicator relative to reference level “A.” In the final development data, the observed numeric values are 10, 20, and 30, giving mean 20 and sample standard deviation 10. Suppose the refitted model has intercept 3, standardized-predictor coefficient 0.2, and “B” coefficient $-0.1$. Apply the pipeline to two new records: (i) missing numeric value and level B; (ii) numeric value 30 and level A.

Method. Use the saved imputation mean, scaling mean and standard deviation, and factor encoding from the final development fit. The coefficients are the hypothetical refit output. Do not recalculate preprocessing values from the two new records.

For record (i), the missing value is replaced by 20. Its standardized value is $(20-20)/10=0$. Level B has indicator 1. The prediction is $3+0.2(0)-0.1(1)=2.9$ thousand dollars, or $2,900.00.

For record (ii), the standardized value is $(30-20)/10=1$. Level A is the reference, so the B indicator is 0. The prediction is $3+0.2(1)-0.1(0)=3.2$ thousand dollars, or $3,200.00. Both results are on the stated response scale. The check is that the imputed value equals the training mean and therefore contributes zero through the standardized numeric predictor.

Common mistake avoided: calculating a new imputation mean or standard deviation from the incoming records would make the pipeline depend on the composition of each prediction batch rather than on the fitted development data.

Common Deployment Errors

Retuning during the refit. If the selected penalty is fixed, refitting is not a new search for a more favorable penalty. A new tuning exercise requires a clearly defined evaluation plan.
Reusing fold-specific coefficients. Cross-validation fits serve the validation procedure. The final coefficients must be estimated using the full intended fitting data and selected settings.
Mixing raw and transformed predictors. Coefficients for standardized or dummy-coded predictors cannot be applied to raw values or a differently ordered matrix.
Changing the evaluation boundary unnoticed. Including a still-reserved test set in fitting means it is no longer an untouched test set.
Leaving unknown factor levels undefined. Establish and test a policy rather than silently assigning an unintended encoding.
Saving only the model object. If preprocessing was external to that object, save the preprocessing rules and learned values alongside it.

Key takeaway: Refit the selected specification—not a newly selected one—on the intended development data. Preserve the preprocessing procedure, learn its data-dependent values from the permitted fitting data, and apply the stored transformation and design-matrix structure unchanged to new observations.

Check Your Understanding

Work through each decision or calculation before checking the answer.

A selected ridge model uses $\alpha=0$ and $\lambda=0.05$. Cross-validation fits used 80 observations, while the development set contains 100 observations and no final test set is reserved. How many observations belong in the refit, and which penalty settings are used? (Answer: Refit on all 100 development observations with $\alpha=0$ and $\lambda=0.05$.)
A numeric predictor was standardized with training mean 60 and standard deviation 8. A new value is 76. If its fitted coefficient is 1.5 and the intercept is 12, what is its linear prediction? (Answer: $z=(76-60)/8=2$, so $12+1.5(2)=15$.)
A logistic refit has intercept $-0.4$, coefficient 0.7 for a standardized predictor, and coefficient $-0.3$ for a factor indicator. A new record has standardized predictor 1 and indicator 1. Find its event probability. (Answer: $\eta=-0.4+0.7-0.3=0$, so $p=1/(1+\exp(0))=0.5000$.)
A fitted factor encoding includes levels “Basic” and “Plus,” but a new record has level “Premium,” which was not in the development data. What should the pipeline do if no “Other” level was defined and fitted? (Answer: Reject or flag the record for a defined resolution; do not silently assign an arbitrary indicator or reference level.)
A final test set has not yet been used to report performance. The team proposes to include its outcomes in the coefficient refit while still calling the resulting test score an untouched final assessment. Is that appropriate? (Answer: No. Keep the test set outside fitting until its evaluation is complete; using its outcomes in the refit means it is no longer an untouched test set.)

What You'll Learn