Choosing Between Raw, Transformed, and Binned Predictors

Choosing a Form for a Predictor

A predictor can enter a model in its original units, after a transformation such as a logarithm, or as a set of intervals. These choices change what patterns the model can represent. A raw predictor in a linear model imposes a constant change in the fitted response for each one-unit increase in the predictor. A logarithmic predictor instead allows equal proportional changes in the predictor to correspond to equal fitted-response changes. Binning gives a constant fitted value within each interval when the model uses a separate mean for each bin.

No form is automatically best. A useful comparison combines plots to assess the shape of the relationship, interpretation to assess whether the model’s statements are meaningful, and validation performance to assess predictions on observations not used to fit the candidate models. The candidate forms and any choices such as bin cut points must be established using training data, not by inspecting validation outcomes.

Definition: For a quantitative predictor $x$, a raw form uses $x$ as recorded; a transformed form replaces it with a specified function such as $\log(x)$; and a binned form assigns $x$ to one of several intervals. A comparison is meaningful only when the response, data split, and evaluation metric are held fixed across candidates.

What the Forms Imply

Consider a simple linear prediction rule for a quantitative response $y$. A raw-predictor model has fitted value $\hat y=a+bx$. Its fitted response changes by $b$ when $x$ increases by one unit. If $b$ is positive, the fitted response increases at a constant rate across the predictor’s range; if $b$ is negative, it decreases at a constant rate.

A logarithmic-predictor model has fitted value $\hat y=a+c\ln(x)$, with $x>0$. It is linear in $\ln(x)$, not in $x$. Increasing $x$ by the same number of units has a smaller fitted effect at larger values of $x$; multiplying $x$ by a fixed positive factor gives a constant fitted-response difference. A log transformation is therefore a candidate when a plot suggests that a response changes more sharply at low predictor values and then levels off. It is not appropriate for zero or negative $x$ unless a separate, substantively justified treatment is specified.

A binned model assigns observations to intervals and fits a separate value for each interval. This can represent a step-like pattern without requiring a straight-line relationship, but observations within a bin receive the same fitted value. Binning therefore discards within-interval distinctions. The choice of cut points can also affect the result substantially.

The scatterplot should use the same response in each comparison. Plotting $y$ against $x$ reveals the raw-scale shape; plotting $y$ against $\ln(x)$ can reveal whether a straight-line pattern is more plausible on the transformed scale. A plot of binned group summaries can make broad changes easier to see, but it hides variation within bins. Visual appearance is evidence for candidate forms, not a substitute for validation.

RMSE: For $n$ validation observations with actual responses $y_i$ and predictions $\hat y_i$, the root mean squared error is $\mathrm{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i-\hat y_i)^2}$. It is expressed in the response’s units, and smaller values mean smaller average squared prediction errors on that validation set.

Why the Raw-Model Slope Has Its Usual Form

For the simple raw-predictor linear model, ordinary least squares chooses $a$ and $b$ to minimize the sum of squared training residuals, $\sum_i(y_i-a-bx_i)^2$. The slope formula follows directly from this minimization. Differentiating with respect to $a$ and setting the derivative to zero gives $a=\bar y-b\bar x$. Substituting this into the sum of squares centers both variables. Differentiating with respect to $b$ and setting the derivative to zero then gives

\hat b= \frac{\sum_i(x_i-\bar x)(y_i-\bar y)} {\sum_i(x_i-\bar x)^2}, \qquad \hat a=\bar y-\hat b\bar x.

The denominator measures variation in the predictor; the numerator measures how predictor and response deviations vary together. This formula requires nonzero variation in $x$. It describes the least-squares fit for a raw linear predictor; a model using $\ln(x)$ instead applies the same fitting logic to the transformed predictor values, not to the original $x$ values.

Worked Example: Comparing Raw, Logarithmic, and Binned Fits

Suppose a hypothetical training set has predictor values $x=1,2,4,8$ and responses $y=5,7,9,11$. A separate validation set has $(x,y)=(3,8)$ and $(6,11)$. Compare a raw linear model, a model using $\log_2(x)$, and a two-bin model with the cut point $x\leq 2$ versus $x>2$. Compute each validation RMSE. Treat the cut point as chosen using training data only.

Raw linear model. The training means are $\bar x=(1+2+4+8)/4=3.75$ and $\bar y=(5+7+9+11)/4=8$. The sum of squared predictor deviations is $7.5625+3.0625+0.0625+18.0625=28.75$. The cross-products are $(1-3.75)(5-8)=8.25$, $(2-3.75)(7-8)=1.75$, $(4-3.75)(9-8)=0.25$, and $(8-3.75)(11-8)=12.75$. Their sum is $23$. Thus $\hat b=23/28.75=0.8$ and $\hat a=8-0.8(3.75)=5$. The fitted rule is $\hat y=5+0.8x$. At validation $x=3$, the prediction is $7.4$, with squared error $(8-7.4)^2=0.36$. At $x=6$, the prediction is $9.8$, with squared error $(11-9.8)^2=1.44$. Therefore, $\mathrm{RMSE}=\sqrt{(0.36+1.44)/2}=\sqrt{0.9}\approx0.9487$, rounded to four decimal places.

Logarithmic model. The training values of $\log_2(x)$ are $0,1,2,3$, while the responses are $5,7,9,11$. They lie exactly on $\hat y=5+2\log_2(x)$. At $x=3$, $\log_2(3)\approx1.5849625$, giving prediction $8.1699250$. The squared error is approximately $(8-8.1699250)^2=0.0288745$. At $x=6$, $\log_2(6)\approx2.5849625$, giving prediction $10.1699250$. The squared error is approximately $(11-10.1699250)^2=0.6890245$. Hence the validation RMSE is $\sqrt{(0.0288745+0.6890245)/2}\approx0.5991$, rounded to four decimal places.

Binned model. The training responses in the lower bin, $x\leq2$, are 5 and 7, so its fitted mean is $(5+7)/2=6$. The upper-bin responses are 9 and 11, so that fitted mean is $(9+11)/2=10$. Both validation values, 3 and 6, fall in the upper bin and receive prediction 10. The squared errors are $(8-10)^2=4$ and $(11-10)^2=1$. Thus the validation RMSE is $\sqrt{(4+1)/2}=\sqrt{2.5}\approx1.5811$, rounded to four decimal places.

Check. The validation RMSEs are approximately 0.9487 for the raw model, 0.5991 for the logarithmic model, and 1.5811 for the binned model. The logarithmic candidate performs best on these two validation cases. The check is consistent with the training pattern: response increases by two when $x$ doubles. This small example does not establish that a logarithm is generally preferable; it illustrates how a fixed validation set can distinguish candidate forms.

Interpreting a Logarithmic Predictor

A coefficient for $\ln(x)$ is not the fitted change for a one-unit increase in $x$. If $x$ changes from $x_1$ to $x_2$, the fitted response difference is $c\ln(x_2/x_1)$. In particular, doubling $x$ changes the fitted response by $c\ln(2)$, regardless of the starting value, provided both values are in the model’s domain. This is a difference in the response’s units, not a percentage change in the response.

Worked Example: Explaining a Log-Scale Coefficient

A hypothetical fitted model predicts an equipment-maintenance cost, measured in dollars, as $\hat y=120+4.5\ln(x)$, where $x$ is operating time in hours and $x>0$. Compare predicted cost at 20 and 40 hours, and interpret the coefficient for a doubling of operating time.

At 20 hours, $\ln(20)\approx2.9957322736$, so $\hat y=120+4.5(2.9957322736)=133.4807952$ dollars. At 40 hours, $\ln(40)\approx3.6888794541$, so $\hat y=120+4.5(3.6888794541)=136.5999575$ dollars. The predicted difference is $136.5999575-133.4807952=3.1191623$ dollars.

The same result follows from the ratio: $4.5\ln(40/20)=4.5\ln(2)\approx3.1191623$ dollars. Thus, under this fitted rule, doubling operating time is associated with an increase of about \$3.12 in predicted cost. The calculation is checked both by subtracting the predictions and by using the ratio formula; money is rounded to the nearest cent only in the final interpretation.

Use Training Data for Every Choice

Transformation and binning decisions are part of the modeling process. If a modeler uses validation outcomes to choose a cut point, select a transformation, or decide which candidate to retain, the validation set has influenced model selection. Its error is then no longer a clean assessment of performance on data that played no role in those choices.

Fit any estimated preprocessing quantities on the training data: for example, a centering constant, a scale factor, or cut points. Apply the resulting fixed rule to validation predictors. Do not recompute the rule from the validation distribution. If a validation predictor falls outside training ranges or into an interval with little support, record that issue and consider its implications; do not quietly revise the rule after seeing validation results.

Worked Example: Scoring a Binned Predictor Without Leakage

A hypothetical training set records maintenance incidents at operating times of 20, 30, 40, 60, 80, and 100 hours, with respective incident counts 2, 3, 4, 9, 11, and 13. A cut point of 50 hours is selected from the training data. Validation observations are $(45,5)$ and $(90,12)$, where each pair gives operating time and observed count. Fit the bin means on training data and calculate validation RMSE.

The lower-bin training counts are 2, 3, and 4, so the fitted value for $x\leq50$ is $(2+3+4)/3=3$. The upper-bin training counts are 9, 11, and 13, so the fitted value for $x>50$ is $(9+11+13)/3=11$. The validation value at 45 hours is assigned the lower-bin prediction 3, and the value at 90 hours receives the upper-bin prediction 11.

The validation squared errors are $(5-3)^2=4$ and $(12-11)^2=1$. Therefore, $\mathrm{RMSE}=\sqrt{(4+1)/2}=\sqrt{2.5}\approx1.5811$, rounded to four decimal places. The validation observations were assigned to bins but did not contribute to either bin mean or the cut point. The calculation would be contaminated if the cut point or fitted means were revised after examining the two validation counts.

Common Traps and Practical Choice

A common arithmetic trap is an incorrect cross-product sum in the raw-model slope. In the first example, a mistaken sum of 25 would give $25/28.75\approx0.8696$, rather than the correct slope $23/28.75=0.8$. The four products are $8.25, 1.75, 0.25,$ and $12.75$, which add to 23. Recalculate each product and their sum before propagating a slope error into predictions or RMSE.

Other mistakes arise from interpreting the candidate forms rather than from arithmetic:

Assuming a transformation must improve the model. A visually plausible transformed relationship still needs validation comparison.
Reading a log-predictor coefficient as a per-unit effect. The fitted change from $x_1$ to $x_2$ is $c\ln(x_2/x_1)$, not simply $c$.
Confusing a logged predictor with a logged response. The model $\hat y=a+c\ln(x)$ predicts response in its original units; its coefficient does not describe a response percentage change.
Choosing bins to make validation error look small. Use training data to select the cut points and estimate bin values, then apply those fixed choices to validation observations.
Choosing solely by one error score. Validation results can vary, especially with small samples. Consider plot shape, interpretability, bin support, and stability alongside the metric.

A practical comparison begins with plots and a small set of defensible candidate forms. Fit each candidate using the same training observations, make validation predictions using training-derived preprocessing, and compute the same metric on the same validation observations. Prefer a more complex form only when the improvement is meaningful enough to justify its interpretive or operational cost. This comparison is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Check Your Understanding

For each problem, identify the relevant form or calculation and check the final result against the stated data.

A fitted model is $\hat y=18+2.7\ln(x)$, with $x>0$. What is the predicted-response difference when $x$ doubles? Give the result to four decimal places. (Answer: $2.7\ln(2)\approx1.8715$ response units.)
A model using $\log_2(x)$ is fitted to training values $(x,y)=(1,4),(2,6),(4,8)$, with an exact rule $\hat y=4+2\log_2(x)$. Predict at $x=8$, and state the assumption needed to use that rule there. (Answer: $\hat y=10$; the same fitted relationship is assumed to apply at $x=8$, an extrapolation beyond the training range.)
A raw model has validation actual values 12 and 15 and predictions 11 and 17. Calculate its validation RMSE. (Answer: $\sqrt{(1^2+(-2)^2)/2}=\sqrt{2.5}\approx1.5811$.)
A training set has outcomes 2 and 6 in a lower bin and 8 and 12 in an upper bin. A fixed training-derived cut point puts validation observations with actual outcomes 5 and 10 in the upper bin; upper-bin predictions are the training mean. Calculate validation RMSE. (Answer: the upper-bin mean is $(8+12)/2=10$, both predictions are 10, and RMSE is $\sqrt{((5-10)^2+(10-10)^2)/2}=\sqrt{12.5}\approx3.5355$.)
A modeler tries three transformations and several cut points, repeatedly comparing validation RMSE, then reports the smallest value as an unbiased estimate of future error. Identify the problem and give a sounder evaluation approach. (Answer: the validation set has been used for model selection, so the minimum RMSE is optimistically biased; select using training data and validation, then assess the chosen procedure on an untouched test set or through an appropriate resampling evaluation.)

What You'll Learn