Separating Formula Effects From Algorithm Effects

Why Separate Formula and Algorithm Effects?

A model can improve because it uses more information or represents that information more flexibly. It can also improve because a different algorithm learns a better prediction rule from the same inputs. These are distinct explanations. If a candidate changes both the formula and the algorithm at once, its performance alone cannot tell us which change mattered.

For example, suppose a hypothetical insurer compares a GLM using a linear age term with a boosted tree using age, a squared age term, and an interaction between age and vehicle type. A better validation score for the tree would not isolate an algorithm advantage: the tree also received a richer set of input terms. A staged comparison changes one aspect at a time, using the same prediction target, validation observations, and evaluation metric wherever possible.

Definition: A formula effect is a change in validation performance when the model formula changes but the algorithm is held fixed. An algorithm effect is a change in validation performance when the algorithm changes but the formula, target, validation observations, and evaluation metric are held fixed. These are controlled comparisons; they do not imply that the effects will be identical under every modeling setup.

Set Up the Comparisons

A model formula specifies the response and the predictor terms supplied to the modeling procedure. A formula change can add a predictor, transform a predictor, or add an interaction. An algorithm is the procedure that uses the available inputs to produce predictions, such as a GLM or a tree-based method. The comparison is meaningful only when the intended change is clearly stated.

For a quantitative response, one common evaluation metric is root mean squared error. For $n$ validation observations with observed responses $y_i$ and predictions $\hat{y}_i$, define the error $e_i=\hat{y}_i-y_i$. Then:

\operatorname{RMSE}=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(\hat{y}_i-y_i)^2}

RMSE is expressed in the response's units, and smaller values indicate better predictions under squared-error loss. It is sensitive to large errors because the errors are squared. This metric is not appropriate as a direct performance measure for a categorical response; the metric must match the prediction target and decision being assessed.

A basic staged plan has two comparisons. First, hold the algorithm fixed and compare a baseline formula with a specified richer formula. Second, hold the formula fixed and compare algorithms. If the formula is to be changed more than once, compare the stages separately—for example, baseline terms, then a transformation, then an interaction. This reveals which addition coincides with a performance change on the chosen validation data.

Fix the evaluation basis.
Use the same target definition, validation observations, and metric for the candidate predictions being contrasted.

Test formula changes within an algorithm.
Compare the baseline and expanded formula while keeping the algorithm fixed.

Test algorithm changes within a formula.
Compare algorithms using the same declared formula and validation setup.

Interpret the contrasts narrowly.
Attribute a measured difference to the change made in that contrast, not to a broader claim that one model class is always superior.

Why Paired Comparisons Help

When two models predict the same validation cases, their errors can be compared case by case. This is preferable to comparing scores calculated on different observations, because the latter may differ due to the cases themselves rather than the models. For squared error, the mean difference in loss between models A and B is:

\frac{1}{n}\sum_{i=1}^{n}\left[(\hat{y}_{i,A}-y_i)^2-(\hat{y}_{i,B}-y_i)^2\right]

This expression compares each pair of predictions against the same observed response. The reason the pairing matters can be seen by writing $L_{i,A}=(\hat{y}_{i,A}-y_i)^2$ and $L_{i,B}=(\hat{y}_{i,B}-y_i)^2$. The average loss difference is the average of $L_{i,A}-L_{i,B}$, not the difference between losses computed on separate samples. In a staged comparison, the same cases keep the evaluation basis constant while the designated model component changes.

RMSE itself is a square root of mean squared error, so an RMSE difference is not numerically the same as a mean difference in squared losses. Report the requested metric, and do not describe a change in RMSE as though it were a change in MSE. Both can be useful, but they are different quantities.

Worked Example: Testing a Transformation Within One Algorithm

A hypothetical claim-cost model is evaluated on four held-out policies. A GLM using the baseline formula produces errors, defined as prediction minus observed cost, of $2,-2,1,-1$ thousand dollars. A GLM with a prespecified nonlinear age transformation produces errors of $1,-1,1,-1$ thousand dollars. Calculate each RMSE and the change in performance.

Method. This is a formula contrast within the same algorithm. The response, validation cases, and error units are unchanged.

For the baseline formula, the squared errors are $2^2=4$, $(-2)^2=4$, $1^2=1$, and $(-1)^2=1$. Their sum is $4+4+1+1=10$. The mean squared error is $10/4=2.5$ thousand-dollars-squared, so:

\operatorname{RMSE}_{\text{baseline}}=\sqrt{2.5}=1.5811\text{ thousand dollars}

For the transformed formula, the squared errors are $1,1,1,1$, with sum $4$. The mean squared error is $4/4=1$, and:

\operatorname{RMSE}_{\text{transformed}}=\sqrt{1}=1.0000\text{ thousand dollars}

The RMSE decreased by $1.5811-1.0000=0.5811$ thousand dollars, or $\$581.10$. Relative to the baseline RMSE, the decrease is $0.5811/1.5811=0.3675$, or $36.75\%$. The MSE decreased from $2.5$ to $1$, which is a $60\%$ decrease: $(2.5-1)/2.5=0.60$. These are distinct percentage changes because RMSE is the square root of MSE.

Check. Squaring the transformed RMSE gives $1.0000^2=1$, equal to its calculated MSE. The comparison supports a gain from this transformation for this GLM on these validation observations; it does not establish that the transformation will help every algorithm or future sample. The common mistake avoided is to call the $60\%$ MSE decrease a $60\%$ RMSE decrease.

Use a Crossed Comparison When Both Factors Matter

A formula improvement may depend on the algorithm. To examine that possibility, compare each formula under each algorithm. This forms a small crossed design: rows can represent formulas and columns algorithms, or vice versa. The differences within a row or column answer different questions. The formula contrast is calculated within an algorithm; the algorithm contrast is calculated within a formula.

If the formula contrast is substantially different across algorithms, the formula and algorithm changes do not act independently in this evaluation. This is sometimes described as an interaction between formula choice and algorithm choice. It is not the same thing as a predictor interaction term in a model formula; it describes how the measured formula contrast varies across algorithms.

Worked Example: A Crossed Formula-and-Algorithm Comparison

On the same four validation claims, a hypothetical team obtains the following error vectors, in thousands of dollars: GLM with baseline formula, $2,-2,2,-2$; GLM with expanded formula, $1,-1,1,-1$; tree algorithm with baseline formula, $1,-1,1,-1$; tree algorithm with expanded formula, $1,-1,1,-1$. Calculate the RMSEs, the formula contrasts, and the algorithm contrasts.

Method. Calculate each RMSE from its error vector. Then calculate formula changes within each algorithm and algorithm changes within each formula.

The baseline-formula GLM has squared errors $4,4,4,4$, so its MSE is $16/4=4$ and its RMSE is $\sqrt{4}=2$. The expanded-formula GLM has squared errors $1,1,1,1$, so its MSE is $4/4=1$ and its RMSE is $1$.

The tree algorithm has the vector $1,-1,1,-1$ under both formulas. For each, the squared errors sum to $4$, the MSE is $1$, and the RMSE is $1$. Thus the RMSE table is:

Formula	GLM RMSE	Tree RMSE
Baseline	2	1
Expanded	1	1

Within the GLM, expanding the formula changes RMSE by $2-1=1$, a decrease of 1. Within the tree algorithm, the decrease is $1-1=0$. The formula contrast therefore differs by $1-0=1$ RMSE unit across these two algorithms.

Within the baseline formula, changing from GLM to tree changes RMSE by $2-1=1$, an improvement of 1. Within the expanded formula, the algorithm change is $1-1=0$. Thus the measured algorithm advantage also depends on which formula is used. The crossed comparison makes that dependence visible instead of assigning the entire difference between the GLM baseline (2) and the tree with expanded formula (1) to the algorithm.

Check. Every entry follows from a squared-error sum of either 16 or 4 divided by four observations. The trap avoided is comparing only the two diagonal models and attributing their one-unit difference to a single cause.

Separate Several Formula Steps Before Comparing Algorithms

When an expanded formula adds multiple features at once, the validation comparison estimates the performance change for that package of changes. It does not identify the contribution of each individual term. If the purpose is to diagnose which step helped, introduce terms in a stated sequence and calculate each contrast while holding the algorithm fixed. A second algorithm can then be assessed on each fixed formula.

Worked Example: Locating the Source of a Staged Gain

A hypothetical analyst compares a baseline formula $F_0$, a formula $F_1$ that adds a squared-distance term, and a formula $F_2$ that also adds a distance-by-territory interaction. On one common validation set, the RMSEs are: GLM, $12.0$, $10.8$, and $9.9$; boosted tree, $11.2$, $10.1$, and $9.8$, in dollars. Compute the stepwise formula changes and the algorithm contrasts.

Method. First subtract successive RMSEs within each algorithm. Then compare algorithms at each fixed formula. A positive decrease denotes lower RMSE for the later candidate.

For the GLM, adding the squared-distance term changes RMSE by $12.0-10.8=1.2$ dollars. Adding the interaction after that changes it by $10.8-9.9=0.9$ dollars. The total baseline-to-$F_2$ decrease is $12.0-9.9=2.1$ dollars, which checks against $1.2+0.9=2.1$.

For the boosted tree, the first formula step changes RMSE by $11.2-10.1=1.1$ dollars. The second step changes it by $10.1-9.8=0.3$ dollars. The total is $11.2-9.8=1.4$ dollars, equal to $1.1+0.3=1.4$.

At fixed $F_0$, the boosted tree's RMSE advantage over the GLM is $12.0-11.2=0.8$ dollars. At $F_1$, it is $10.8-10.1=0.7$ dollars. At $F_2$, it is $9.9-9.8=0.1$ dollars. Therefore the data show formula-step improvements within both algorithms, while the difference between algorithms narrows as the formula changes. The $F_2$ tree's score of $9.8$ is only $12.0-9.8=2.2$ dollars below the baseline GLM, but that diagonal difference combines formula changes and an algorithm change.

Check. The sequential formula reductions add to the total reduction for each algorithm. These validation results describe these candidates on this validation set; they do not prove that either term will improve performance in a new sample. The trap avoided is labeling the full $2.2$-dollar diagonal difference an algorithm gain.

Interpretation and Common Traps

A disciplined comparison supports a limited, defensible conclusion: for example, “Adding the specified transformation reduced validation RMSE by 0.58 thousand dollars within this GLM.” It does not support a broader claim that transformations always help or that one algorithm is generally best. Validation scores are evidence about the specified candidates under the specified setup.

Changing two things at once. A richer formula and a new algorithm create a confounded contrast. Stage the changes or evaluate a crossed set of candidates.
Comparing different validation cases. A score difference may reflect different case difficulty. Align the evaluation observations for a controlled comparison.
Changing the target or metric. A model predicting claim frequency cannot be compared directly with one predicting claim cost by a single RMSE. Define the same response and metric first.
Overstating a bundled formula gain. If several terms are added together, the comparison evaluates the bundle, not each term's isolated contribution.
Ignoring algorithm-specific formula behavior. The same named term need not be used in the same way by every algorithm. State what features each candidate receives and interpret each contrast accordingly.
Confusing RMSE and MSE changes. RMSE is the square root of MSE, so percentage changes in them are generally not equal.

The central principle is to define the contrast before reading the score. Hold the evaluation basis fixed, change one intended component at a time, and use a crossed comparison when the benefit of one component may depend on another. This approach is aligned with the SOA Exam PA (Predictive Analytics) syllabus because it makes model comparisons interpretable rather than merely ranking candidate scores.

Key takeaway: A model score establishes which candidate performed better in a stated comparison; only a controlled staged comparison supports an explanation of whether the gain came from the formula, the algorithm, or their combination.

Check Your Understanding

For each problem, identify the relevant contrast and calculate the requested quantity.

A GLM has validation RMSE 5.4 under a baseline formula and 4.8 after adding a prespecified transformation, using the same target and cases. What is the absolute and relative RMSE decrease? (Answer: 0.6 RMSE units; $0.6/5.4=11.11\%$.)
On the same validation set, baseline and expanded formula RMSEs are respectively 8 and 6 for a GLM, and 7 and 6 for a tree algorithm. What is the formula decrease within each algorithm, and how much does that contrast differ? (Answer: GLM decrease 2, tree decrease 1; the formula contrast differs by 1 RMSE unit.)
A model's four validation errors are $3,0,4,0$. A second formula under the same algorithm gives errors $1,0,2,0$. Calculate both RMSEs and the MSE decrease. (Answer: RMSEs $2.5000$ and $1.1180$; MSE decreases from $6.25$ to $1.25$, a decrease of $5.00$.)
A baseline GLM has RMSE 10.0. A tree with a transformed formula has RMSE 8.7. Can the 1.3-unit difference be attributed to the algorithm alone? State what additional comparisons are needed. (Answer: No; compare both algorithms on the baseline formula and both on the transformed formula, using the same validation target, cases, and metric.)
For one algorithm, three staged formulas have RMSEs 9.6, 9.1, and 8.4. Calculate the reduction at each step and the total reduction; state whether the results isolate the effect of each individual term if the second step adds two terms together. (Answer: reductions are 0.5 and 0.7, total 1.2; the second-step result evaluates the two added terms as a bundle, not separately.)

What You'll Learn