Simple Baselines for Classification and Numeric Prediction

Why Begin With a Simple Baseline?

A predictive model is easier to interpret when there is a clear reference point. A baseline is a deliberately simple prediction rule against which a more elaborate model can later be assessed. For classification, a natural baseline is to predict the most frequent class. For numeric prediction, a natural baseline is to predict the historical mean of the response. Neither rule uses information about the individual case being predicted.

A baseline is not a claim that every case is alike. It is a starting prediction that describes what can be said before using case-specific predictors. For an insurer, for example, a constant prediction of “no reported claim” does not mean every policyholder has the same risk; it records the simplest class prediction available from the chosen data. A constant prediction of a historical average claim amount summarizes the response values in a specified reference set, rather than estimating a distinct amount for every policyholder.

These baselines are aligned with the SOA Exam PA (Predictive Analytics) syllabus. Their usefulness depends on a clear response definition and a well-defined set of observations. The earlier material on defining operational success is relevant: the prediction target, observation unit, and time period must be settled before a baseline can be interpreted.

Definition: A baseline prediction rule is a simple rule used as a reference for prediction. A majority-class baseline always predicts the most frequent response category in a specified training set. A historical-mean baseline always predicts the arithmetic mean of a numeric response in that training set. These rules do not use predictors to vary the prediction from case to case.

The Majority-Class Baseline

Suppose the response is a class label $Y$ with possible classes $c_1,\ldots,c_K$, and the training set contains $n$ observations. Let $n_k$ be the number of training observations whose response is $c_k$. A majority-class baseline predicts a class with the largest count:

\hat{c} \in \operatorname*{arg\,max}_{c_k} n_k.

If exactly one class has the largest count, the rule is unambiguous. If multiple classes tie for the largest count, there is no unique majority class. A tie-breaking rule should be specified in advance, such as a fixed ordering of labels. The chosen label is still a constant prediction; the tie-break does not make it more informative.

The rule has a simple interpretation under a particular objective: if every wrong classification counts equally, predicting the most frequent class makes the fewest errors on the same labeled data. If the rule predicts $c_k$, it is correct on $n_k$ training observations and wrong on $n-n_k$. The number of errors is minimized by making $n_k$ as large as possible. This justification is limited to equal penalties for all incorrect classes; it does not settle which prediction is best when different mistakes have different consequences.

Worked Example: A Binary Claim-Reporting Baseline

A hypothetical insurer defines a binary response as whether a policy had a reported claim during the next 12 months. In a training set of 24 policies, 6 have a reported claim and 18 do not. Determine the majority-class baseline and calculate its fraction correct on this training set.

Step 1: Identify the class counts. The two class counts are $n_{\text{claim}}=6$ and $n_{\text{no claim}}=18$. They add to $6+18=24$, agreeing with the stated training-set size.

Step 2: Select the most frequent class. Since $18>6$, the majority class is “no claim.” The baseline therefore predicts “no claim” for every policy, without using any policy characteristics.

Step 3: Calculate the training-set fraction correct. The rule is correct for the 18 policies with no reported claim, so the fraction correct is $18/24=0.75$, or 75%.

Check. The rule predicts one class for all 24 policies; it is correct for 18 and incorrect for 6, and $18+6=24$. The method is the majority-class rule. A common mistake is to treat the 75% as evidence that the rule distinguishes high-risk from low-risk policies. It does not: every policy receives the same prediction. This calculation describes the training set and, by itself, does not establish performance on new policies.

The Historical-Mean Baseline

For a numeric response $Y$, let $y_1,\ldots,y_n$ be the observed values in the training set. The historical-mean baseline predicts their arithmetic mean for every future case:

\hat{y}_{\text{base}}=\bar{y} =\frac{1}{n}\sum_{i=1}^{n}y_i.

This formula treats each recorded observation equally. Its interpretation therefore depends on what a row represents. If each row represents one policy-month, the mean is the average response per policy-month in the reference data, not necessarily an average per policy or per dollar of exposure. The response units also carry through: if $y_i$ is measured in dollars, the baseline prediction is in dollars.

The historical mean is not simply “the average of the values we happen to have.” The data and target must refer to the intended prediction setting. For example, a mean calculated from one product, period, or population may not represent another. A mean can still serve as a transparent reference, but its scope should be stated rather than hidden behind the word “historical.”

Theorem: For a fixed set of numeric responses $y_1,\ldots,y_n$, the arithmetic mean $\bar{y}$ minimizes the sum of squared deviations $\sum_{i=1}^{n}(y_i-a)^2$ over all constant predictions $a$. This result applies to equally weighted observations and squared-error loss.

Why the Mean Minimizes Squared Error

To justify the result, consider a constant prediction $a$ and its total squared error on the training observations:

Q(a)=\sum_{i=1}^{n}(y_i-a)^2.

Differentiate with respect to $a$. Setting the derivative to zero gives the candidate minimum:

Q'(a)=-2\sum_{i=1}^{n}(y_i-a) =2na-2\sum_{i=1}^{n}y_i=0, \qquad a=\frac{1}{n}\sum_{i=1}^{n}y_i=\bar{y}.

The second derivative is $Q''(a)=2n$, which is positive because $n>0$. Thus $Q(a)$ is a convex quadratic, and its stationary point at $\bar{y}$ is its unique minimum. This derivation explains why the mean is the appropriate constant baseline when the objective is squared error; it does not say that the mean is optimal for every other error measure or decision objective.

Worked Example: Mean Baseline for Monthly Service Costs

A hypothetical service team records five monthly costs per covered unit in a training sample: $80, $100, $120, $140, and $160. Construct the historical-mean baseline and calculate its mean squared error and root mean squared error on these five observations.

Step 1: Add the observed costs. The total is $80+100+120+140+160=600$ dollars.

Step 2: Calculate the mean. There are $n=5$ observations, so $\bar{y}=600/5=\$120$. The baseline predicts $120 for each observation.

Step 3: Calculate each residual and squared residual. The residual is observed value minus prediction:

For $80: $80-120=-40$, and $(-40)^2=1{,}600$.
For $100: $100-120=-20$, and $(-20)^2=400$.
For $120: $120-120=0$, and $0^2=0$.
For $140: $140-120=20$, and $20^2=400$.
For $160: $160-120=40$, and $40^2=1{,}600$.

Step 4: Calculate the summaries. The squared residuals sum to $1{,}600+400+0+400+1{,}600=4{,}000$ dollars squared. Dividing by 5 gives a mean squared error of $4{,}000/5=800$ dollars squared. The root mean squared error is $\sqrt{800}\approx \$28.28$, rounded to two decimals.

Check. The residuals sum to $-40-20+0+20+40=0$, as expected when deviations are calculated from the mean. The method is to predict the training mean, then calculate squared residuals observation by observation. A common mistake is to report $4{,}000$ as the mean squared error; that is the sum of squared errors, not the mean, because it has not been divided by $n=5$.

Choosing and Describing the Baseline

A baseline is meaningful only when its construction matches the prediction task. State the response, the data used to calculate the rule, and what the constant prediction represents. For a classification task, say which class is always predicted and how that class was selected. For a numeric task, report the mean and its units. These details make the baseline interpretable without suggesting that it captures individual differences.

For model development, calculate the baseline from the designated training data, not from outcomes that should be unavailable when the prediction is made. If the baseline is being assessed on separate evaluation data, keep the prediction rule fixed and apply it there; do not use those evaluation outcomes to choose its class or recalculate its mean. Otherwise, information from the evaluation outcomes has been used to construct the supposed prediction. The same separation is needed when evaluating a more complex model.

The baseline itself is also sensitive to the observation unit. Suppose one data set contains one row per claim and another contains one row per policy. A mean claim amount per claim and a mean amount per policy are different summaries because their rows represent different quantities. Likewise, changing the time window or including a different population changes the data summarized. A precise description prevents a baseline from being mistaken for a universal rate or average.

Worked Example: A Tied Multiclass Baseline

A hypothetical claims triage data set has 40 training cases with one of three response labels: “routine” for 16 cases, “priority” for 16 cases, and “urgent” for 8 cases. A separate set of 10 cases is reserved for evaluation, and its labels are not available when the baseline is chosen. Specify a defensible majority-class baseline and explain what can be concluded from the training counts.

Step 1: Check the counts. The counts sum to $16+16+8=40$, matching the training-set size.

Step 2: Find the largest count. The largest count is 16, shared by “routine” and “priority.” There is no unique majority class.

Step 3: State a tie rule and the resulting prediction. If a fixed label-order rule specified before evaluation selects “priority” ahead of “routine,” the baseline predicts “priority” for every case. Under that rule, it matches 16 of the 40 training labels, giving a training fraction correct of $16/40=0.40$, or 40%. Choosing “routine” instead would give the same training fraction.

Check. Both tied classes occur 16 times, and $16/40=0.40$. The method is the majority-class rule with a prespecified tie-break. A common mistake is to inspect the 10 evaluation labels and choose whichever tied class is more common there. That would use evaluation outcomes to construct the baseline. Since those labels are unavailable at prediction time, the baseline must be fixed from the training information alone.

Common Mistakes and Limits

Using evaluation outcomes to set the baseline. Select the majority class or calculate the mean from the designated training data. Keep the rule fixed when applying it to separate evaluation cases.
Calling a constant rule individualized. The majority class and historical mean are the same for every case. They do not use predictors to distinguish cases.
Ignoring a tie. When the largest class counts tie, report the tie and state a prespecified rule if one prediction must be produced. Do not claim there is a unique most frequent class.
Confusing a sum with an average. For numeric outcomes, mean squared error divides the squared-error total by the number of observations. Keep the units in mind: squared error is in squared response units, while its square root returns to the response units.
Leaving the reference data vague. Identify the population, observation unit, response definition, and time period that the training mean or class count summarizes.

A baseline is useful because it gives a simple point of comparison and makes the response distribution visible. It is not proof that a future model will improve on it, nor does it establish whether a prediction is useful for a particular decision. Those conclusions require an evaluation design and success criteria appropriate to the task.

Key takeaway: A majority-class baseline always predicts a most frequent training class; a historical-mean baseline always predicts the training-response mean. Each is a transparent constant reference. State what data define it, handle classification ties explicitly, and do not use evaluation outcomes to construct it.

Check Your Understanding

For each question, identify the baseline rule and calculate the requested value.

A training set contains 30 binary outcomes: 21 “not renewed” and 9 “renewed.” What class does the majority baseline always predict, and what fraction of the training labels does it match? (Answer: “Not renewed”; $21/30=0.70$, or 70%.)
A numeric training response has values 4, 7, 9, and 12 units. What is the historical-mean baseline? (Answer: $(4+7+9+12)/4=8$ units.)
Five training costs are $10, $10, $20, $30, and $30. Find the mean baseline and its mean squared error on these observations. (Answer: Mean $=\$20$; squared errors are $100,100,0,100,100$, so mean squared error $=400/5=80$ dollars squared.)
A three-class training set has counts 14, 14, and 7. Is there a unique majority class? If a prespecified tie-break selects the first of the tied classes, what fraction of training cases does the resulting baseline match? (Answer: No unique majority class; the baseline matches $14/35=0.40$, or 40%.)
A team calculates a mean baseline from 80 training records and then recalculates the mean using those records plus 20 evaluation outcomes before scoring the evaluation set. Is the evaluation baseline constructed without using evaluation outcomes? (Answer: No; the 20 evaluation outcomes were used to set the prediction. Calculate the baseline from the 80 training records and keep it fixed for evaluation.)

What You'll Learn