Micro and Macro Error Averages

Why Average Errors Across Groups?

A prediction model is often assessed across distinct groups, such as product types, regions or customer segments. Two summaries can answer different questions: how well did the model perform across all observations combined, and how well did it perform for a typical group? The first gives larger groups more influence; the second gives each group equal influence. These are the micro and macro perspectives.

Neither summary is universally preferable. The right choice depends on the decision being supported. A pooled result may be appropriate when each observation should count equally. An equal-group result can make small groups more visible, but it gives a small group the same influence as a large one. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.

Definition: Suppose \(G\) nonempty groups partition \(N\) observations, with \(n_g\) observations in group \(g\), so \(N=\sum_{g=1}^G n_g\). Let \(y_{gi}\) be the observed value and \(\hat{y}_{gi}\) its prediction. Define the error as \(e_{gi}=y_{gi}-\hat{y}_{gi}\); therefore, a positive error means underprediction. A micro-average pools the observations before calculating a metric. A macro-average calculates that metric separately in each group, then takes the unweighted mean of the \(G\) group metrics.

The formulas below assume every observation is counted equally, groups are nonoverlapping, and every group included in the macro-average contains at least one observation. If groups have unequal observation counts, the two averaging rules generally differ. We use ME for mean error, MAE for mean absolute error, MSE for mean squared error, and RMSE for root mean squared error.

Definitions and the Weighting Difference

For group \(g\), calculate each metric from that group’s \(n_g\) errors. The micro versions calculate the corresponding metric from all \(N\) errors at once. The macro versions average the group-specific metrics with weight \(1/G\) per group.

\begin{aligned} \mathrm{ME}_g &= \frac{1}{n_g}\sum_{i=1}^{n_g} e_{gi}, &\quad \mathrm{ME}_{\text{micro}} &= \frac{1}{N}\sum_{g=1}^G\sum_{i=1}^{n_g} e_{gi}, &\quad \mathrm{ME}_{\text{macro}} &= \frac{1}{G}\sum_{g=1}^G \mathrm{ME}_g,\\ \mathrm{MAE}_g &= \frac{1}{n_g}\sum_{i=1}^{n_g}|e_{gi}|, &\quad \mathrm{MAE}_{\text{micro}} &= \frac{1}{N}\sum_{g=1}^G\sum_{i=1}^{n_g}|e_{gi}|, &\quad \mathrm{MAE}_{\text{macro}} &= \frac{1}{G}\sum_{g=1}^G \mathrm{MAE}_g,\\ \mathrm{MSE}_g &= \frac{1}{n_g}\sum_{i=1}^{n_g}e_{gi}^{\,2}, &\quad \mathrm{MSE}_{\text{micro}} &= \frac{1}{N}\sum_{g=1}^G\sum_{i=1}^{n_g}e_{gi}^{\,2}, &\quad \mathrm{MSE}_{\text{macro}} &= \frac{1}{G}\sum_{g=1}^G \mathrm{MSE}_g. \end{aligned}

For RMSE, the micro version is the square root of the pooled MSE. The macro version, by the definition of macro-averaging, is the mean of the group RMSEs. These are different operations and should not be interchanged.

\mathrm{RMSE}_{\text{micro}}=\sqrt{\mathrm{MSE}_{\text{micro}}}, \qquad \mathrm{RMSE}_{\text{macro}}=\frac{1}{G}\sum_{g=1}^G\sqrt{\mathrm{MSE}_g}.

Why Micro-Averages Weight Groups by Size

The pooled calculation can be rewritten in terms of group metrics. This shows precisely how group size affects micro-averaging.

Derivation: For any metric that is an average of per-observation quantities \(z_{gi}\), the pooled average is \(\frac{1}{N}\sum_g\sum_i z_{gi}\). The group average is \(\bar{z}_g=\frac{1}{n_g}\sum_i z_{gi}\), so \(\sum_i z_{gi}=n_g\bar{z}_g\). Substituting gives \(\frac{1}{N}\sum_g n_g\bar{z}_g=\sum_g\frac{n_g}{N}\bar{z}_g\). Thus micro ME, MAE and MSE are weighted averages of their group values, with weights \(n_g/N\). The weights sum to one because \(\sum_g n_g=N\). Micro RMSE is the square root of this weighted average of group MSEs; it is not a weighted average of group RMSEs.

Macro-averaging instead uses the same weight \(1/G\) for every group, regardless of its size. As a result, an error metric can be lower under the micro rule but higher under the macro rule. ME also allows positive and negative errors to cancel; MAE and MSE do not. The aggregation rule does not change that feature of the underlying metric.

Worked Example: Calculating Four Metrics Across Unequal Groups

A hypothetical forecasting review has two groups. Group A has errors \(1,-1,2,0\); Group B has errors \(3,-1\). Calculate micro and macro ME, MAE, MSE and RMSE. Report errors and RMSE to four decimal places where needed. Method: first calculate every group metric, then pool the observations for micro metrics and average the group metrics for macro metrics. This avoids treating the two groups as equal-sized in the pooled calculation.

Group A: It has \(n_A=4\) observations. The error sum is \(1-1+2+0=2\), the absolute-error sum is \(1+1+2+0=4\), and the squared-error sum is \(1+1+4+0=6\). Therefore, \(\mathrm{ME}_A=2/4=0.5\), \(\mathrm{MAE}_A=4/4=1\), \(\mathrm{MSE}_A=6/4=1.5\), and \(\mathrm{RMSE}_A=\sqrt{1.5}\approx1.2247\).

Group B: It has \(n_B=2\). The error sum is \(3-1=2\), the absolute-error sum is \(3+1=4\), and the squared-error sum is \(9+1=10\). Thus \(\mathrm{ME}_B=2/2=1\), \(\mathrm{MAE}_B=4/2=2\), \(\mathrm{MSE}_B=10/2=5\), and \(\mathrm{RMSE}_B=\sqrt{5}\approx2.2361\).

There are \(N=4+2=6\) observations. The pooled error sum is \(2+2=4\), so \(\mathrm{ME}_{\text{micro}}=4/6=0.6667\). The pooled absolute-error sum is \(4+4=8\), giving \(\mathrm{MAE}_{\text{micro}}=8/6=1.3333\). The pooled squared-error sum is \(6+10=16\), so \(\mathrm{MSE}_{\text{micro}}=16/6=2.6667\) and \(\mathrm{RMSE}_{\text{micro}}=\sqrt{16/6}\approx1.6330\).

For the macro results, give each of the two groups weight \(1/2\): \(\mathrm{ME}_{\text{macro}}=(0.5+1)/2=0.75\), \(\mathrm{MAE}_{\text{macro}}=(1+2)/2=1.5\), and \(\mathrm{MSE}_{\text{macro}}=(1.5+5)/2=3.25\). The macro RMSE is \((\sqrt{1.5}+\sqrt{5})/2\approx(1.2247+2.2361)/2=1.7304\).

Check: The micro metrics are weighted toward Group A, which has four of the six observations. The macro metrics give Group A and Group B equal influence. Also, \(\sqrt{\mathrm{MSE}_{\text{macro}}}=\sqrt{3.25}\approx1.8028\), which is not the macro RMSE \(1.7304\); taking a square root after averaging is a different calculation from averaging the group square roots.

Group Size Can Change the Summary

The difference is especially clear when one group is much smaller than another. A large group can dominate a micro result because it supplies more observations. Under a macro summary, the small group can have substantial influence even if it contains few observations. That is a property of the chosen summary, not an arithmetic error.

Worked Example: A Small Group With Larger Errors

In a hypothetical portfolio review, Group S has two prediction errors, \(2,2\). Group L has six errors, \(0,0,0,0,0,4\). Calculate micro and macro ME, MAE and RMSE. Round to four decimal places. Method: compute group-level totals before pooling, then use group size for micro metrics and equal weights for macro metrics. This avoids giving each group the same weight when the question asks for performance per observation.

For Group S, \(n_S=2\). The error sum and absolute-error sum are both \(4\), and the squared-error sum is \(2^2+2^2=8\). Therefore, \(\mathrm{ME}_S=2\), \(\mathrm{MAE}_S=2\), \(\mathrm{MSE}_S=8/2=4\), and \(\mathrm{RMSE}_S=2\).

For Group L, \(n_L=6\). Its error sum is \(4\), its absolute-error sum is \(4\), and its squared-error sum is \(4^2=16\). Therefore, \(\mathrm{ME}_L=4/6=0.6667\), \(\mathrm{MAE}_L=4/6=0.6667\), \(\mathrm{MSE}_L=16/6=2.6667\), and \(\mathrm{RMSE}_L=\sqrt{8/3}\approx1.6330\).

Across all eight observations, the error sum is \(4+4=8\), so \(\mathrm{ME}_{\text{micro}}=8/8=1\). The absolute-error sum is also \(8\), giving \(\mathrm{MAE}_{\text{micro}}=8/8=1\). The squared-error sum is \(8+16=24\), so \(\mathrm{MSE}_{\text{micro}}=24/8=3\) and \(\mathrm{RMSE}_{\text{micro}}=\sqrt{3}\approx1.7321\).

For two equally weighted groups, \(\mathrm{ME}_{\text{macro}}=(2+4/6)/2=4/3\approx1.3333\) and \(\mathrm{MAE}_{\text{macro}}=(2+4/6)/2=4/3\approx1.3333\). Also, \(\mathrm{MSE}_{\text{macro}}=(4+8/3)/2=10/3\approx3.3333\), while \(\mathrm{RMSE}_{\text{macro}}=(2+\sqrt{8/3})/2\approx1.8165\).

Check: Group S contributes only two of the eight observations, so it has one-quarter of the micro weight. It has one-half of the macro weight. The higher macro MAE therefore reflects the greater influence assigned to Group S, not a change in any observation’s error.

Choosing an Aggregation Rule for a Model Comparison

The averaging rule can affect which model appears preferable. To see why, compare models using the same two groups and the same metric. A model with low error in a large group but high error in a small group may look good under a pooled score and less good under an equal-group score. Conversely, a model that performs consistently across groups may be favored by the macro result.

Worked Example: Opposite Model Rankings Under Micro and Macro MAE

A hypothetical insurer compares two forecasting models. Group A has two observations, and Group B has eight. Model P has errors \(3,3\) in A and eight zero errors in B. Model Q has two zero errors in A and eight errors of \(1\) in B. Calculate each model’s micro and macro MAE, then state which model each summary favors. Method: calculate absolute errors within each group before aggregating. This avoids confusing a group’s observation count with the equal group weight used in a macro score.

For Model P, Group A has absolute-error sum \(3+3=6\), so its MAE is \(6/2=3\). Group B has absolute-error sum \(0\), so its MAE is \(0/8=0\). The pooled absolute-error sum is \(6\) over \(10\) observations, giving \(\mathrm{MAE}_{\text{micro}}=6/10=0.6\). The equal-group result is \(\mathrm{MAE}_{\text{macro}}=(3+0)/2=1.5\).

For Model Q, Group A has absolute-error sum \(0\), giving MAE \(0/2=0\). Group B has absolute-error sum \(1+1+1+1+1+1+1+1=8\), giving MAE \(8/8=1\). Its pooled MAE is \(8/10=0.8\), and its macro MAE is \((0+1)/2=0.5\).

The micro comparison favors Model P because \(0.6<0.8\). The macro comparison favors Model Q because \(0.5<1.5\). Check: Model P has smaller total absolute error across all observations, whereas Model Q has the smaller average of the two group MAEs. The rankings differ because the summaries answer different questions.

Common Mistakes and Interpretation

Taking an unweighted mean of group metrics and calling it micro. In the second example, the incorrect calculation \((2+0.6667)/2=1.3333\) is the macro MAE. The micro MAE is \((4+4)/8=1\), because the groups contain two and six observations.
Taking the square root at the wrong stage. Macro RMSE is the mean of the group RMSEs. It is not generally \(\sqrt{\mathrm{MSE}_{\text{macro}}}\). State which quantity is requested and apply that definition.
Forgetting cancellation in ME. A group’s positive and negative errors can sum to zero even when its absolute errors are not zero. ME describes signed bias; MAE and MSE describe error magnitude differently.
Reporting a metric without its averaging rule. “MAE is 1.2” is incomplete if group weighting matters. Label the result micro or macro, and explain what receives equal influence.

Key takeaway: Micro-averaging weights observations equally, so group \(g\) receives weight \(n_g/N\). Macro-averaging weights groups equally, so each receives weight \(1/G\). Calculate the metric at the level requested, and do not assume the two summaries lead to the same conclusion.

Check Your Understanding

For each problem, use \(e=y-\hat{y}\). Give answers to four decimal places where needed.

Group A has errors \(1,-1\), and Group B has error \(2\). Find micro ME and macro ME. (Answer: micro ME \(=0.6667\); macro ME \(=1\).)
Group A has one error \(4\), and Group B has three errors \(0,0,0\). Find micro MAE and macro MAE. (Answer: micro MAE \(=1\); macro MAE \(=2\).)
Group A has errors \(1,3\), and Group B has errors \(2,2\). Find micro RMSE and macro RMSE. (Answer: micro RMSE \(=\sqrt{4.5}\approx2.1213\); macro RMSE \(=(\sqrt{5}+2)/2\approx2.1180\).)
Group A has errors \(-2,-2\), and Group B has four errors \(1,1,1,1\). Find micro ME and macro ME. (Answer: micro ME \(=0\); macro ME \(=-0.5\).)
A has two observations and B has four. Model P has errors \(0,0\) in A and \(2,2,2,2\) in B. Model Q has errors \(3,3\) in A and \(0,0,0,0\) in B. Compare their micro and macro MAEs. (Answer: micro MAE is \(1.3333\) for P and \(1\) for Q, so micro favors Q; macro MAE is \(1\) for P and \(1.5\) for Q, so macro favors P.)

What You'll Learn