Why Chart Annotations Matter
A performance chart can be numerically accurate and still be difficult to interpret. A line labelled “Model B” does not tell a reader what was predicted, how performance was measured, or whether a lower or higher value is preferable. An axis marked “Score” may hide whether the values are dollars, percentages, or counts. If the chart is copied into a slide without its surrounding explanation, those omissions become more serious.
Chart annotations supply the context needed to read the plotted values correctly. They do not change the model results; they identify what the results represent and make relevant comparisons easier to see. The goal is not to add every detail to the graphic. It is to include the essential context in a clear hierarchy: title, axes, units, legend, and carefully chosen callouts. This reporting practice is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
A Practical Annotation Framework
A useful chart title does more than name a metric. It briefly states what is being compared and under what evaluation context. For example, “Validation RMSE by quarter, reference and candidate models” is more informative than “RMSE.” A subtitle or note can supply details that would make the title too long, such as the outcome being predicted or the validation period.
Axes should identify both the quantity and its unit. Use “False-positive rate” rather than “FPR” for readers who may not know the abbreviation, and include a percentage scale if the values are displayed as percentages. For a loss prediction chart, label the axis “RMSE (thousand dollars per claim),” not simply “Error.” State whether higher or lower values indicate better performance when that direction is not obvious to the intended reader.
Legends should use model names that remain meaningful outside the analysis file. “Candidate model” and “Reference model” are more informative than internal labels such as “fit_04” and “fit_02.” If the chart has only one series, a legend may be unnecessary. Direct labels placed near lines or bars can be clearer than a legend, provided the labels do not overlap the data.
Callouts are short notes that draw attention to a specific, calculated feature of the chart. A useful callout names the comparison and, where appropriate, its calculation. It should not imply that the chart proves a cause, nor should it make a result sound more general than its validation context allows. Keep a callout close to the point it describes, and ensure that it does not cover data.
State what outcome and model results are being shown, and name the validation period or population needed to interpret them.
Name each axis, give its units or percentage scale, and clarify whether higher or lower performance is preferable.
Use readable model names and explain diagonal lines, targets, or other reference marks.
Check the arithmetic, state the comparison precisely, and avoid claims that go beyond the plotted evidence.
Scale and Comparison Context
The axis scale affects how readers perceive differences. It must not conceal relevant information or exaggerate a small difference. For bar charts, the usual rule is to start the value axis at zero because bar length represents magnitude. A truncated baseline can make a modest difference look much larger than the values justify.
This can be shown directly. Suppose two bars represent values \(90\) and \(100\). With a zero baseline, their lengths are proportional to \(90\) and \(100\), so the ratio of lengths is \(90/100=0.9\). If the displayed axis starts at \(80\), the bar lengths above the baseline are \(90-80=10\) and \(100-80=20\). The apparent ratio becomes \(10/20=0.5\), even though the value ratio remains \(0.9\). The truncated axis therefore changes the visual comparison.
A line chart has a different purpose: position and slope often show change across ordered values, rather than encoding magnitude through bar length. A line chart may use a nonzero axis minimum when that scale is appropriate, but the tick marks must be clear and the displayed range must not be hidden. If a narrow range is used to make small movements visible, label it plainly and consider providing values at the plotted points. Do not choose a scale solely because it makes one model look better.
Comparison context also includes the evaluation setup. Two performance values are meaningful as a model comparison only when readers can tell whether they refer to the same outcome, metric definition, and validation context. If they do not, the chart title or a concise note should say what differs. In particular, a callout should not silently compare results from different populations or periods as if they were directly equivalent.
Worked Chart Annotations
Worked Example: Comparing Validation RMSE Across Periods
A hypothetical insurer evaluates a reference model and a candidate model on the same held-out claims in each of three periods. RMSE is reported in thousands of dollars per claim, and lower RMSE indicates smaller typical prediction errors on this scale. The values are: reference model \(2.40, 2.32, 2.28\); candidate model \(2.10, 2.08, 2.04\). Specify the annotations for a line chart and calculate an appropriate callout for the final period.
Method. First identify the chart’s comparison and metric, then calculate the relative reduction using the reference value as the denominator. This avoids the common mistake of describing a raw difference as a percentage.
Title and axes. Use “Validation RMSE by period: reference and candidate claims models” as the title. Label the horizontal axis “Validation period” with the three period names. Label the vertical axis “RMSE (thousand dollars per claim); lower is better.” Since the values range from \(2.04\) to \(2.40\), a line-chart axis from \(1.8\) to \(2.6\) can show variation while retaining visible context. Show the tick marks and point values so the range is not ambiguous.
Legend. Use “Reference model” and “Candidate model.” Both series use the same validation observations within each period, which is relevant comparison context and can be stated in a brief note.
Callout calculation. In the final period, the absolute RMSE difference is \(2.28-2.04=0.24\) thousand dollars per claim, or \(240\) dollars per claim. Relative to the reference RMSE, the reduction is
Round the percentage to one decimal place and label the callout “Final-period RMSE was 10.5% lower than the reference model.” This is a descriptive comparison of these validation results, not a claim about every future period. A suitable chart should be checked to confirm that the final-period points are \(2.28\) and \(2.04\), and that the stated reduction uses the reference value in the denominator.
Worked Example: Annotating ROC Operating Points
A hypothetical lender compares two probability models on separate validation samples of \(1{,}000\) applicants each. Each sample has \(200\) applicants who experienced the defined event and \(800\) who did not. At a probability cutoff of \(0.30\), the reference model has \(100\) true positives, \(100\) false negatives, \(160\) false positives, and \(640\) true negatives. The candidate model has \(120\), \(80\), \(160\), and \(640\), respectively. Determine the plotted operating points for a receiver operating characteristic chart and specify its annotations.
Method. A ROC chart plots the true-positive rate against the false-positive rate. Calculate each rate from its own relevant group: true positives plus false negatives for the true-positive rate, and false positives plus true negatives for the false-positive rate. This avoids using the total sample size as the denominator for both rates.
For the reference model, the true-positive rate is \(100/(100+100)=100/200=0.50\), or \(50\%\). Its false-positive rate is \(160/(160+640)=160/800=0.20\), or \(20\%\). Its operating point is therefore \((20\%,50\%)\), where the first coordinate is the false-positive rate.
For the candidate model, the true-positive rate is \(120/(120+80)=120/200=0.60\), or \(60\%\). Its false-positive rate is \(160/(160+640)=160/800=0.20\), or \(20\%\). Its operating point is \((20\%,60\%)\).
Title the chart “Validation ROC curves for event prediction.” Label the horizontal axis “False-positive rate” and the vertical axis “True-positive rate,” both with percentage tick marks from \(0\%\) to \(100\%\). Label the model curves in the legend, identify the diagonal reference line as “No-discrimination reference,” and mark each operating point with a callout stating “Cutoff 0.30.” A concise comparison is “At the displayed cutoff, the candidate point has a 10 percentage-point higher true-positive rate at the same 20% false-positive rate.” The arithmetic is \(60\%-50\%=10\) percentage points; it is not a \(10\%\) relative increase. Confirm that the chart uses the same event definition and cutoff for the two displayed points.
Worked Example: Showing Segment RMSE Without Hiding Exposure
A hypothetical service team compares prediction errors across three claim-size groups. The validation results are:
- Group A: \(400\) claims and sum of squared errors \(144{,}000{,}000\) dollars squared.
- Group B: \(250\) claims and sum of squared errors \(90{,}000{,}000\) dollars squared.
- Group C: \(100\) claims and sum of squared errors \(49{,}000{,}000\) dollars squared.
Calculate the RMSE for each group and overall, then describe how to annotate a bar chart. RMSE is the square root of the mean squared error, where the mean is taken over the observations being summarized.
Method. Divide each group’s sum of squared errors by its claim count, then take the square root. For the overall value, pool the sums of squared errors and counts before taking the square root. This avoids the common mistake of taking an unweighted average of subgroup RMSE values.
Group A has mean squared error \(144{,}000{,}000/400=360{,}000\) dollars squared, so its RMSE is \(\sqrt{360{,}000}=600\) dollars per claim. Group B has mean squared error \(90{,}000{,}000/250=360{,}000\), so its RMSE is also \(600\) dollars per claim. Group C has mean squared error \(49{,}000{,}000/100=490{,}000\), so its RMSE is \(700\) dollars per claim.
The pooled sum of squared errors is \(144{,}000{,}000+90{,}000{,}000+49{,}000{,}000=283{,}000{,}000\). The total count is \(400+250+100=750\). Therefore the pooled RMSE is
The simple average of the three subgroup RMSEs would be \((600+600+700)/3=633.33\) dollars, which is not the pooled RMSE because the groups have different counts. Round the pooled result to cents only at the final step.
For the chart, use the title “Validation RMSE by claim-size group.” Label the vertical axis “RMSE (dollars per claim); lower is better,” start its bar-chart scale at zero, and label the bars “Group A,” “Group B,” and “Group C.” Add the sample counts beside the group labels or in a clearly linked note: \(n=400\), \(n=250\), and \(n=100\). If the pooled value is shown as a reference line, label it “Pooled RMSE: 614.27 dollars per claim.” The counts prevent readers from treating estimates based on differently sized groups as if they had identical sample sizes.
Callouts, Legends, and Final Checks
An annotation should help the reader notice a meaningful feature, not narrate every mark. A callout may identify a selected operating point, a reference value, or a percentage change, but its wording should match the computation. A difference of \(10\) percentage points is not the same as a relative increase of \(10\%\). If a callout gives a relative change, state the comparison baseline and use the correct denominator.
Use color consistently, but do not rely on color alone to distinguish models. Pair color with line style, point shape, or direct labels when practical. Keep the legend close enough to the chart to make the mapping clear. If a reference line is present, explain what it represents; an unexplained target or diagonal may be mistaken for another model result.
Before finalizing a chart, check that a reader can identify the predicted outcome, the metric and its direction, the validation context, the units, every plotted series, and the meaning of any reference mark. Then check the visual scale and recalculate each callout from the displayed values. These checks are particularly important when a chart is separated from its report or presented to decision-makers who did not participate in the analysis.
Check Your Understanding
For each scenario, identify the annotation or calculation that would make the chart interpretable and avoid the stated comparison trap.
- A line chart shows quarterly mean absolute error values of \(18\), \(16\), and \(15\) thousand dollars for a candidate model, with lower values better. State a suitable vertical-axis label and calculate the relative reduction from the first to the final quarter using the first quarter as the baseline. (Answer: “Mean absolute error (thousand dollars); lower is better”; reduction \(=(18-15)/18=16.7\%\).)
- A ROC operating point has \(45\) true positives, \(15\) false negatives, \(80\) false positives, and \(320\) true negatives. Give its coordinates in the order false-positive rate, true-positive rate. (Answer: \((20\%,75\%)\).)
- A bar chart compares values \(72\) and \(80\). The value axis starts at \(60\). What ratio do the displayed bar lengths suggest, and what is the actual value ratio? (Answer: displayed ratio \(=(72-60)/(80-60)=0.60\); actual ratio \(=72/80=0.90\).)
- Two segments have RMSEs of \(500\) dollars based on \(100\) observations and \(700\) dollars based on \(300\) observations. Calculate the pooled RMSE if the subgroup mean squared errors are respectively \(250{,}000\) and \(490{,}000\) dollars squared. (Answer: \(\sqrt{(100(250{,}000)+300(490{,}000))/400}=\sqrt{430{,}000}\approx655.74\) dollars.)
- A candidate model’s final-period RMSE is \(1.8\) thousand dollars and the reference model’s is \(2.0\) thousand dollars. Write a precise relative-reduction callout using the reference as baseline. (Answer: “Final-period RMSE was 10.0% lower than the reference model,” since \((2.0-1.8)/2.0=10.0\%\).)