Root Mean Squared Logarithmic Error

Why Use a Logarithmic Error Metric?

A prediction that misses an outcome by 10 units can have very different significance depending on the scale: it is a large miss for an outcome near 5 and a relatively small one for an outcome near 1,000. Root mean squared logarithmic error, or RMSLE, compares outcomes on a transformed scale. It can be useful when proportional differences matter more than absolute differences, provided that the observed values and predictions are nonnegative.

RMSLE is calculated from paired observations and predictions. It does not require a particular fitted model or probability distribution. Its interpretation does depend on the measurement units, because the standard formula adds one to each value before taking logarithms. That detail is especially important when outcomes can be zero.

Definition: For \(n\) observed outcomes \(y_1,\ldots,y_n\) and corresponding predictions \(\hat{y}_1,\ldots,\hat{y}_n\), the root mean squared logarithmic error is

\operatorname{RMSLE} = \sqrt{ \frac{1}{n} \sum_{i=1}^{n} \left[ \ln(1+\hat{y}_i)-\ln(1+y_i) \right]^2 }.

The usual definition requires \(y_i\geq 0\) and \(\hat{y}_i\geq 0\) for every observation. The logarithm is the natural logarithm, and each observed value is paired with its prediction.

What the Logarithmic Difference Measures

For observation \(i\), the signed logarithmic error is \(e_i=\ln(1+\hat{y}_i)-\ln(1+y_i)\). The logarithm rule gives an equivalent expression:

e_i = \ln\left(\frac{1+\hat{y}_i}{1+y_i}\right).

Thus, the error is the logarithm of a ratio of values after adding one. If the adjusted prediction is twice the adjusted observation, then \(e_i=\ln 2\), regardless of the observation's particular size. If the adjusted prediction is half the adjusted observation, then \(e_i=-\ln 2\). Since RMSLE squares each error, those two adjusted ratios contribute equally.

The added one lets the formula include zero: \(\ln(1+0)=0\). It does not mean that an outcome of zero is treated as if its true value were one. Rather, it defines a transformation that is finite at zero. The resulting measure is not exactly a percentage error: its ratio is \((1+\hat{y}_i)/(1+y_i)\), not \(\hat{y}_i/y_i\).

Why the Formula Uses a Square and a Square Root

The transformed errors \(e_i\) can be positive or negative. Averaging them directly would allow overpredictions and underpredictions to cancel. Squaring makes every contribution nonnegative and gives larger transformed errors more influence. Taking the square root returns the measure to the scale of the logarithmic errors, rather than leaving it in squared-log units.

The formula follows by applying the usual root-mean-square construction to the transformed observations. Define \(z_i=\ln(1+y_i)\) and \(\hat{z}_i=\ln(1+\hat{y}_i)\). The difference on that transformed scale is \(\hat{z}_i-z_i\). Its mean squared value is the sum of squared differences divided by \(n\); taking its square root gives RMSLE. Substituting the definitions of \(z_i\) and \(\hat{z}_i\) yields the displayed formula.

This justification also shows what is being averaged: each paired observation contributes one squared transformed error, with equal weight. Unless a different weighted metric is explicitly defined, a row with a large outcome does not automatically receive more weight just because its outcome is larger.

Worked Example: Calculating RMSLE for Three Work Areas

A hypothetical maintenance team records the number of equipment interruptions at three work areas. The observed counts are 5, 17, and 0; the corresponding predicted counts are 8, 12, and 0. Calculate RMSLE, showing each transformed error and its square.

Method. Use the paired observations in the stated order, calculate \(\ln(1+\hat{y}_i)-\ln(1+y_i)\) for each pair, square the result, average the three squares, and take the square root.

For the first area, \(y_1=5\) and \(\hat{y}_1=8\):

e_1=\ln(9)-\ln(6)=\ln(1.5)\approx 0.4054651081, \qquad e_1^2\approx 0.1644019539.

For the second area, \(y_2=17\) and \(\hat{y}_2=12\):

e_2=\ln(13)-\ln(18)=\ln(13/18)\approx -0.3254224004, \qquad e_2^2\approx 0.1058997282.

For the third area, both values are zero, so \(e_3=\ln(1)-\ln(1)=0\), and \(e_3^2=0\). Therefore,

\operatorname{RMSLE} = \sqrt{\frac{0.1644019539+0.1058997282+0}{3}} = \sqrt{0.0901005607} \approx 0.3001676.

Rounded to four decimal places, RMSLE is 0.3002. As a check, all three squared contributions are nonnegative, and the zero-zero pair contributes no error. The method avoids the mistake of omitting a zero observation or attempting to take the logarithm of zero without first adding one.

Interpreting RMSLE and Its Limits

A smaller RMSLE indicates smaller average squared differences on the log-transformed scale for the data being evaluated. It does not express the typical error in the original outcome units. For example, an RMSLE of 0.3 is not a statement that predictions are off by 0.3 units or by 30 percent.

The ratio expression helps explain the metric's emphasis. For positive values much larger than one, adding one changes their ratio relatively little, so the transformed difference is close to \(\ln(\hat{y}_i/y_i)\). In that setting, similar multiplicative errors can have similar log errors even when the raw-unit differences are very different. Near zero, however, the added one matters substantially, and it would be misleading to interpret RMSLE as a pure percentage-error measure.

The measurement unit also matters. Changing outcomes from dollars to cents changes their numerical values, and therefore changes the effect of the added one. RMSLE values calculated on different unit conventions are not automatically comparable. Choose and report a consistent unit that makes sense for the outcome before evaluating predictions.

Worked Example: Comparing Two Prediction Sets

For a hypothetical two-item inventory, the observed quantities are 4 and 10. Model A predicts 8 and 10; Model B predicts 10 and 8. Calculate each RMSLE and decide which is smaller. The prediction sets contain the same two predicted values, but those values are paired with different observations.

Method. Calculate the transformed error for each observation-prediction pair. A metric must preserve the pairing; it cannot be calculated by comparing the two prediction sets without their corresponding observations.

For Model A, the errors are \(\ln(9)-\ln(5)=\ln(9/5)\) and \(\ln(11)-\ln(11)=0\). Hence,

\operatorname{RMSLE}_A = \sqrt{\frac{[\ln(9/5)]^2+0^2}{2}} \approx \sqrt{\frac{0.3454894603}{2}} \approx 0.4156.

For Model B, the errors are \(\ln(11)-\ln(5)=\ln(11/5)\) and \(\ln(9)-\ln(11)=\ln(9/11)\). Thus,

\operatorname{RMSLE}_B = \sqrt{ \frac{[\ln(11/5)]^2+[\ln(9/11)]^2}{2} } \approx \sqrt{\frac{0.6216650091+0.0402687280}{2}} \approx 0.5753.

Model A has the smaller RMSLE. The comparison is a useful check against assuming that swapping predictions must leave the metric unchanged: the predictions are scored against specific observations, and the resulting adjusted ratios are not the same. The calculation avoids the common mistake of treating two sets of predictions as interchangeable without preserving their pairings.

Zeros, Invalid Inputs, and Implementation

Zero observed values and zero predictions are valid under the standard RMSLE definition. If both are zero, that pair contributes zero. If one is zero and the other is positive, the contribution is finite; for example, an observed value of zero and a prediction of 3 produce an error of \(\ln(4)-\ln(1)=\ln 4\).

Negative observed values or predictions do not meet the usual nonnegative-outcome requirement. Some negative numbers greater than \(-1\) have a defined \(\ln(1+x)\), but that mathematical fact does not make them valid inputs for the standard nonnegative-outcome use of RMSLE. A prediction below \(-1\) makes the real-valued logarithm undefined. Do not silently replace negative values with zero: that changes the predictions and the metric.

In R, for finite, nonnegative vectors named actual and pred with matching lengths, the expression sqrt(mean((log1p(pred) - log1p(actual))^2)) calculates RMSLE. The function log1p(x) calculates \(\ln(1+x)\) and is designed to handle values near zero accurately. Before calculating, check that the vectors are paired in the correct order, have equal lengths, and contain no missing, infinite, or negative values.

Common Mistakes and Exam Checks

Using \(\ln(y)\) instead of \(\ln(1+y)\). The standard formula includes one so that zero outcomes are valid.
Using base-10 logarithms. RMSLE uses the natural logarithm. A different base changes the numerical value.
Forgetting to square, average, or take the square root. Each operation is part of the definition; averaging the absolute log errors or the signed errors gives a different metric.
Pairing predictions incorrectly. Each prediction must be compared with its own observed outcome before the errors are aggregated.
Calling RMSLE a percentage. It is an average root-square measure of adjusted log ratios, not a percentage error in the original units.
Ignoring units or invalid inputs. The added one makes scale choices relevant, and negative inputs are not valid for the standard nonnegative-outcome metric.

When reporting RMSLE, state the unit convention, the evaluation observations, and whether all outcomes and predictions were nonnegative. This makes the calculation reproducible and helps readers judge whether a log-based metric is appropriate for the decision at hand.

Check Your Understanding

Work each problem using the standard natural-log RMSLE definition. Answers are rounded as stated where needed.

A prediction is 19 when the observed value is 9. Calculate RMSLE for this single pair. (Answer: \(\ln 2\approx 0.6931\).)
Observed values are 0 and 3; predictions are 0 and 7. Calculate RMSLE to four decimal places. (Answer: \(\ln 2/\sqrt{2}\approx 0.4901\).)
Observed values are 2 and 8; predictions are 5 and 8. Calculate RMSLE to four decimal places. (Answer: \(\ln 2/\sqrt{2}\approx 0.4901\).)
Observed values are 4 and 10; predictions are 5 and 12. Calculate RMSLE to four decimal places. (Answer: \(\sqrt{([\ln(6/5)]^2+[\ln(13/11)]^2)/2}\approx 0.1749\).)
A data set has observed values 0 and 4, with predictions \(-1\) and 4. Is the standard RMSLE valid without changing the predictions? Explain the input issue. (Answer: No; the prediction \(-1\) is negative and \(\ln(1-1)=\ln(0)\) is undefined.)

What You'll Learn