Why Transform Positive, Right-Skewed Data?
In a right-skewed group, many observations may be relatively small while a few are much larger. Those large values can stretch a graph’s horizontal scale and strongly affect summaries such as the mean. A logarithm offers a way to describe the same positive observations on a different scale: it gives more room to distinguish smaller values and compresses the spacing among large ones.
A log transformation does not delete unusual observations or prove that they are errors. It changes the scale on which the observations are displayed and summarized. The transformed values may be less right-skewed, but this is not guaranteed; the shape must still be inspected. Here we use the natural logarithm, written \(\ln(x)\), and apply it only when \(x>0\).
Use the same logarithm base and the same original measurement units for all groups being compared. This tutorial uses natural logarithms. Another fixed base would also be possible, but it would put the transformed values on a differently scaled log axis. A transformed mean or median is therefore a summary on the log scale; it is not a mean or median in the original units.
How the Logarithm Compresses Values
The logarithm is increasing: if \(a<b\) and both are positive, then \(\ln(a)<\ln(b)\). Thus, transforming positive observations preserves their order. But the spacing changes. In particular, a fixed multiplicative change in the original values becomes a fixed additive difference on the log scale.
For example, doubling a value changes its natural logarithm by \(\ln(2)\), regardless of whether the value changes from 5 to 10 or from 500 to 1,000. By contrast, a fixed additive increase does not produce a fixed log-scale gap.
Worked Example: Equal Ratios, Unequal Original Gaps
Suppose a hypothetical group has four positive measurements: 10, 100, 1,000, and 10,000 units. Calculate their natural logarithms and compare the spacing before and after transformation.
The transformed observations, rounded to four decimal places, are:
On the original scale, the consecutive gaps are \(100-10=90\), \(1{,}000-100=900\), and \(10{,}000-1{,}000=9{,}000\) units. On the log scale, the consecutive gaps are approximately \(2.3026\), \(2.3026\), and \(2.3025\) log units. The small difference in the last displayed gap is due to rounding; each original value is ten times the previous one, so each exact transformed gap is \(\ln(10)\).
The transformation has not made the original measurements equal or removed the larger observations. It has represented equal tenfold changes by nearly equal distances, compressing the large absolute gaps.
A Short Justification for Compression
The ratio identity explains why multiplicative changes become additive gaps. There is also a direct way to see why a fixed positive increase makes a smaller log-scale gap when it is added to an already large value. If \(c>0\) is a fixed increase and \(x>0\), then:
As \(x\) increases, \(c/x\) decreases, so \(1+c/x\) decreases. Because the logarithm is increasing, \(\ln(1+c/x)\) also decreases. Therefore, adding the same amount \(c\) produces a smaller log-scale gap at a larger starting value. This is the compression of large values in precise form.
Comparing Group Summaries on the Log Scale
To compare groups after transformation, first transform each observation, then calculate the desired summaries from those transformed values. State clearly that the summaries are on the log scale. The following example uses equal-sized hypothetical groups whose observations are all positive.
Worked Example: Comparing Two Groups After a Log Transformation
Suppose hypothetical measurements from Group A are 2, 3, 4, 8, and 32 units. Measurements from Group B are 4, 6, 8, 16, and 64 units. The values in Group B are twice the corresponding values in Group A. Compare the transformed means and medians using natural logarithms.
All ten observations are positive, so each can be transformed. For Group A, the transformed values are approximately \(0.6931,\ 1.0986,\ 1.3863,\ 2.0794,\ 3.4657\). Their sum is \(8.7231\) using the displayed rounded values, giving:
For Group B, the transformed values are approximately \(1.3863,\ 1.7918,\ 2.0794,\ 2.7726,\ 4.1589\). Their sum is \(12.1890\), so:
The difference between the transformed means, calculated using unrounded logarithms, is approximately \(0.6931\) log units. This is \(\ln(2)\), consistent with every Group B observation being twice its paired Group A observation. The transformed means describe average log values; the difference is not a difference of \(0.6931\) original measurement units.
For the medians, each sorted list has five values, so its middle transformed value is the median. The transformed median for Group A is \(\ln(4)\approx1.3863\); for Group B it is \(\ln(8)\approx2.0794\). Their difference is approximately \(0.6931\) log units. These summaries locate the groups on the transformed scale; report them as such rather than labeling them in the original units.
The example illustrates a useful feature of the log scale: when one group is a constant multiple of another observation by observation, the transformed observations have a constant additive shift. A group comparison may consequently look different on the log scale than it did in the original units. Which scale is useful depends on the question and the measurement process; the transformation does not decide that choice by itself.
Domain and Measurement Units Matter
The natural logarithm is defined for positive numbers. In particular, \(\ln(0)\) is undefined, and the real-valued natural logarithm is also undefined for a negative number. Before transforming a group, check that every included observation is strictly positive. Do not silently omit zeros or negative values to make the calculation work.
Worked Example: A Zero and a Change of Units
Suppose a hypothetical group’s turnaround times are recorded as 0, 2, 5, and 20 hours. The three positive observations transform to \(\ln(2)\approx0.6931\), \(\ln(5)\approx1.6094\), and \(\ln(20)\approx2.9957\). But the zero observation has no natural logarithm, so the transformed group is not defined for all four original observations as stated.
Dropping the zero would change which observations are summarized. Adding a constant before transformation, such as using \(\ln(x+1)\), would be a different transformation and would require a substantive reason for choosing the constant and its unit. It should not be done automatically just to avoid an undefined value.
Units also affect the numerical log values. The positive times 2, 5, and 20 hours are 120, 300, and 1,200 minutes. For example, \(\ln(120)=\ln(2)+\ln(60)\), so the numerical log value changes when hours are replaced by minutes. For every observation converted using the same units, the same constant \(\ln(60)\) is added. Thus group comparisons must use consistent units; log values reported in one unit cannot be casually mixed with values reported in another.
Interpreting Results and Avoiding Common Mistakes
A log transformation can be useful when the question is naturally concerned with relative or multiplicative differences, or when a few very large positive observations dominate the display of smaller ones. It can make a right-skewed distribution easier to inspect, but does not guarantee symmetry or make every statistical summary appropriate. Examine the transformed observations and explain which scale your comparison uses.
- Mistake: “The log removed the outlier.” The observation is still present and remains the largest if it was largest originally. Its distance from other values has changed. An unusual observation should still be investigated in context.
- Mistake: “A transformed mean is in the original units.” A mean computed from \(\ln(x)\) values is on the log scale. Label the scale and do not describe a log-scale difference as a number of original units.
- Mistake: “A log transformation always fixes right skew.” It often compresses a long upper tail, but the resulting shape depends on the data. Check the transformed distribution rather than assuming it is symmetric.
- Mistake: “Zeros can be logged if they are unimportant.” Zero has no real natural logarithm. Decide how to handle it using the context and a clearly justified analysis plan; do not omit it without explanation.
Check Your Understanding
Answer each question using the ideas and calculations in this tutorial.
- Why do observations of 5 and 10 have the same log-scale gap as observations of 500 and 1,000?
- A positive observation is multiplied by 3. What is the change in its natural logarithm?
- In a group comparison, the mean of the transformed values in Group A is 1.8 and in Group B is 2.5. What scale are these means on, and in what units should their difference be reported?
- What should an analyst check before taking the natural logarithm of every observation in a group?
- Explain why changing measurements from hours to minutes changes the numerical log values, and why comparisons should use a consistent unit.