Jensen's Inequality and Log-Scale Mean Predictions

Why Exponentiating a Log Prediction Can Miss the Mean

A regression model for a logged outcome predicts on the log scale. Exponentiating that prediction returns a value on the original outcome scale, but it does not generally return the conditional arithmetic mean of the outcome. The reason is curvature: the exponential function gives more weight to upward deviations than downward deviations of the same size.

Jensen’s inequality makes this distinction precise. It shows that the arithmetic mean of exponentiated log outcomes is at least as large as the exponential of their mean log outcome. When log-scale errors are normal, the difference has a simple formula involving their variance. The result matters whenever the quantity of interest is an average outcome in the original units rather than a typical value on the log scale.

Definition: Suppose the outcome \(Y\) is positive and, at predictor value \(x\), its natural logarithm follows the conditional model \(\ln Y=m(x)+\varepsilon\), where \(m(x)=E(\ln Y\mid X=x)\) and \(E(\varepsilon\mid X=x)=0\). The exponentiated mean log outcome is \(\exp(m(x))\). The conditional arithmetic mean is \(E(Y\mid X=x)\). They are different summaries in general.

The model statement specifies the conditional mean on the log scale; it does not by itself specify the full distribution of the errors. For the general inequality below, it is enough that the relevant expectations exist. A normal error distribution is an additional assumption used later to obtain an exact arithmetic-mean formula.

Jensen’s Inequality Applied to a Log-Outcome Model

Theorem: If \(Z\) is a random variable with a finite mean and the expectation of \(\exp(Z)\) exists, then \(E\{\exp(Z)\}\geq \exp\{E(Z)\}\). If \(Z\) is not constant with probability one, the inequality is strict.

The exponential function is strictly convex because its second derivative is \(\exp(z)>0\) for every real number \(z\). Jensen’s inequality says that for a convex function, the function evaluated at a mean is no greater than the mean of the function. Taking the convex function to be \(\exp(z)\) gives the stated result. If the random variable takes more than one value with positive probability, strict convexity makes the inequality strict.

Apply the theorem to \(Z=\ln Y\), conditional on \(X=x\). Since \(\ln Y=m(x)+\varepsilon\) and the conditional mean of \(\varepsilon\) is zero, \(E(\ln Y\mid X=x)=m(x)\). Therefore,

E(Y\mid X=x) =E\{\exp(\ln Y)\mid X=x\} \geq \exp\{E(\ln Y\mid X=x)\} =\exp\{m(x)\}.

If \(\ln Y\) varies among outcomes at this \(x\), the inequality is strict. Thus, exponentiating the conditional mean log outcome generally underestimates the conditional arithmetic mean. Equality holds when the conditional log outcome is constant, so there is no conditional variation in \(Y\).

This comparison concerns two different summaries of the conditional outcome distribution. The exponentiated mean log outcome is its geometric-mean-scale summary; under conditionally normal log errors, it is also the conditional median. The arithmetic mean instead averages the original \(Y\) values. The earlier material on exponentiated predictions establishes those interpretations; here the focus is why the arithmetic mean is larger.

Worked Example: A Two-Point Log-Error Distribution

Suppose a hypothetical outcome is measured in units, and at a fixed predictor value its log model is \(\ln Y=0.80+\varepsilon\). The error is \(-0.50\) or \(0.50\), each with probability \(1/2\). Calculate the exponentiated mean log outcome and the arithmetic conditional mean.

Check the model conditions. The outcome is positive because it is defined by exponentiating its log. Also, \(E(\varepsilon\mid X=x)=\{(-0.50)+(0.50)\}/2=0\), so \(m(x)=0.80\). The log outcome is not constant, so Jensen’s inequality predicts a strict difference.

The exponentiated mean log outcome is \(\exp(0.80)=2.22554\), or about \(2.226\) units. For the arithmetic mean, exponentiate each possible log outcome and average:

E(Y\mid X=x) =\frac{\exp(0.80-0.50)+\exp(0.80+0.50)}{2} =\exp(0.80)\frac{\exp(-0.50)+\exp(0.50)}{2} =2.5096\text{ units}.

The arithmetic mean, \(2.5096\) units, is greater than the exponentiated mean log outcome, \(2.22554\) units. This example does not assume normal errors; the general Jensen inequality is enough to establish the direction. The direct calculation supplies the numerical difference for this particular two-point distribution.

The Exact Adjustment Under Normal Log-Scale Errors

Assume now that, conditional on \(X=x\), the log-scale error is normally distributed with mean zero and variance \(\sigma^2\): \(\varepsilon\mid X=x\sim N(0,\sigma^2)\). This assumption gives a closed-form expression for the conditional arithmetic mean. The variance convention here is that the second parameter of the normal distribution is the variance.

\ln Y\mid X=x\sim N\{m(x),\sigma^2\} \quad\Longrightarrow\quad E(Y\mid X=x)=\exp\left\{m(x)+\frac{\sigma^2}{2}\right\}.

A short derivation shows where the variance adjustment comes from. The conditional density of \(\varepsilon\) is normal, so its exponential mean is the integral of \(\exp(e)\) against that density. Completing the square in the exponent gives:

\begin{aligned} E\{\exp(\varepsilon)\mid X=x\} &=\int_{-\infty}^{\infty} \exp(e)\frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{e^2}{2\sigma^2}\right)\,de\\ &=\exp\left(\frac{\sigma^2}{2}\right) \int_{-\infty}^{\infty} \frac{1}{\sigma\sqrt{2\pi}} \exp\left(-\frac{(e-\sigma^2)^2}{2\sigma^2}\right)\,de\\ &=\exp\left(\frac{\sigma^2}{2}\right). \end{aligned}

The last integral is one: its integrand is the density of a normal distribution with mean \(\sigma^2\) and variance \(\sigma^2\). Since \(Y=\exp\{m(x)\}\exp(\varepsilon)\), multiplying this result by \(\exp\{m(x)\}\) yields the formula. The correction factor relative to simply exponentiating \(m(x)\) is \(\exp(\sigma^2/2)\), which exceeds one whenever \(\sigma^2>0\).

The zero conditional error mean and the normal distribution are assumptions about the log-scale errors, not consequences of applying a logarithm. The normal formula also requires the conditional variance to be the \(\sigma^2\) used at that predictor value. If conditional error variance changes with \(x\), the formula uses the variance at each \(x\); a single common adjustment would not apply to all predictor values.

Worked Example: Normal Log Errors at One Predictor Value

In a hypothetical setting, suppose \(Y\) is a positive measurement in millilitres. At a specified predictor value, the fitted mean log outcome is \(m(x)=1.20\), and the conditional log errors are assumed to follow \(N(0,0.16)\). Find the exponentiated fitted log mean and the conditional arithmetic mean.

Check the assumptions. The outcome is positive, the model has mean-zero conditional log errors, and the specified conditional error variance is \(0.16\). Thus \(\sigma^2=0.16\), and the normal-error formula applies.

Exponentiating the fitted mean log outcome gives \(\exp(1.20)=3.32012\) millilitres. The arithmetic mean includes the variance adjustment:

E(Y\mid X=x) =\exp\left(1.20+\frac{0.16}{2}\right) =\exp(1.28) =3.59664\text{ millilitres}.

The adjusted mean is about \(3.597\) millilitres, higher than \(3.320\) millilitres. Their ratio is \(\exp(0.08)\approx1.0833\), so under these assumptions the conditional arithmetic mean is about \(8.33\%\) greater than the exponentiated mean log outcome. This percentage comparison follows from the specified log-scale variance; it is not a general percentage difference for all log-outcome models.

Why Log-Scale Variability Changes the Original-Scale Mean

The variance adjustment is not a change to the fitted mean log outcome. It reflects how a distribution with the same mean log outcome can have different arithmetic means when its spread changes. Because \(\exp(z)\) curves upward, positive deviations on the log scale increase \(Y\) more than equally large negative deviations decrease it. Within the normal-error family, increasing the variance while holding the mean fixed raises the arithmetic mean relative to its exponentiated log-scale center. For arbitrary distributions, variance alone does not order this comparison; Jensen’s inequality only guarantees that the arithmetic mean is at least the exponentiated mean.

The normal formula makes this dependence explicit. If two conditional distributions have the same \(m(x)\) but different variances, their exponentiated mean log outcomes are the same, while their arithmetic means are not. The distinction can matter in practice when reporting an expected quantity, such as mean resource use or mean concentration, in the outcome’s original units.

Worked Example: Same Log-Scale Center, Different Variances

Consider two hypothetical groups with outcomes measured in units. At the predictor values being compared, both groups have conditional mean log outcome \(m(x)=0.50\). Assume mean-zero normal log errors, with variance \(0.04\) for Group A and \(0.64\) for Group B. Compare their exponentiated mean log outcomes and arithmetic means.

Check the assumptions. Both outcomes are positive by construction; both conditional log-error distributions are normal with mean zero. Each group has its own specified conditional variance, so use the corresponding variance in its arithmetic-mean formula.

For both groups, the exponentiated mean log outcome is \(\exp(0.50)=1.64872\) units. For Group A, \(\sigma_A^2/2=0.04/2=0.02\), and for Group B, \(\sigma_B^2/2=0.64/2=0.32\). Hence:

\begin{aligned} E(Y\mid X=x,\text{ Group A}) &=\exp(0.50+0.02)=\exp(0.52)=1.68203\text{ units},\\ E(Y\mid X=x,\text{ Group B}) &=\exp(0.50+0.32)=\exp(0.82)=2.27050\text{ units}. \end{aligned}

The groups share the same exponentiated log-scale center, but Group B’s conditional arithmetic mean is larger because its log-scale variance is larger. This comparison isolates the role of variability: it does not say that variance alone determines every difference between groups, since their log-scale centers and other model features could also differ in another setting.

What Can and Cannot Be Concluded

Jensen’s inequality establishes the direction of the discrepancy without requiring normality: exponentiating a conditional mean log outcome cannot exceed the conditional arithmetic mean, provided the expectations exist. It does not, by itself, quantify the size of the gap. The simple adjustment \(\exp\{\sigma^2/2\}\) is exact under conditionally normal log errors with variance \(\sigma^2\), and should not be treated as a universal correction for every error distribution.

In applied work, \(m(x)\) and \(\sigma^2\) are not usually known; a fitted regression supplies estimates. Substituting fitted quantities into the normal formula gives a model-based estimate of the conditional arithmetic mean, not a guarantee that the assumptions hold. The distinction between estimating a mean and describing uncertainty around a prediction is also important: the variance adjustment here addresses the mean on the original scale, not a confidence or prediction interval.

Common Mistake

Common mistake: “The fitted mean log outcome is \(0.80\), so \(\exp(0.80)\) must be the arithmetic mean of \(Y\).” This treats exponentiation as if it preserved averages. The correction is that the exponential function is convex: Jensen’s inequality gives \(E(Y\mid X=x)\geq\exp\{m(x)\}\). Under normal log errors, calculate the arithmetic mean as \(\exp\{m(x)+\sigma^2/2\}\), and state the distributional assumptions behind that calculation.

A related mistake is to add \(\sigma/2\) rather than \(\sigma^2/2\) in the normal formula. The adjustment uses half the variance, not half the standard deviation. For example, if \(\sigma=0.40\), then \(\sigma^2=0.16\), so the adjustment in the exponent is \(0.08\).

Key takeaway: Exponentiating the mean log outcome gives \(\exp\{m(x)\}\), while the original-scale arithmetic mean is at least that large. With mean-zero normal log errors of variance \(\sigma^2\), it is exactly \(\exp\{m(x)+\sigma^2/2\}\).

Check Your Understanding

Answer each question using the assumptions stated in it.

State Jensen’s inequality for the exponential function and explain when the inequality is strict.
At a predictor value, \(m(x)=0.40\) and the conditional log errors are normal with mean zero and variance \(0.25\). Calculate the exponentiated mean log outcome and the conditional arithmetic mean.
Two groups have the same mean log outcome but different conditional log-error variances. Under normal log errors, which group has the larger arithmetic mean, and why?
Does Jensen’s inequality require normal log errors? What additional assumption permits the exact \(\exp\{m(x)+\sigma^2/2\}\) formula?
A learner uses \(\exp\{m(x)\}\) as the arithmetic mean because it is on the original outcome scale. Identify the error in this reasoning.