Logarithmic Jensen Bounds and Geometric Means

Why Compare an Expected Logarithm With a Logarithm of an Expectation?

For a positive random variable, two natural summaries are $E[\log X]$ and $\log E[X]$. They are not interchangeable: one averages after taking logarithms, while the other takes a logarithm after averaging. Concavity of the logarithm determines their ordering. The resulting bound also explains why arithmetic means dominate geometric means.

The comparison requires care about positivity and integrability. In particular, a finite mean for $X$ guarantees that the positive part of $\log X$ is integrable, but it does not guarantee that $E[\log X]$ is finite: the expectation can equal $-\infty$. The inequality remains meaningful in that case.

Definition: Suppose $X>0$ almost surely and $E[X]<\infty$. The logarithmic mean of $X$ on the log scale is $E[\log X]$, which may be $-\infty$. If $E[|\log X|]<\infty$, the associated geometric mean is $\exp(E[\log X])$. When $E[\log X]=-\infty$, it is convenient to set $\exp(E[\log X])=0$.

Throughout, $\log$ denotes the natural logarithm. For $x>0$, its derivatives are $(\log x)'=1/x$ and $(\log x)''=-1/x^2<0$, so it is strictly concave. That concavity gives an upper bound on the average log, rather than a lower bound.

The Logarithmic Jensen Bound

Theorem: If $X>0$ almost surely and $0<E[X]<\infty$, then $$ E[\log X]\leq \log E[X]. $$ The left side is interpreted as an extended expectation and may be $-\infty$. Equality holds if and only if $X=E[X]$ almost surely.

The assumptions ensure that the right side is a finite real number. Also, $(\log X)^+$, the positive part of $\log X$, is integrable: for $x\geq1$, $\log x\leq x$, and for $0<x<1$, $(\log x)^+=0$. Thus $E[\log X]$ is well-defined as a finite value or $-\infty$, not as an indeterminate difference of infinite quantities.

Proof: Put $m=E[X]$, so $0<m<\infty$. For every $x>0$, the tangent-line inequality for the logarithm at $m$ is $$ \log x\leq \log m+\frac{x-m}{m}. $$ Indeed, with $u=x/m>0$, this is equivalent to $\log u\leq u-1$. The latter follows because $u-1-\log u$ has derivative $1-1/u$, decreases up to $u=1$, increases after $u=1$, and equals zero at $u=1$. Apply the inequality pointwise to $X$. The right side is integrable, and taking expectations gives $$ E[\log X]\leq \log m+\frac{E[X]-m}{m}=\log m=\log E[X]. $$ For equality, define $D=\log m+(X-m)/m-\log X$. The pointwise inequality gives $D\geq0$, and the scalar equality condition just proved gives $D=0$ exactly when $X=m$. If $E[\log X]=-\infty$, equality with the finite value $\log m$ is impossible. Otherwise $D$ is integrable and $E[D]=\log m-E[\log X]$. Thus equality in the bound implies $E[D]=0$, which for a nonnegative random variable implies $D=0$ almost surely. Hence $X=m$ almost surely. Conversely, if $X=m$ almost surely, both sides equal $\log m$. $\square$

Exponentiating the bound preserves its direction because the exponential function is increasing. In particular, $$ \exp(E[\log X])\leq E[X]. $$ This compares the geometric mean with the arithmetic mean of a random quantity. The inequality can be strict even when $X$ is bounded and has a perfectly finite logarithmic expectation.

Worked Example: A Two-Point Distribution

Let $X$ equal $1$ or $9$, each with probability $1/2$. Calculate its mean and geometric mean, then verify the bound.

The arithmetic mean is $$ E[X]=\frac12(1)+\frac12(9)=5. $$ The expected logarithm is $$ E[\log X]=\frac12\log 1+\frac12\log 9 =0+\frac12\log 9=\log 3. $$ Therefore the geometric mean is $\exp(E[\log X])=3$. The comparison is $3<5$, or equivalently $E[\log X]=\log 3<\log 5=\log E[X]$. Since $X$ is not constant, the strict inequality agrees with the theorem's equality condition.

Arithmetic–Geometric Mean Inequalities

A finite collection of positive numbers can be treated as the possible values of a discrete random variable. This turns the logarithmic Jensen bound into the weighted arithmetic–geometric mean inequality.

Theorem: Let $a_1,\ldots,a_k>0$, and let $p_1,\ldots,p_k\geq0$ satisfy $\sum_{i=1}^k p_i=1$. Then $$ \prod_{i=1}^k a_i^{p_i}\leq \sum_{i=1}^k p_i a_i. $$ Equality holds if and only if all $a_i$ for which $p_i>0$ are equal.

Proof: Define a discrete random variable $X$ that takes value $a_i$ with probability $p_i$, ignoring any indices with $p_i=0$. Then $X>0$ and $$ E[X]=\sum_{i=1}^k p_i a_i,\qquad E[\log X]=\sum_{i=1}^k p_i\log a_i. $$ The logarithmic Jensen bound yields $$ \sum_{i=1}^k p_i\log a_i\leq \log\left(\sum_{i=1}^k p_i a_i\right). $$ Exponentiating both sides gives $$ \prod_{i=1}^k a_i^{p_i}\leq \sum_{i=1}^k p_i a_i. $$ By the equality condition for the logarithmic Jensen bound, equality occurs exactly when $X$ is constant with probability one, which means that all values having positive weight are equal. $\square$

Taking $p_i=1/k$ gives the familiar unweighted inequality for positive numbers: $$ \left(\prod_{i=1}^k a_i\right)^{1/k}\leq \frac{1}{k}\sum_{i=1}^k a_i. $$ The left side is the geometric mean and the right side is the arithmetic mean. The weights matter: a value assigned probability $p_i$ contributes $p_i\log a_i$ to the expected log and $p_i a_i$ to the arithmetic mean.

Worked Example: Comparing the Means of Three Numbers

Compare the geometric and arithmetic means of $2,8,18$. Give the calculation and identify whether equality is possible.

The arithmetic mean is $$ \frac{2+8+18}{3}=\frac{28}{3}\approx9.333. $$ The geometric mean is $$ (2\cdot8\cdot18)^{1/3}=288^{1/3}\approx6.604, $$ since $6.604^3$ is approximately $288$. Thus $6.604<9.333$. Equality would require all three equally weighted values to be equal, but $2$, $8$, and $18$ are distinct. The inequality is therefore strict.

Applying the Bound to Continuous Distributions

The same result applies to continuous random variables; no finite-support assumption is needed. The key conditions are almost-sure positivity and a finite, positive expectation. A continuous example also shows how the expected log can be computed directly from a density.

Worked Example: A Uniform Random Variable

Suppose $X\sim\operatorname{Uniform}(1,4)$. Find $E[X]$, $E[\log X]$, and the geometric mean, and check the inequality.

The density is $f(x)=1/3$ for $1<x<4$. Hence $$ E[X]=\frac13\int_1^4 x\,dx =\frac13\left[\frac{x^2}{2}\right]_1^4 =\frac{16-1}{6}=\frac52. $$ For the expected logarithm, use the antiderivative $x\log x-x$: $$ E[\log X]=\frac13\int_1^4\log x\,dx =\frac13[x\log x-x]_1^4 =\frac{4\log4-3}{3}\approx0.8484. $$ The geometric mean is therefore $$ \exp(E[\log X])=\exp\left(\frac{4\log4-3}{3}\right)\approx2.336. $$ The arithmetic mean is $2.5$, so $2.336<2.5$. The comparison also holds on the log scale: $0.8484<\log(2.5)\approx0.9163$.

What the Gap Tells You

The difference $\log E[X]-E[\log X]$, when finite, is the Jensen gap for the logarithm. It is nonnegative and is zero precisely when $X$ is constant almost surely. It measures the effect of variability on the comparison, though this bound alone does not give a universal numerical formula for that gap.

Geometric means are useful when quantities combine multiplicatively. For example, if positive growth factors over several periods are $g_1,\ldots,g_n$, their compounded product is $\prod_{i=1}^n g_i$. The corresponding per-period factor is the geometric mean $(\prod_{i=1}^n g_i)^{1/n}$, not the arithmetic mean of the factors. Taking logarithms turns the product into a sum, which explains why averages of log values naturally describe multiplicative behavior.

Worked Example: A Sample Arithmetic Mean and Geometric Mean

A sample of four positive measurements is $1,2,4,8$. Compare the sample arithmetic mean with the sample geometric mean by treating each observation as equally likely.

The arithmetic mean is $$ \bar{x}=\frac{1+2+4+8}{4}=\frac{15}{4}=3.75. $$ The average log is $$ \frac14(\log1+\log2+\log4+\log8) =\frac14(0+\log2+2\log2+3\log2) =\frac32\log2. $$ Exponentiating gives the geometric mean $2^{3/2}=\sqrt{8}\approx2.828$. Thus $2.828<3.75$. The sample values are not all equal, so strict inequality is expected. More generally, applying the finite arithmetic–geometric mean inequality to any positive observations $x_1,\ldots,x_n$ gives $$ \exp\left(\frac1n\sum_{i=1}^n\log x_i\right) \leq \frac1n\sum_{i=1}^n x_i. $$

Common Mistake

A common error is to replace $E[\log X]$ by $\log E[X]$, or to reverse their order because logarithms are increasing. Being increasing means that applying $\log$ preserves an inequality between two fixed numbers; it does not mean that logarithm commutes with expectation. For a nonconstant positive $X$, the correct comparison is $$ E[\log X]<\log E[X]. $$ The direction comes from concavity, not merely from monotonicity. Another important check is the domain: the theorem as stated requires $X>0$ almost surely. If $X$ can be zero, $\log X$ is not a finite real-valued random variable there, so the assumptions and interpretation must be addressed rather than silently taking its logarithm.

Key takeaway: For a positive random variable with finite mean, averaging its logarithm gives no more than taking the logarithm of its mean. Equality occurs only for a constant random variable; in finite weighted form, this is exactly the arithmetic–geometric mean inequality.

Check Your Understanding

Use the positivity, integrability, and equality conditions carefully.

State the logarithmic Jensen bound and its assumptions when $E[X]$ is finite.
If $X$ is positive and takes values $3$ and $12$ with equal probabilities, calculate $E[X]$ and $\exp(E[\log X])$.
For positive $a_1,\ldots,a_k$ with weights $p_i$ summing to one, write the weighted arithmetic–geometric mean inequality.
When does equality hold in the logarithmic Jensen bound? What does that condition mean for a finite collection with equal weights?
Explain why the fact that logarithm is increasing does not justify replacing $E[\log X]$ with $\log E[X]$.