Jensen's Inequality For Exponentials And Logarithms

From Convexity to Exponential Bounds

Jensen’s inequality turns convexity into a comparison between a function of an average and an average of function values. For the exponential function, that comparison controls how large the average of exponentials must be relative to the exponential of the average. Applying the same result to a logarithm gives a geometric-mean versus arithmetic-mean bound. The normalisation of the measure matters: these statements are averages on a probability space, or on a finite-measure set after dividing by its measure.

Throughout, \((X,\mathcal{M},\mathbb{P})\) is a probability space, and functions are real-valued and measurable unless stated otherwise. A real-valued function \(f\) is integrable when \(\int_X |f|\,d\mathbb{P}<\infty\). We use the integral Jensen inequality established earlier: if \(f\) is integrable and \(\Phi:\mathbb{R}\to\mathbb{R}\) is convex, then, whenever \(\Phi(f)\) is measurable, \(\Phi(\int f\,d\mathbb{P})\le\int\Phi(f)\,d\mathbb{P}\), with the right side allowed to be \(+\infty\) when \(\Phi(f)\ge0\). The exponential is convex because its second derivative is positive everywhere.

Proof. Since \(f\) is integrable, \(\int_X f\,d\mathbb{P}\) is a finite real number. Apply integral Jensen with the convex function \(\Phi(s)=e^s\). The function \(e^f\) is measurable and nonnegative, so its integral is defined in \([0,\infty]\), and Jensen gives the first inequality. For any fixed real \(t\), \(tf\) is integrable because \(\int_X|tf|\,d\mathbb{P}=|t|\int_X|f|\,d\mathbb{P}<\infty\). Applying the first inequality to \(tf\) proves the second. \(\square\)

If \((X,\mathcal{M},\mu)\) has finite, positive total measure \(M=\mu(X)\), use the probability measure \(\mathbb{P}(E)=\mu(E)/M\). For integrable \(f\), this gives the finite-measure form $$ \exp\left(\frac{1}{M}\int_X f\,d\mu\right)\le \frac{1}{M}\int_X e^f\,d\mu. $$ The same formula applies on a measurable set \(E\) with \(0<\mu(E)<\infty\), using the restricted measure and \(M=\mu(E)\). It does not assert an average over a set of measure zero, or over a set of infinite measure without an additional normalisation.

Logarithmic Bounds and Weighted Means

The logarithmic comparison follows from the exponential bound, with care when the logarithm is not integrable. Here \(g\) may be zero: we define \(\log g=-\infty\) where \(g=0\). The resulting integral can equal \(-\infty\), but under the assumptions below it is well-defined as an extended integral. In fact, if \(g\ge0\) is integrable, then \(\log^+g\le g\), where \(\log^+g=\max(\log g,0)\); hence the positive part of \(\log g\) is integrable.

Proof. For each positive integer \(n\), set \(h_n=\log(\max(g,e^{-n}))\). This is measurable. Its negative part is bounded by \(n\), and its positive part is at most \(g\), since \(\log^+g\le g\); because \(\mathbb{P}(X)=1\) and \(g\) is integrable, \(h_n\) is integrable. Also \(e^{h_n}=\max(g,e^{-n})\). The exponential Jensen bound gives $$ \exp\left(\int_X h_n\,d\mathbb{P}\right)\le \int_X\max(g,e^{-n})\,d\mathbb{P}. $$ As \(n\) increases, \(h_n\) decreases pointwise to \(\log g\), including the value \(-\infty\) where \(g=0\). More explicitly, the positive part of \(h_n\) is \(\log^+g\), and its negative part increases to \((\log g)^-\). The Monotone Convergence Theorem applied to these negative parts therefore gives \(\int_X h_n\,d\mathbb{P}\to\int_X\log g\,d\mathbb{P}\), allowing limit \(-\infty\). Meanwhile, \(\max(g,e^{-n})\to g\) pointwise and is bounded by \(g+1\), an integrable function on this probability space. The Dominated Convergence Theorem gives \(\int_X\max(g,e^{-n})\,d\mathbb{P}\to\int_Xg\,d\mathbb{P}\). Taking limits in the displayed inequality proves the result, using \(e^{-\infty}=0\) if necessary. The last displayed exponential form follows by exponentiating when \(\log g\) is integrable. \(\square\)

For a finite weighted list, take \(X=\{1,\ldots,k\}\), assign point \(i\) probability \(w_i\), and suppose \(w_i>0\) with \(\sum_{i=1}^k w_i=1\). Applying the logarithmic bound to positive values \(a_i\) gives the weighted geometric-mean inequality $$ \prod_{i=1}^k a_i^{w_i}\le \sum_{i=1}^k w_i a_i. $$ Indeed, the integral of \(\log g\) is \(\sum_i w_i\log a_i\), whose exponential is the product on the left, while the integral of \(g\) is the sum on the right. The positive weights ensure that each value is included in the finite probability space.

Parameters, Sharpness, and a Common Pitfall

The real parameter in the exponential bound is useful when values have different scales or signs. For any integrable real-valued \(f\), the bound at parameter \(t\) compares \(e^{t\int f\,d\mathbb{P}}\) with \(\int e^{tf}\,d\mathbb{P}\); the sign of \(t\) does not change the direction. For a finite weighted list \(x_i\), it reads $$ \exp\left(t\sum_{i=1}^k w_i x_i\right)\le\sum_{i=1}^k w_i e^{t x_i}. $$ This follows by applying the theorem to the function with value \(x_i\) at point \(i\), not by assuming that \(t\) is positive.

The constants in these inequalities cannot be improved as universal bounds: if \(f\) is constant with value \(c\), then both sides of the exponential inequality equal \(e^c\). If \(g\) is the positive constant \(c\), then both sides of the logarithmic inequality equal \(\log c\). These examples show that the bounds are attained. The full description of equality cases is a separate question.

Common mistake: reversing the logarithmic inequality. A tempting but incorrect calculation is to claim that the geometric mean is at least the arithmetic mean. For the two values \(1\) and \(9\) with equal weights, that claim would give \(3\ge5\), which is false. The correct application is \(\exp(\frac12\log1+\frac12\log9)\le\frac12(1+9)\), or \(3\le5\). The direction comes from Jensen applied to the concave logarithm, or equivalently from the exponential bound proved above; it is not reversed when one passes from logarithms to exponentials because the exponential function is increasing.