Jensen's Inequality for Convex Functions

Why a Supporting Line Gives an Expectation Bound

A convex function bends upward: values away from a point cannot fall below a suitable line through that point. For a random variable, the useful point is its mean. If a line supports the function there, taking expectations preserves the line’s value at the mean, giving a lower bound for the expected function value.

The argument does not require a derivative. A convex function may have a corner at the mean, and any appropriate supporting slope will do. The domain and integrability assumptions matter: the function must be defined at the random variable’s values and at its mean, and the expectation must not be an undefined difference between two infinities.

In this tutorial, \(I\) is an open interval and \(\phi:I\to\mathbb{R}\) is finite-valued and convex. A random variable \(X\) is integrable if \(E[|X|]<\infty\). When \(P(X\in I)=1\), this condition ensures that \(m=E[X]\) is finite and also lies in \(I\). A finite convex function on an open interval is continuous, so \(\phi(X)\) is measurable. Its expectation may be \(+\infty\); the theorem below permits that case, but never an undefined expectation of the form \(+\infty-\infty\).

The Supporting-Line Property

The key geometric fact is that a finite convex function has at least one supporting line at any interior point. This does not mean that the graph has a unique tangent there. At a corner there may be many valid slopes.

Proof. For \(x<m\) and \(y>m\), define the left and right secant slopes \(s_-(x)=(\phi(m)-\phi(x))/(m-x)\) and \(s_+(y)=(\phi(y)-\phi(m))/(y-m)\). Convexity implies \(s_-(x)\leq s_+(y)\). To see this, write \(m\) as a convex combination of \(x\) and \(y\): \(m=\frac{y-m}{y-x}x+\frac{m-x}{y-x}y\). Applying convexity and rearranging gives the stated slope inequality.

Set \(A=\sup_{x<m}s_-(x)\) and \(B=\inf_{y>m}s_+(y)\), where the points range over \(I\). The slope inequality gives \(A\leq B\). These bounds are finite: choose any \(x_0<m<y_0\) in \(I\). Every left slope is at most \(s_+(y_0)\), and \(A\geq s_-(x_0)\); also every right slope is at least \(s_-(x_0)\), and \(B\leq s_+(y_0)\). Thus both \(A\) and \(B\) are finite. Choose \(c\in[A,B]\). If \(x<m\), then \(s_-(x)\leq c\), which rearranges to \(\phi(x)\geq\phi(m)+c(x-m)\). If \(x>m\), then \(s_+(x)\geq c\), giving the same inequality. At \(x=m\), equality holds. Therefore the line supports \(\phi\) at \(m\). \(\square\)

When \(\phi\) is differentiable at \(m\), its derivative is a supporting slope, so one may take \(c=\phi'(m)\). The lemma is more general: it provides a slope even when that derivative does not exist.

Jensen’s Inequality

Proof. Write \(m=E[X]\). First, \(m\in I\). If the lower endpoint \(a\) of \(I\) is finite, then \(X-a>0\) almost surely. A strictly positive random variable has strictly positive expectation, so \(m-a=E[X-a]>0\). (Indeed, if \(Y>0\) almost surely, then some event \(\{Y\geq 1/k\}\) has positive probability, and hence \(E[Y]>0\).) If the upper endpoint \(b\) is finite, the same argument applied to \(b-X>0\) gives \(m<b\). There is no endpoint restriction to check when an endpoint is infinite.

By the supporting-line lemma, there is a finite \(c\) such that \(\phi(x)\geq\phi(m)+c(x-m)\) for all \(x\in I\). Since \(X\in I\) almost surely, define \(R=\phi(X)-\phi(m)-c(X-m)\geq0\) almost surely. The affine term \(\phi(m)+c(X-m)\) is integrable because \(E[|X|]<\infty\). The inequality \(\phi(X)\geq\phi(m)+c(X-m)\) ensures that the negative part of \(\phi(X)\) is integrable; consequently \(E[\phi(X)]\) is well-defined, possibly \(+\infty\). Taking expectations in the identity defining \(R\), with \(E[R]\geq0\), yields \[ E[\phi(X)]=\phi(m)+c(E[X]-m)+E[R] =\phi(m)+E[R]\geq\phi(m). \] Since \(m=E[X]\), this is Jensen’s inequality. \(\square\)

If one wants an ordinary finite-valued comparison rather than an extended expectation, it is enough to add the assumption \(E[|\phi(X)|]<\infty\). That extra condition is not needed for the inequality itself: the supporting line prevents the expectation from being \(-\infty\).

Worked Applications

Assumptions and Common Pitfalls

Jensen’s inequality is useful precisely because it can be applied to many functions and distributions, but its conclusion depends on matching the function’s domain to the random variable and checking that the expectation is meaningful.

• Using a function outside its domain. The theorem requires \(P(X\in I)=1\), where \(\phi\) is defined and convex. Convexity on some unrelated interval does not establish the inequality for \(X\).
• Assuming differentiability is necessary. It is not. The incorrect reasoning is, “There is no derivative at the mean, so there is no tangent line and Jensen cannot be used.” The correction is to use any supporting slope, whose existence follows from convexity at an interior point.
• Ignoring integrability. If \(E[|X|]=\infty\), the finite mean \(E[X]\) required on the right may not exist. If \(E[|\phi(X)|]\) is not finite, the left side can still be \(+\infty\), but the supporting-line argument ensures it is not \(-\infty\).
• Reversing the inequality for a convex function. Convexity places the graph above its supporting line, so averaging gives \(E[\phi(X)]\geq\phi(E[X])\), not the reverse.