Tutorials › Probability Theory (Graduate) › Jensen’s Inequality for Expectations

Home  /  Tutorials  /  Integrability and Uniform Integrability

Integrability and Uniform Integrability · Tutorial 139 of 946

Jensen’s Inequality for Expectations

Learn how convexity turns an average input into a lower bound for the average output, and apply Jensen’s inequality to powers and exponentials.

Intermediate 9 min read

What You'll Learn

  • State Jensen’s inequality with precise domain and integrability assumptions
  • Prove the inequality using a supporting line at the mean
  • Identify the equality case for a strictly convex function
  • Apply the result to absolute moments and exponential expectations
  • Recognize why the mean must lie in the function’s domain

Why Convexity Controls Averages

A convex function bends in a way that makes averaging its outputs different from applying it to an average input. For a random variable \(X\), the basic comparison is between \(\varphi(\mathbb{E}X)\) and \(\mathbb{E}[\varphi(X)]\). Convexity says that the second quantity cannot be smaller, provided both sides are defined and the expectations are meaningful.

This principle is useful for turning information about a mean into a bound on a transformed expectation. Powers yield lower bounds on moments, while the exponential function yields bounds on moment-generating expectations. The argument rests on a geometric fact: at the mean, a convex function has a supporting line lying below its graph.

Definition: Let \(I\) be an open interval in \(\mathbb{R}\). A function \(\varphi:I\to\mathbb{R}\) is convex if, for every \(x,y\in I\) and every \(\lambda\in[0,1]\), \[ \varphi(\lambda x+(1-\lambda)y) \leq \lambda\varphi(x)+(1-\lambda)\varphi(y). \] It is strictly convex if this inequality is strict whenever \(x\ne y\) and \(0<\lambda<1\).

The inequality compares the function at a point between \(x\) and \(y\) with the corresponding weighted average of its values. Jensen’s inequality extends that two-point comparison to a random variable, which may take many values or a continuum of values.

Jensen’s Inequality for an Expectation

Theorem (Jensen’s inequality): Let \(X\) be an integrable real-valued random variable, let \(I\) be an open interval, and suppose \(X\in I\) almost surely. Let \(\varphi:I\to\mathbb{R}\) be convex, and suppose \(\varphi(X)\) is integrable. Then \(\mathbb{E}X\in I\), and \[ \varphi(\mathbb{E}X)\leq \mathbb{E}[\varphi(X)]. \]

Here integrability means \(\mathbb{E}|X|<\infty\) and \(\mathbb{E}|\varphi(X)|<\infty\). The domain condition matters: the expression \(\varphi(\mathbb{E}X)\) must be defined. Since \(X\) lies almost surely in an open interval and is integrable, its expectation lies in that interval as well. For example, if \(X\in(a,b)\) almost surely, then \(X-a>0\) and \(b-X>0\) almost surely; each positive random variable has strictly positive expectation, giving \(a<\mathbb{E}X<b\). The same argument applies when an endpoint is infinite, omitting the corresponding bound.

We prove the theorem by finding a line that touches the graph at \(m=\mathbb{E}X\) and lies below it everywhere on \(I\). First, convexity implies that for \(a<b<c\) in \(I\), the secant slopes satisfy \[ \frac{\varphi(b)-\varphi(a)}{b-a} \leq \frac{\varphi(c)-\varphi(a)}{c-a} \leq \frac{\varphi(c)-\varphi(b)}{c-b}. \] Indeed, write \(b\) as a convex combination of \(a\) and \(c\), apply the definition of convexity, and rearrange. This slope ordering is what ensures that a supporting line exists.

Proof. Set \(m=\mathbb{E}X\), which belongs to \(I\). For \(x<m\) and \(y>m\), define the left and right secant slopes \[ L(x)=\frac{\varphi(m)-\varphi(x)}{m-x}, \qquad R(y)=\frac{\varphi(y)-\varphi(m)}{y-m}. \] The secant-slope ordering gives \(L(x)\leq R(y)\) for every such pair \(x,y\). Choose fixed points \(x_0<m<y_0\). Then every \(L(x)\leq R(y_0)\), while \(L(x_0)\leq R(y)\) for every \(y>m\). Consequently, the numbers \[ s=\sup_{x<m}L(x) \] and \(\inf_{y>m}R(y)\) are finite, and \(s\leq\inf_{y>m}R(y)\). Thus \(L(x)\leq s\) for \(x<m\) and \(R(y)\geq s\) for \(y>m\). Rearranging these inequalities, and including equality at \(x=m\), gives \[ \varphi(z)\geq\varphi(m)+s(z-m) \qquad\text{for every }z\in I. \] Apply this pointwise with \(z=X\) almost surely and take expectations. All terms are integrable, and \(\mathbb{E}(X-m)=0\), so \[ \mathbb{E}[\varphi(X)] \geq \mathbb{E}\bigl[\varphi(m)+s(X-m)\bigr] =\varphi(m). \] This is the desired inequality. \(\square\)

The proof does not require differentiability. For a differentiable convex function, the supporting-line slope can be chosen as \(s=\varphi'(m)\); the secant argument also works at a point where the derivative does not exist.

When Does Equality Hold?

For a strictly convex function, the inequality is strict unless \(X\) is constant almost surely. This gives a useful diagnostic: equality is not generally expected when the random variable genuinely varies.

Proposition: Under the hypotheses of Jensen’s inequality, if \(\varphi\) is strictly convex, then \[ \mathbb{E}[\varphi(X)]=\varphi(\mathbb{E}X) \quad\text{if and only if}\quad X=\mathbb{E}X\ \text{almost surely}. \]
Proof. Let \(m=\mathbb{E}X\) and choose the supporting-line slope \(s\) from the proof of Jensen’s inequality. Define \[ Y=\varphi(X)-\varphi(m)-s(X-m). \] The supporting-line inequality gives \(Y\geq0\) almost surely. If \(X\ne m\) at a point where the inequality holds, then the supporting-line inequality is strict there. To see this, equality at some \(z\ne m\) would make the convex graph agree with its supporting line at both \(z\) and \(m\). Convexity and the supporting-line lower bound would then force agreement on the whole segment between them, contradicting strict convexity. Thus \(Y=0\) exactly when \(X=m\), up to null sets. Taking expectations yields \[ \mathbb{E}Y=\mathbb{E}[\varphi(X)]-\varphi(m), \] because \(\mathbb{E}(X-m)=0\). If equality holds in Jensen’s inequality, then \(\mathbb{E}Y=0\). A nonnegative random variable with expectation zero is zero almost surely, so \(X=m\) almost surely. Conversely, if \(X=m\) almost surely, then \(\mathbb{E}[\varphi(X)]=\varphi(m)\) directly. \(\square\)

Worked Example: A Two-Point Variable and a Square

Worked Example: A Two-Point Variable and a Square

Let \(X\) take the values \(0\) and \(2\), each with probability \(1/2\). The function \(\varphi(x)=x^2\) is convex on \(\mathbb{R}\), and both \(X\) and \(\varphi(X)\) are integrable. First calculate the mean: \[ \mathbb{E}X=\tfrac12(0)+\tfrac12(2)=1. \] Then \[ \varphi(\mathbb{E}X)=1^2=1, \qquad \mathbb{E}[\varphi(X)] =\tfrac12(0^2)+\tfrac12(2^2)=2. \] Thus Jensen’s inequality gives \(1\leq2\). The inequality is strict because \(X\) is not constant and \(x^2\) is strictly convex.

Application to Powers

For every real \(p\geq1\), the function \(\varphi(x)=|x|^p\) is convex on \(\mathbb{R}\). Jensen’s inequality therefore gives a useful comparison between the absolute value of the mean and the \(p\)-th absolute moment.

Corollary: If \(X\) is integrable and \(\mathbb{E}|X|^p<\infty\) for some \(p\geq1\), then \[ |\mathbb{E}X|^p\leq\mathbb{E}|X|^p. \]

To verify the hypotheses, \(|X|^p\) is integrable by assumption, and \(X\) is integrable by assumption. The function \(x\mapsto|x|^p\) is convex for \(p\geq1\), so the theorem applies on \(I=\mathbb{R}\). Its conclusion is exactly the displayed bound. For \(p=1\), the result reads \(|\mathbb{E}X|\leq\mathbb{E}|X|\); for larger \(p\), it bounds the mean using a higher absolute moment.

Worked Example: A Third Absolute Moment

Let \(X\) equal \(-1\) or \(3\), each with probability \(1/2\), and take \(p=3\). The mean is \[ \mathbb{E}X=\tfrac12(-1)+\tfrac12(3)=1, \] so \(|\mathbb{E}X|^3=1\). On the other hand, \[ \mathbb{E}|X|^3 =\tfrac12|-1|^3+\tfrac12|3|^3 =\tfrac12(1)+\tfrac12(27)=14. \] Hence \(1\leq14\), as Jensen’s inequality requires. The strict inequality is consistent with the fact that \(X\) is not constant and \(x\mapsto|x|^3\) is strictly convex.

Application to Exponential Expectations

For each fixed \(t\in\mathbb{R}\), the function \(\varphi(x)=e^{tx}\) is convex on \(\mathbb{R}\), since its second derivative is \(t^2e^{tx}\geq0\). If \(X\) is integrable and \(e^{tX}\) is integrable, Jensen’s inequality gives \[ e^{t\mathbb{E}X}\leq\mathbb{E}[e^{tX}]. \] The extra integrability condition is important: integrability of \(X\) alone does not guarantee that its exponential is integrable.

Worked Example: A Symmetric Two-Point Variable

Suppose \(X\) equals \(-a\) and \(a\), each with probability \(1/2\), where \(a>0\). Its mean is zero. For any \(t\in\mathbb{R}\), the exponential of \(tX\) has only two finite values, so it is integrable. Direct calculation gives \[ \mathbb{E}[e^{tX}] =\tfrac12 e^{-ta}+\tfrac12 e^{ta} =\frac{e^{ta}+e^{-ta}}{2}. \] Jensen’s inequality gives \[ e^{t\mathbb{E}X}=e^0=1 \leq \frac{e^{ta}+e^{-ta}}{2}. \] The final inequality can also be checked directly: for \(u=ta\), \(e^u+e^{-u}\geq2\), since \((e^{u/2}-e^{-u/2})^2\geq0\). If \(t\ne0\), the variable \(tX\) is nonconstant and the exponential is strictly convex as a function of its input, so the inequality is strict.

Assumptions and a Common Pitfall

Jensen’s inequality is sometimes remembered only as a formula, while its domain and integrability conditions are overlooked. The formula compares \(\varphi(\mathbb{E}X)\) with \(\mathbb{E}[\varphi(X)]\), so both expressions must be defined. In the theorem above, \(X\) lies in an open interval \(I\), its mean lies in \(I\), and \(\varphi(X)\) is integrable. These conditions ensure the comparison is an inequality between finite real numbers.

The direction also depends on convexity. For a concave function, the corresponding inequality is reversed, provided the expectations exist. Finally, Jensen’s inequality is a general statement about convex functions, not a rule that can be applied to an arbitrary transformation: one must establish convexity on an interval containing the values of \(X\).

Key takeaway: Under appropriate domain and integrability assumptions, convexity gives \[ \varphi(\mathbb{E}X)\leq\mathbb{E}[\varphi(X)]. \] A supporting line at the mean proves the inequality and also identifies when equality is possible.

Check Your Understanding

Use the assumptions and proof strategy above to answer the following questions.

  1. Why must \(\mathbb{E}X\) belong to the domain of \(\varphi\) for Jensen’s inequality to make sense?
  2. What role does the supporting line at \(m=\mathbb{E}X\) play in the proof?
  3. If \(X\) is integrable and \(\mathbb{E}|X|^4<\infty\), what bound does Jensen’s inequality give for \(|\mathbb{E}X|^4\)?
  4. What additional integrability condition is needed to apply Jensen’s inequality to \(e^{tX}\)?
  5. For a strictly convex function, what does equality in Jensen’s inequality imply about \(X\)?

Previous

Next

Back to Tutorials
Clinical Biostats