Conditional Jensen's Inequality

Why Condition on Information?

Ordinary Jensen’s inequality compares $E[\phi(X)]$ with $\phi(E[X])$ when $\phi$ is convex. In many statistical problems, however, information about another random variable $Y$ is available. The relevant comparison then uses the mean of $X$ given that information: does averaging $\phi(X)$ conditionally still dominate applying $\phi$ to the conditional mean? It does, under the appropriate integrability conditions.

The result is useful whenever the distribution of $X$ varies with an observed covariate, group, or latent state. It says that the convexity bound holds separately within each conditional distribution, rather than only after all groups have been pooled.

Conditional Distributions and the Inequality

For real-valued $X$ and $Y$, a conditional distribution of $X$ given $Y$ is a probability kernel $K(y,A)$: for each $y$, $A\mapsto K(y,A)$ is a probability distribution on the real line, and for each Borel set $A$, $y\mapsto K(y,A)$ is measurable. It represents the conditional law in the sense that, for Borel sets $A$ and $B$,

P(X\in A,\ Y\in B)=\int_B K(y,A)\,P_Y(dy),

where $P_Y$ is the distribution of $Y$. Such a conditional distribution exists for real-valued random variables. For a measurable function $g$ with $E[|g(X)|]<\infty$, integration against the kernel gives a version of the conditional expectation:

E[g(X)\mid Y]=\int g(x)\,K(Y,dx) \quad \text{almost surely}.

Here conditioning on $Y$ means conditioning on the information $\sigma(Y)$ generated by $Y$. In a continuous model, the event $Y=y$ may have probability zero; the conditional distribution is described by a kernel, not by an ordinary ratio involving $P(Y=y)$.

Theorem: Let $X$ and $Y$ be real-valued random variables. Let $I$ be an interval, let $X\in I$ almost surely, and let $\phi:I\to\mathbb{R}$ be convex. Suppose $E[|X|]<\infty$, $E[|\phi(X)|]<\infty$, and $E[X\mid Y]\in I$ almost surely. Then $$ E[\phi(X)\mid Y]\geq \phi(E[X\mid Y]) \quad \text{almost surely}. $$

The requirement $E[X\mid Y]\in I$ ensures that $\phi$ is defined at the conditional mean. It holds, for example, when $I$ is a closed interval and $X\in I$ almost surely, or when $I$ is open and $X\in I$ almost surely. The statement is an almost-sure inequality: versions of conditional expectations can differ on probability-zero sets, so the theorem does not assert an inequality at every possible numerical value of $Y$.

Proof Using Conditional Distributions

Write $K(y,dx)$ for a conditional distribution of $X$ given $Y=y$. Integrability of $X$ and $\phi(X)$ implies that, for $P_Y$-almost every $y$, the conditional integrals of $|x|$ and $|\phi(x)|$ are finite. Also, $K(y,I)=1$ for $P_Y$-almost every $y$, since $P(X\in I)=1$. On a common set of full $P_Y$-measure, define

m(y)=\int x\,K(y,dx), \qquad q(y)=\int \phi(x)\,K(y,dx).

By the conditional-distribution identity, $m(Y)$ is a version of $E[X\mid Y]$, and $q(Y)$ is a version of $E[\phi(X)\mid Y]$. For almost every such $y$, $K(y,\cdot)$ is a probability distribution supported on $I$, $x$ is integrable under it, and $\phi(x)$ is integrable under it. Ordinary Jensen’s inequality applied to that probability distribution gives

\int \phi(x)\,K(y,dx) \geq \phi\left(\int x\,K(y,dx)\right).

Thus $q(y)\geq \phi(m(y))$ for $P_Y$-almost every $y$. Substituting the conditional-expectation versions at $Y$ proves the stated inequality almost surely. $\square$

Worked Applications

Worked Example: A Bernoulli Variable Given a Group

Suppose $P(Y=0)=0.4$ and $P(Y=1)=0.6$. Conditional on $Y=0$, let $X$ be Bernoulli with success probability $0.2$; conditional on $Y=1$, let it be Bernoulli with success probability $0.8$. Apply conditional Jensen’s inequality with $\phi(x)=x^2$.

The function $x^2$ is convex. For a Bernoulli variable, $X^2=X$, so within either group $E[X^2\mid Y]=E[X\mid Y]$. Therefore,

\begin{aligned} Y=0:\quad & E[X^2\mid Y=0]=0.2 \geq (E[X\mid Y=0])^2=(0.2)^2=0.04,\\ Y=1:\quad & E[X^2\mid Y=1]=0.8 \geq (E[X\mid Y=1])^2=(0.8)^2=0.64. \end{aligned}

The two conditional gaps are $0.2-0.04=0.16$ and $0.8-0.64=0.16$. Averaging over the groups gives

E[X^2]=0.4(0.2)+0.6(0.8)=0.56 \geq 0.4(0.04)+0.6(0.64)=0.40 =E[(E[X\mid Y])^2].

The pointwise conditional comparisons are the theorem’s main content; the unconditional comparison follows by averaging them.

Worked Example: A Continuous Conditional Distribution

Let $Y\sim \operatorname{Uniform}(1,3)$, let $U\sim \operatorname{Uniform}(0,1)$ be independent of $Y$, and set $X=YU$. Given $Y=y$, $X$ is uniform on $(0,y)$. Find both sides of conditional Jensen’s inequality for $\phi(x)=x^2$.

For a uniform random variable on $(0,y)$, integration gives

E[X\mid Y=y]=\frac{1}{y}\int_0^y x\,dx=\frac{y}{2}, \qquad E[X^2\mid Y=y]=\frac{1}{y}\int_0^y x^2\,dx=\frac{y^2}{3}.

Consequently, for $Y>0$,

E[X^2\mid Y]=\frac{Y^2}{3} \geq \left(E[X\mid Y]\right)^2 =\frac{Y^2}{4},

because $1/3\geq 1/4$. The gap is $Y^2/12$, which is strictly positive almost surely. This example also illustrates why conditioning on a continuous variable is not limited to conditioning on positive-probability events: the conditional distribution is specified for almost every $y$ through its kernel.

Worked Example: A Discrete Conditional Law With Continuous Conditioning

Let $Y\sim\operatorname{Uniform}(0,1)$. Independently, let $S$ take the values $-1$ and $1$ with probability $1/2$ each, and set $X=Y+S$. For the convex function $\phi(x)=e^x$, calculate the two conditional quantities.

Given $Y=y$, $X$ takes $y-1$ and $y+1$ with equal probabilities. Hence

E[X\mid Y=y]=\frac{(y-1)+(y+1)}{2}=y

and

E[e^X\mid Y=y] =\frac{e^{y-1}+e^{y+1}}{2} =e^y\frac{e^{-1}+e}{2} \geq e^y =e^{E[X\mid Y=y]}.

The inequality is strict because $e+e^{-1}>2$: indeed, $(e^{1/2}-e^{-1/2})^2>0$ implies $e+e^{-1}-2>0$. Thus the result applies whether the conditional law is continuous or discrete; what matters is that each conditional law has a finite mean and the required integrability.

Averaging the Conditional Bound

Taking expectations of the conditional inequality recovers an unconditional comparison. More precisely, under the theorem’s assumptions, and when $E[|\phi(E[X\mid Y])|]<\infty$,

Corollary: $$ E[\phi(X)]\geq E[\phi(E[X\mid Y])]\geq \phi(E[X]). $$

For the first inequality, take expectations on both sides of $E[\phi(X)\mid Y]\geq\phi(E[X\mid Y])$ and use the defining property $E[E[\phi(X)\mid Y]]=E[\phi(X)]$. For the second, apply ordinary Jensen’s inequality to the integrable random variable $E[X\mid Y]$: its expectation is $E[X]$ by the law of iterated expectations, and its transformed expectation is finite by assumption. This proves the displayed chain.

The middle term can be larger than $\phi(E[X])$, because conditioning preserves variation in the conditional means across values of $Y$. Conditional Jensen’s inequality first accounts for variation within each conditional distribution; ordinary Jensen’s inequality then accounts for variation among the conditional means.

Common Mistake: Treating Conditioning as Substitution

A tempting but incorrect argument is: “Given $Y$, replace $X$ by its conditional mean, so $E[\phi(X)\mid Y]=\phi(E[X\mid Y])$.” This drops the variation of $X$ that remains after $Y$ is known. For a convex $\phi$, that variation is exactly why the conditional expectation of $\phi(X)$ can be larger.

For example, in the Bernoulli model with $Y=0$, the conditional mean is $0.2$, but $E[X^2\mid Y=0]=0.2$, not $(0.2)^2=0.04$. The correct conclusion is the inequality $E[\phi(X)\mid Y]\geq\phi(E[X\mid Y])$, not equality in general. Equality requires additional conditions on the conditional distribution and the convex function.

A second error is to claim that the inequality holds for every numerical value $y$, even where a chosen conditional-distribution version is undefined or arbitrary. The theorem guarantees the comparison almost surely under the distribution of $Y$; values in a probability-zero set do not affect that assertion.

Check Your Understanding

Use the assumptions and conditional-distribution argument above to answer these questions.

For convex $\phi$, state the almost-sure comparison in conditional Jensen’s inequality.
In the Bernoulli example, calculate $E[X^2\mid Y=1]$ and $(E[X\mid Y=1])^2$. Why are they unequal?
If $X\mid Y=y$ is uniform on $(0,y)$ for $y>0$, calculate $E[X\mid Y=y]$ and $E[X^2\mid Y=y]$, then compare the latter with the square of the former.
Why does a continuous conditioning variable not require the event $Y=y$ to have positive probability?
Which property of conditional expectation lets the conditional inequality imply $E[\phi(X)]\geq E[\phi(E[X\mid Y])]$?