Tutorials › Statistical Theory I - Probability and Distribution Theory › Tangent-Line Bounds for Expectations

Home  /  Tutorials  /  Moment Inequalities and Jensen

Moment Inequalities and Jensen · Tutorial 386 of 994

Tangent-Line Bounds for Expectations

Use supporting tangent lines to turn convexity or concavity into explicit expectation bounds, and check when those bounds are valid and sharp.

Intermediate 9 min read

What You'll Learn

  • Transfer a pointwise affine inequality to an expectation under explicit integrability conditions.
  • Prove when a differentiable convex or concave function lies on one side of its tangent line.
  • Derive expectation bounds from a tangent taken at any point in the domain.
  • Calculate and compare tangent-line bounds for discrete random variables.
  • Recognize why a tangent that is only locally valid cannot be used as a global expectation bound.

From Pointwise Bounds to Expectation Bounds

A pointwise inequality can become a useful statement about averages. If a function of a random variable is always above a line, then its expectation is at least the expectation of that line, provided the relevant expectations exist. Tangent lines supply such affine bounds for convex and concave functions. The point of this method is not merely to identify a direction of inequality: choosing where to draw the tangent determines the numerical bound.

The distinction between a tangent that is globally valid and one that is only a local approximation matters. Convexity or concavity on the entire range of the random variable gives the global guarantee needed for an expectation bound.

Definition: An affine function of \(x\) has the form \(a+bx\), where \(a\) and \(b\) are constants. It is a pointwise lower bound for \(g\) on a set \(S\) if \(g(x)\ge a+bx\) for every \(x\in S\); it is a pointwise upper bound if \(g(x)\le a+bx\) throughout \(S\).
Theorem: Suppose \(X\) is integrable, \(P(X\in S)=1\), and \(g(X)\) is integrable. If \(g(x)\ge a+bx\) for all \(x\in S\), then
$$ E[g(X)]\ge a+bE[X]. $$
If instead \(g(x)\le a+bx\) for every \(x\in S\), the expectation inequality is reversed.

Proof. In the lower-bound case, the random variable \(g(X)-(a+bX)\) is nonnegative almost surely. It is integrable because both \(g(X)\) and \(a+bX\) are integrable. Taking expectations preserves its nonnegativity, so

$$ 0\le E[g(X)-(a+bX)] =E[g(X)]-a-bE[X]. $$

Rearranging gives the claimed inequality. Applying the same argument to \((a+bX)-g(X)\) proves the upper-bound case. \(\square\)

Tangent Lines to Convex and Concave Functions

Let \(I\) be an open interval, and let \(g:I\to\mathbb{R}\) be differentiable. If \(g\) is convex on \(I\), its tangent line at any \(t\in I\) is a global lower bound on \(I\). If \(g\) is concave, the tangent line is a global upper bound. A tangent line at \(t\) is the affine function \(g(t)+g'(t)(x-t)\).

Theorem: Let \(g\) be differentiable and convex on an open interval \(I\). For every \(t,x\in I\),
$$ g(x)\ge g(t)+g'(t)(x-t). $$
If \(g\) is differentiable and concave on \(I\), the inequality is reversed.

Proof. For a differentiable convex function, \(g'\) is nondecreasing on \(I\). First take \(x>t\). The mean value theorem gives a point \(c\in(t,x)\) such that

$$ g(x)-g(t)=g'(c)(x-t). $$

Since \(g'(c)\ge g'(t)\) and \(x-t>0\), this difference is at least \(g'(t)(x-t)\). Now take \(x<t\). The mean value theorem gives \(c\in(x,t)\) with the same difference formula. Here \(g'(c)\le g'(t)\), while \(x-t<0\); multiplying by this negative quantity again gives

$$ g'(c)(x-t)\ge g'(t)(x-t). $$

The result also holds at \(x=t\), where equality is immediate. For concave \(g\), apply the convex result to \(-g\) and multiply by \(-1\), which reverses the inequality. \(\square\)

Combining the tangent-line result with the expectation theorem gives a bound at any chosen tangent point. The random variable must lie in the interval where the function has the stated curvature, and the expectations in the formula must exist.

Expectation bound from a tangent: If \(g\) is differentiable and convex on \(I\), \(P(X\in I)=1\), \(E[X]\) is finite, and \(g(X)\) is integrable, then for every \(t\in I\),
$$ E[g(X)]\ge g(t)+g'(t)\bigl(E[X]-t\bigr). $$
For concave \(g\), the inequality is reversed.

The tangent point need not equal \(E[X]\). Choosing \(t=E[X]\), when this point belongs to \(I\), makes the right-hand side \(g(E[X])\); choosing another \(t\) can give a different bound. Thus the formula is useful both as a direct consequence of curvature and as a way to obtain bounds adapted to a particular calculation.

Worked Tangent-Line Bounds

Worked Example: A Quadratic Lower Bound at Different Tangency Points

Let \(X\) take values \(1,3,5\) with probabilities \(1/4,1/2,1/4\), respectively. Use tangents to \(g(x)=x^2\) to bound \(E[X^2]\).

First calculate the mean and second moment directly:

$$ E[X]=\frac14(1)+\frac12(3)+\frac14(5)=3, \qquad E[X^2]=\frac14(1)+\frac12(9)+\frac14(25)=11. $$

The function is convex, and its tangent at \(t\) is \(t^2+2t(x-t)=2tx-t^2\). At \(t=2\), this gives \(x^2\ge4x-4\), so

$$ E[X^2]\ge4E[X]-4=4(3)-4=8. $$

At \(t=3\), the tangent gives \(x^2\ge6x-9\), and therefore \(E[X^2]\ge6(3)-9=9\). The actual value, \(11\), satisfies both bounds. More generally, the bound at \(t\) is \(6t-t^2=9-(t-3)^2\), which is largest at \(t=3\). The calculation shows how the tangent point affects the strength of the bound.

Worked Example: A Concave Tangent Gives an Upper Bound

Let \(X\) equal \(0\) or \(4\), each with probability \(1/2\), and take \(g(x)=\sqrt{1+x}\) on \(I=(-1,\infty)\). This function is concave because

$$ g'(x)=\frac{1}{2\sqrt{1+x}}, \qquad g''(x)=-\frac{1}{4(1+x)^{3/2}}<0. $$

The mean is \(E[X]=2\), and the expectation being bounded is

$$ E[g(X)]=\frac12\sqrt{1}+\frac12\sqrt{5} =\frac{1+\sqrt{5}}{2}. $$

Use the tangent at \(t=1\). Its value at \(x\) is \(\sqrt{2}+(x-1)/(2\sqrt{2})\). Concavity makes this a pointwise upper bound, so

$$ E[g(X)]\le \sqrt{2}+\frac{E[X]-1}{2\sqrt{2}} =\sqrt{2}+\frac{1}{2\sqrt{2}} =\frac{5\sqrt{2}}{4}. $$

Numerically, the exact expectation is about \(1.618\), while the bound is about \(1.768\), so the inequality has the required direction. Taking \(t=2=E[X]\) instead gives the upper bound \(\sqrt{3}\), about \(1.732\). Both bounds are valid; they arise from different tangent lines.

Worked Example: A Cubic Bound on a Nonnegative Support

Suppose \(P(X=1)=3/4\) and \(P(X=3)=1/4\). For \(g(x)=x^3\), the second derivative is \(g''(x)=6x\), which is nonnegative for \(x\ge0\). Thus \(g\) is convex on the support interval \((0,\infty)\). The tangent at \(t=2\) is \(8+12(x-2)=12x-16\), and its difference from the function factors as

$$ x^3-(12x-16)=(x-2)^2(x+4)\ge0 \qquad (x\ge0). $$

The mean and third moment are

$$ E[X]=\frac34(1)+\frac14(3)=\frac32, \qquad E[X^3]=\frac34(1)+\frac14(27)=\frac{15}{2}. $$

The tangent inequality yields \(E[X^3]\ge12E[X]-16=18-16=2\), which is true but not sharp: \(15/2\ge2\). The factorization also identifies why the support condition matters. For \(x<-4\), the factor \(x+4\) is negative, so this tangent is not a lower bound there. It can be used here because \(X\) is nonnegative, not because every tangent to a cubic is a global lower bound on the real line.

When the Bound Is Sharp—and a Common Pitfall

The gap between a convex function and its tangent at \(t\) is nonnegative at every point in the interval. Consequently, the expectation bound is an equality precisely when that gap is zero almost surely, assuming the gap is integrable. For a strictly convex function, the tangent touches the graph only at \(t\), so equality requires \(P(X=t)=1\). For a function that is not strictly convex, it may touch the tangent at additional points, and \(X\) can be supported on those points as well.

A common mistake is to draw a tangent and assume that it bounds the function everywhere. A tangent is always a local first-order approximation when the derivative exists, but it is a global lower or upper bound only with an appropriate curvature condition on the whole range under consideration. The cubic example makes this distinction concrete: its tangent at \(2\) is a lower bound for nonnegative \(x\), but not for all real \(x\). Before taking expectations, check the support of \(X\), the curvature on that support, and the integrability of the quantities involved.

The method is useful whenever a nonlinear expectation is difficult to compute but a suitable affine bound is easy to average. It converts information about the mean into a bound on the expectation of a transformed variable, while keeping the assumptions visible. The direction comes from convexity or concavity; the numerical value comes from the chosen tangent point.

Check Your Understanding

For each question, identify the support and the direction of the tangent inequality before taking an expectation.

  1. If \(g\) is differentiable and convex on an interval containing all possible values of \(X\), what pointwise inequality does its tangent at \(t\) satisfy?
  2. Suppose \(E[X]=4\) and \(g(x)=x^2\). What lower bound does the tangent at \(t=1\) give for \(E[X^2]\)?
  3. Why does a concave tangent produce an upper bound on \(E[g(X)]\), rather than a lower bound?
  4. For a convex function, what condition on the tangent gap makes the expectation bound an equality?
  5. Why is convexity on the support of \(X\) relevant when using a tangent line to bound \(E[g(X)]\)?

Previous

Next

Back to Tutorials