Finite Jensen Inequality With Proof

From Two Points to Any Finite Weighted Average

Convexity compares the value of a function at a point between two inputs with the corresponding weighted average of its values at those inputs. The finite Jensen inequality extends that comparison to any finite collection of inputs. Its proof is an induction: combine all but one input into a single weighted average, apply the induction hypothesis to that group, and then use two-point convexity to include the remaining input.

We work with a real interval \(I\), a real-valued function \(f:I\to\mathbb{R}\), and finitely many points \(x_1,\ldots,x_n\in I\). No measure space or integral is involved. An interval contains every point between any two of its points, so every weighted average below remains in the domain.

The normalization \(\sum_i\lambda_i=1\) is essential: it makes \(\sum_i\lambda_i x_i\) a weighted average, rather than an arbitrary linear combination. A weight of zero contributes nothing; in particular, the value of \(f\) at an input with zero weight has no effect on either side of the inequality.

The Finite Jensen Inequality

Proof. We prove the assertion by induction on the number \(k\) of positive weights. At least one weight is positive, because all the weights are nonnegative and sum to \(1\). Remove the zero-weight terms and relabel the remaining inputs and weights. It is enough to prove the result when all \(k\) weights are positive: the weighted average and the weighted sum of function values are unchanged by removing zero-weight terms.

For \(k=1\), the only weight is \(1\), so both sides are \(f(x_1)\). Now suppose the result holds for \(k-1\) positive weights, where \(k\ge2\), and consider \(k\) positive weights summing to \(1\). Set \(t=\sum_{i=1}^{k-1}\lambda_i=1-\lambda_k\). Since both the first \(k-1\) weights and \(\lambda_k\) are positive, \(0<t<1\). Define \(\alpha_i=\lambda_i/t\) for \(1\le i\le k-1\). These numbers are positive and satisfy \(\sum_{i=1}^{k-1}\alpha_i=1\). Let \(y=\sum_{i=1}^{k-1}\alpha_i x_i\), which belongs to \(I\) because it is a weighted average of points in the interval \(I\).

By the induction hypothesis, \(f(y)\le\sum_{i=1}^{k-1}\alpha_i f(x_i)\). Also, the full weighted average is \(ty+\lambda_kx_k\), since \(t\alpha_i=\lambda_i\). Two-point convexity, applied with weights \(t\) and \(\lambda_k=1-t\), therefore gives

The first inequality is exactly the definition of convexity, and the second follows by multiplying the induction inequality by \(t>0\) and adding \(\lambda_k f(x_k)\). This proves the result for \(k\), completing the induction. \(\square\)

Equality for Strictly Convex Functions

Proof. Remove all zero-weight terms as in the preceding proof, so every remaining weight is positive. If there is only one remaining input, equality holds, and the condition that all positive-weight inputs are equal is satisfied. If all the inputs are equal to a point \(x\), then their weighted average is \(x\), and the right-hand side is \(\sum_i\lambda_i f(x)=f(x)\). Thus equality holds.

For the converse, use induction on the number \(k\) of positive weights. The case \(k=1\) is already settled. Let \(k\ge2\), and assume equality holds in the finite Jensen inequality. Use \(t,\alpha_i,y\) as in the induction proof. That proof gives a chain \(A\le B\le C\), where \(A=f(ty+\lambda_kx_k)\), \(B=tf(y)+\lambda_kf(x_k)\), and \(C=t\sum_{i=1}^{k-1}\alpha_i f(x_i)+\lambda_k f(x_k)\). Equality in Jensen means \(A=C\). Since \(A\le B\le C\), this forces \(A=B\) and \(B=C\).

Because \(0<t<1\), strict convexity and \(A=B\) imply \(y=x_k\): if \(y\ne x_k\), strict convexity would give \(f(ty+\lambda_kx_k)<tf(y)+\lambda_kf(x_k)\). Also, \(B=C\) and \(t>0\) imply \(f(y)=\sum_{i=1}^{k-1}\alpha_i f(x_i)\). The induction hypothesis applied to the first \(k-1\) inputs shows they are all equal. Their weighted average \(y\) equals that common value, and \(y=x_k\); hence all \(k\) inputs are equal. This proves both directions of the equality condition. \(\square\)

Worked Weighted Averages

Normalization and the Role of Strict Convexity

A frequent error is to use nonnegative coefficients without checking that they sum to \(1\). For a concrete failure, let the convex function be the constant function \(f(x)=1\), take \(x_1=x_2=0\), and use coefficients \(\lambda_1=\lambda_2=\tfrac14\). If one incorrectly applies Jensen without normalization, the alleged inequality becomes \(f(\tfrac14(0)+\tfrac14(0))\le\tfrac14 f(0)+\tfrac14 f(0)\), or \(1\le\tfrac12\), which is false. The correction is to verify the hypothesis first: these coefficients sum to \(\tfrac12\), not \(1\), so the finite Jensen inequality does not apply.

Another mistake is to demand that every listed input be equal in the equality condition. The theorem requires equality only among inputs with positive weights. In the zero-weight example, the input \(100\) does not affect the average or the function-value sum. Conversely, for strictly convex \(f\), two distinct inputs with positive weights force strict inequality, as long as the remaining hypotheses hold. For a merely convex function, distinct inputs can still give equality, as the absolute-value example demonstrates.

The finite inequality is useful whenever a quantity is formed as a finite weighted average: it converts the value of a convex function at that average into a bound by the average of the function values. The proof shows precisely why the two-point definition is enough, and the equality condition identifies when that bound is sharp for strictly convex functions.