Linearity Under Absolute Integrability

Why Linearity Needs an Existence Condition

For a finite-support random variable, expectation is a finite weighted sum, so rearranging terms establishes linearity directly. With infinitely many possible values—or a continuous distribution—expectations involve infinite sums or integrals. Rearranging or combining those expressions is safe only when the expectations are well-defined. The key condition is absolute integrability.

Linearity does not require independence. It requires that the random variables being combined have finite absolute expectations. Under that condition, the expectation of their sum exists, and the usual algebraic rule is valid.

Definition: A real-valued random variable \(X\) is integrable if \(E[|X|]<\infty\). Its expectation is then the finite, well-defined signed integral \(E[X]=\int_{\Omega}X\,dP\). Equivalently, writing \(X^+=\max(X,0)\) and \(X^-=\max(-X,0)\), both \(E[X^+]\) and \(E[X^-]\) are finite and \(E[X]=E[X^+]-E[X^-]\).

The absolute value in the definition matters: positive and negative contributions must each have a finite total, not merely appear to cancel. When \(X\) has a probability mass function \(f\), this condition is \(\sum_x |x|f(x)<\infty\); when it has a density \(f\), it is \(\int_{\mathbb R}|x|f(x)\,dx<\infty\). For a general distribution, expectation is the integral with respect to its probability measure.

Integrability Is Preserved by Finite Linear Combinations

Before adding expectations, check that the random variable on the left has a finite absolute expectation. For any real numbers \(a_1,\ldots,a_m\) and random variables \(X_1,\ldots,X_m\), the triangle inequality gives

\left|\sum_{j=1}^{m}a_jX_j\right| \leq \sum_{j=1}^{m}|a_j|\,|X_j|.

Thus, if every \(X_j\) is integrable, the right side has finite expectation. This establishes existence before any algebra is done.

Theorem: Let \(X_1,\ldots,X_m\) be integrable real-valued random variables on the same probability space, and let \(a_1,\ldots,a_m\) be real constants. Then \(\sum_{j=1}^{m}a_jX_j\) is integrable and \[ E\!\left[\sum_{j=1}^{m}a_jX_j\right] =\sum_{j=1}^{m}a_jE[X_j]. \] No independence assumption is needed.

Proof. First, the pointwise triangle inequality and monotonicity of the integral for nonnegative functions imply

E\!\left[\left|\sum_{j=1}^{m}a_jX_j\right|\right] \leq \sum_{j=1}^{m}|a_j|E[|X_j|]<\infty.

So the random variable on the left is integrable. For additivity, it suffices to establish the rule for two integrable random variables \(X\) and \(Y\). Their positive and negative parts have finite expectations, and pointwise

$$ X+Y=(X^++Y^+)-(X^-+Y^-). $$

Both grouped terms are nonnegative and integrable. Additivity of the integral for nonnegative integrable functions therefore gives

\begin{aligned} E[X+Y] &=E[X^++Y^+]-E[X^-+Y^-]\\ &=E[X^+]+E[Y^+]-E[X^-]-E[Y^-]\\ &=E[X]+E[Y]. \end{aligned}

For any real \(a\), \(aX\) is integrable because \(E[|aX|]=|a|E[|X|]<\infty\). If \(a\geq 0\), its positive and negative parts are \(aX^+\) and \(aX^-\); if \(a<0\), they are \((-a)X^-\) and \((-a)X^+\). In either case, the signed-integral definition gives \(E[aX]=aE[X]\). Applying this scalar rule and two-variable additivity repeatedly proves the displayed result for \(m\) terms. \(\square\)

The proof uses only integrability and pointwise algebra. In particular, \(X\) and \(Y\) may be dependent, and their distributions need not be of the same type. If a joint distribution is used to calculate the expectation, the same result follows by integrating the sum with respect to that joint distribution.

How Linearity Appears in Sums and Integrals

The theorem can be read in the forms most useful for calculations. For jointly discrete random variables with joint mass function \(p(x,y)\), if \(E[|X|]\) and \(E[|Y|]\) are finite, then

E[X+Y]=\sum_{x,y}(x+y)p(x,y) =\sum_{x,y}xp(x,y)+\sum_{x,y}yp(x,y) =E[X]+E[Y].

The absolute expectations ensure these sums are absolutely convergent, so splitting the summand does not rely on a conditionally convergent rearrangement. If \((X,Y)\) has a joint density \(f_{X,Y}\), the corresponding expression is

E[X+Y]=\int_{\mathbb R^2}(x+y)f_{X,Y}(x,y)\,dx\,dy =E[X]+E[Y],

provided \(E[|X|]\) and \(E[|Y|]\) are finite. Indeed, \(|x+y|\leq |x|+|y|\), so the integral of the absolute value of the sum is finite. These are particular ways of evaluating the general probability-space integral; a density is not required for the theorem.

Worked Example: A Difference of Infinite-Support Discrete Variables

Let \(X\) and \(Y\) be independent with \(P(X=k)=P(Y=k)=2^{-(k+1)}\) for \(k=0,1,2,\ldots\). Find \(E[X-Y]\), and verify that the expectations used are well-defined.

The probabilities sum to one because \(\sum_{k=0}^{\infty}2^{-(k+1)}=1\). For either variable,

E[X]=\sum_{k=0}^{\infty}\frac{k}{2^{k+1}}=1, \qquad E[Y]=1.

For the sum calculation, use \(\sum_{k=0}^{\infty}kr^k=r/(1-r)^2\) for \(0\leq r<1\). At \(r=1/2\), the expectation is \(\frac12\sum_{k=0}^{\infty}k(1/2)^k=\frac12\cdot 2=1\). Thus \(E[|X|]\) and \(E[|Y|]\) are finite. Also,

E[|X-Y|]\leq E[X+Y]=E[X]+E[Y]=2<\infty.

The difference is integrable, so linearity applies:

$$ E[X-Y]=E[X]-E[Y]=1-1=0. $$

Independence specifies the model but is not needed for this final linearity step.

Worked Example: Dependent Continuous Variables

Let \(U\sim\operatorname{Uniform}(-1,1)\), and define \(X=U\) and \(Y=U^2\). Find \(E[2X-3Y]\).

The variables are dependent because both are functions of the same \(U\). Their absolute expectations are finite: \(|U|\leq 1\) and \(U^2\leq 1\). Using the density \(1/2\) on \((-1,1)\),

\begin{aligned} E[X]&=\frac12\int_{-1}^{1}u\,du =\frac12\left[\frac{u^2}{2}\right]_{-1}^{1}=0,\\ E[Y]&=\frac12\int_{-1}^{1}u^2\,du =\frac12\left[\frac{u^3}{3}\right]_{-1}^{1} =\frac12\cdot\frac23=\frac13. \end{aligned}

The proposed linear combination is integrable since \(|2U-3U^2|\leq 2|U|+3U^2\leq 5\). Therefore,

E[2X-3Y]=2E[X]-3E[Y]=2(0)-3\left(\frac13\right)=-1.

As a check, direct integration gives \(\frac12\int_{-1}^{1}(2u-3u^2)\,du=0-1=-1\), in agreement with linearity.

Worked Example: A Sum Under a Mixed Distribution

Suppose \(X\) has probability \(1/4\) at zero and, with probability \(3/4\), is uniform on \((0,2)\). Set \(Z=\mathbf{1}\{X>1\}\). Find \(E[X+2Z]\).

The density of the continuous part is \((3/4)(1/2)=3/8\) on \((0,2)\), alongside the atom of size \(1/4\) at zero. The atom contributes zero to \(E[X]\), so

E[X]=\frac38\int_0^2 x\,dx =\frac38\left[\frac{x^2}{2}\right]_0^2 =\frac38\cdot 2=\frac34.

The event \(X>1\) lies in the continuous part, and hence

E[Z]=P(X>1)=\frac38\int_1^2 1\,dx=\frac38.

Both variables are integrable: \(E[|X|]=3/4\), and \(Z\) is bounded by one. Linearity now gives

E[X+2Z]=E[X]+2E[Z]=\frac34+2\left(\frac38\right)=\frac32.

Directly, the atom contributes zero to this expectation, while the continuous part contributes \(\frac38\int_0^2 (x+2\mathbf{1}\{x>1\})\,dx =\frac38(2+2)=\frac32\). The mixed calculation and the linearity calculation agree.

Common Pitfall: Treating Cancellation as Integrability

Linearity is not a rule for adding arbitrary expressions that look like expectations. For example, let \(C\) have the standard Cauchy density \(f(c)=1/(\pi(1+c^2))\). Its absolute first moment is infinite:

E[|C|]=\frac{2}{\pi}\int_0^\infty\frac{c}{1+c^2}\,dc=\infty.

The integral diverges because its value up to \(R\) is \(\frac{1}{\pi}\log(1+R^2)\), which tends to infinity. Although symmetry gives a Cauchy principal value of zero, that principal value is not an expectation: the positive and negative parts both have infinite expectations. In particular, one cannot claim \(E[C]+E[-C]=0\), because neither expectation is defined. The random variable \(C+(-C)\) is identically zero and does have expectation zero, but this fact does not make the two separate expectations exist.

The correction is to check absolute integrability of each term before applying the linearity theorem. The theorem gives a sufficient condition, not a necessary one: two nonintegrable terms can cancel pointwise to produce an integrable sum, but that does not license adding their undefined expectations.

Key takeaway: If each variable in a finite linear combination has finite absolute expectation, then the combination is integrable and its expectation is the same linear combination of the individual expectations. Dependence does not prevent this result; undefined expectations do.

Check Your Understanding

Use absolute integrability as the first check in each problem.

If \(E[|X|]=2\) and \(E[|Y|]=5\), what bound can you give for \(E[|3X-2Y|]\)?
Does linearity require \(X\) and \(Y\) to be independent? State the condition it does require.
Suppose a jointly discrete pair has \(E[|X|]<\infty\) and \(E[|Y|]<\infty\). Why is it legitimate to split the sum for \(E[X+Y]\) into the two marginal expectations?
Why does a symmetric principal value of zero not establish that a Cauchy random variable has expectation zero?
If \(X\) and \(Y\) are both nonintegrable but \(X+Y=0\) with probability one, which expectation is defined, and why can the linearity theorem not be applied to \(E[X]+E[Y]\)?

Linearity Under Absolute Integrability

What You'll Learn