Proving Linearity for Finite-Support Variables

Why Linearity Can Be Proved Directly

When random variables are added, their probability distribution can be complicated: it depends on how the variables occur together. Their expectation, however, has a simple structure. For variables with finite support, that structure follows from rearranging finite sums. No limit argument is needed, and independence is not required.

The finite-support setting makes the reasoning transparent. An expectation is a weighted sum of possible values, with probabilities as weights. Scaling a variable scales each value in that sum. Adding variables lets us regroup a sum over joint outcomes into the separate sums for each variable. The finiteness of the support ensures that all these rearrangements are valid.

Finite-Support Expectations

Definition: A random variable $X$ has finite support if there is a finite set $S_X$ such that $P(X\in S_X)=1$. For each $x\in S_X$, let $p_X(x)=P(X=x)$. Its expectation is the finite sum $$ E[X]=\sum_{x\in S_X}x\,p_X(x). $$ The support may be taken to contain just the values with positive probability. If a finite set includes additional values with probability zero, the expectation is unchanged.

For two finite-support random variables $X$ and $Y$, the pair $(X,Y)$ also has finite support: it lies in $S_X\times S_Y$ with probability one, and that product set is finite. Write $p_{X,Y}(x,y)=P(X=x,Y=y)$ for the joint probability mass function. The marginal masses are obtained by summing over the other coordinate, for example $p_X(x)=\sum_y p_{X,Y}(x,y)$. These facts allow the expectation of a sum to be written as a finite sum over joint outcomes.

Scalar Multiplication

Theorem: If $X$ has finite support and $c$ is a real number, then $cX$ has finite support and $$ E[cX]=cE[X]. $$

First suppose $c\ne 0$. Distinct support values of $X$ remain distinct after multiplication by $c$, and $P(cX=cx)=P(X=x)=p_X(x)$. Therefore the expectation of $cX$ is

E[cX]=\sum_{x\in S_X}(cx)p_X(x) =c\sum_{x\in S_X}x p_X(x) =cE[X].

Every sum here is finite, so taking $c$ outside the sum is an ordinary algebraic operation. If $c=0$, then $cX$ takes only the value $0$. Directly from the finite-support definition,

E[0X]=0\cdot P(X\in S_X)=0=0E[X].

Thus the identity holds for every real $c$, including zero. In either case, scaling a finite set of possible values produces a finite set, so $cX$ is also finite-support. $\square$

Additivity

Theorem: If $X$ and $Y$ have finite support, then $X+Y$ has finite support and $$ E[X+Y]=E[X]+E[Y]. $$ No independence assumption is needed.

To prove the identity, use the joint probability mass function. The variable $X+Y$ takes values in the finite set $\{x+y:x\in S_X,\ y\in S_Y\}$. Its expectation can be calculated by summing the value $x+y$ over joint outcomes:

\begin{aligned} E[X+Y] &=\sum_{x\in S_X}\sum_{y\in S_Y}(x+y)p_{X,Y}(x,y)\\ &=\sum_{x\in S_X}\sum_{y\in S_Y}x\,p_{X,Y}(x,y) +\sum_{x\in S_X}\sum_{y\in S_Y}y\,p_{X,Y}(x,y)\\ &=\sum_{x\in S_X}x\left(\sum_{y\in S_Y}p_{X,Y}(x,y)\right) +\sum_{y\in S_Y}y\left(\sum_{x\in S_X}p_{X,Y}(x,y)\right)\\ &=\sum_{x\in S_X}x\,p_X(x)+\sum_{y\in S_Y}y\,p_Y(y)\\ &=E[X]+E[Y]. \end{aligned}

The second line splits each summand into two terms, and the next line regroups the finite sums by $x$ and $y$. The inner sums are the marginal probabilities, which gives the stated expectations. Since the sums are finite, the regrouping requires no additional convergence condition. The proof uses the joint distribution but nowhere assumes that it factors into a product of marginal distributions. $\square$

Combining the two identities also gives $E[aX+bY]=aE[X]+bE[Y]$ for finite-support $X,Y$ and real $a,b$: apply scalar multiplication to each variable and additivity to the resulting pair. Repeating additivity gives the corresponding identity for any finite number of finite-support random variables.

Worked Applications

Worked Example: Additivity Without Independence

Suppose the joint outcomes of $(X,Y)$ are $(0,0)$ with probability $0.25$, $(1,1)$ with probability $0.50$, and $(2,0)$ with probability $0.25$. These probabilities are nonnegative and sum to $0.25+0.50+0.25=1$. Calculate $E[X]$, $E[Y]$, and $E[X+Y]$.

The marginal expectation of $X$ is

$$ E[X]=0(0.25)+1(0.50)+2(0.25)=0+0.50+0.50=1. $$

The marginal expectation of $Y$ is

$$ E[Y]=0(0.25)+1(0.50)+0(0.25)=0.50. $$

The three possible sums $X+Y$ are $0,2,2$, with probabilities $0.25,0.50,0.25$. Thus

$$ E[X+Y]=0(0.25)+2(0.50)+2(0.25)=0+1+0.50=1.50. $$

Indeed, $E[X]+E[Y]=1+0.50=1.50$. Here $X$ and $Y$ are dependent: for example, $P(X=2,Y=0)=0.25$, whereas $P(X=2)P(Y=0)=0.25(0.50)=0.125$. The additivity calculation still holds.

Worked Example: Sum of Independent Bernoulli Variables

Let $X$ and $Y$ be independent Bernoulli variables with $P(X=1)=0.4$ and $P(Y=1)=0.7$. Each variable takes values $0$ and $1$. Calculate the expectation of their sum both from its distribution and from additivity.

Independence gives the four joint probabilities:

\begin{aligned} P(X=0,Y=0)&=(0.6)(0.3)=0.18,\\ P(X=1,Y=0)&=(0.4)(0.3)=0.12,\\ P(X=0,Y=1)&=(0.6)(0.7)=0.42,\\ P(X=1,Y=1)&=(0.4)(0.7)=0.28. \end{aligned}

These add to $0.18+0.12+0.42+0.28=1$. The sum is $0$ in the first outcome, $1$ in the second and third, and $2$ in the fourth. Therefore

$$ E[X+Y]=0(0.18)+1(0.12+0.42)+2(0.28) =0+0.54+0.56=1.10. $$

Separately, $E[X]=0(0.6)+1(0.4)=0.4$ and $E[Y]=0(0.3)+1(0.7)=0.7$, so $E[X]+E[Y]=0.4+0.7=1.10$, matching the direct calculation. Independence was useful for finding the joint probabilities in this example, but it is not a condition for the additivity identity itself.

Worked Example: A Linear Combination From a Joint Distribution

Suppose $(X,Y)$ takes the values $(-1,0)$, $(0,2)$, and $(2,1)$ with probabilities $0.2$, $0.5$, and $0.3$, respectively. Find $E[2X-3Y]$, first by computing the transformed values and then by using the two expectation identities.

At the three joint outcomes, $2X-3Y$ takes the values $-2$, $-6$, and $1$. Direct calculation gives

$$ E[2X-3Y]=(-2)(0.2)+(-6)(0.5)+1(0.3) =-0.4-3+0.3=-3.1. $$

For comparison, compute the separate expectations:

$$ E[X]=(-1)(0.2)+0(0.5)+2(0.3)=-0.2+0+0.6=0.4, $$

$$ E[Y]=0(0.2)+2(0.5)+1(0.3)=0+1+0.3=1.3. $$

Scalar multiplication and additivity give $E[2X-3Y]=2E[X]-3E[Y]$, and the right-hand side is $2(0.4)-3(1.3)=0.8-3.9=-3.1$. The two calculations agree.

Common Pitfall and Why the Result Matters

A frequent error is to think that additivity requires independence, or to confuse the expectation of a sum with a product of expectations. For example, an argument such as “$E[X+Y]=E[X]+E[Y]$ only if $X$ and $Y$ are independent” imposes an unnecessary condition. The proof above shows exactly what is required here: finite support, so that the defining joint sum is finite and can be regrouped. The dependent example satisfies additivity as well.

Another mistake is to calculate $E[X+Y]$ by treating $X$ and $Y$ as if their marginal distributions alone determine the distribution of their sum. In general, they do not: the joint distribution matters. Additivity is a useful exception in the sense that it determines the expectation of the sum from the two marginal expectations, even when the full distribution of the sum still depends on how $X$ and $Y$ are related.

Key takeaway: For finite-support variables, expectation is a finite weighted sum. Scaling multiplies every value by the scale factor, while additivity follows by regrouping the joint sum into its marginals. Dependence changes the joint distribution, but does not prevent either identity.

Check Your Understanding

Use finite sums and the joint probability mass function where needed.

If $X$ takes the values $-2$ and $3$ with probabilities $0.4$ and $0.6$, calculate $E[-2X]$ directly and compare it with $-2E[X]$.
In the additivity proof, which inner sums become the marginal probability masses?
Can $E[X+Y]=E[X]+E[Y]$ be used when $X$ and $Y$ are dependent but both have finite support? Explain briefly.
For the joint outcomes $(0,1)$ and $(2,0)$, each with probability $0.5$, calculate $E[X+Y]$ and $E[X]+E[Y]$.
Why is it not generally valid to find the full distribution of $X+Y$ from the two marginal distributions alone?