Means and Variances of Linear Combinations

Why Coefficients Affect Spread

Many random quantities combine several sources of variation. A total score might give one component three times the weight of another; a net amount might subtract a cost from a gain. To describe such a quantity, it is useful to know how its component means and variances determine its own mean and variance.

The coefficients play different roles in these calculations. They multiply means with their signs intact, but they multiply variances by their squares. Thus, a negative coefficient reverses a component’s direction when calculating the mean, while still adding a nonnegative amount to the variance when the components are independent.

Definition: Let $X$ and $Y$ be discrete random variables with finite means and variances, and let $a$ and $b$ be fixed constants. The linear combination $W=aX+bY$ has mean $\mathop{\rm E}(W)=a\mathop{\rm E}(X)+b\mathop{\rm E}(Y)$. If $X$ and $Y$ are independent, its variance is $\mathop{\rm Var}(W)=a^2\mathop{\rm Var}(X)+b^2\mathop{\rm Var}(Y)$.

A discrete random variable has finite means and variances, for example, when it can take only finitely many possible values. More generally, the results here require that the expectations and variances involved are defined and finite. Independence means that the joint probabilities factor: for every possible pair of values $x,y$, $P(X=x,Y=y)=P(X=x)P(Y=y)$. The variance formula requires this condition; the mean formula does not.

Why the Variance Formula Works

The mean formula follows from the linearity of expectation: multiplying a random variable by a constant multiplies its mean by that constant, and the expectation of a sum is the sum of the expectations. These rules do not require independence. For variance, start by subtracting the mean of $W$. Write $\mu_X=\mathop{\rm E}(X)$ and $\mu_Y=\mathop{\rm E}(Y)$. Then

W-\mathop{\rm E}(W) =a(X-\mu_X)+b(Y-\mu_Y).

Variance is the expected squared deviation from the mean. Squaring the last expression produces two squared terms and a cross-product term:

\begin{aligned} \mathop{\rm Var}(W) &=a^2\mathop{\rm E}\big[(X-\mu_X)^2\big] +b^2\mathop{\rm E}\big[(Y-\mu_Y)^2\big]\\ &\quad+2ab\mathop{\rm E}\big[(X-\mu_X)(Y-\mu_Y)\big]. \end{aligned}

The first two expectations are the component variances. To see why the last expectation is zero under independence, sum over all possible values of $X$ and $Y$. Independence lets the joint probability in each term split into a product, so the sum factors:

\begin{aligned} \mathop{\rm E}\big[(X-\mu_X)(Y-\mu_Y)\big] &=\sum_x\sum_y (x-\mu_X)(y-\mu_Y)P(X=x)P(Y=y)\\ &=\left(\sum_x(x-\mu_X)P(X=x)\right) \left(\sum_y(y-\mu_Y)P(Y=y)\right)\\ &=0\cdot 0=0. \end{aligned}

Each factor is zero because the expected deviation of a variable from its own mean is zero. With the cross-product gone, the two squared-coefficient contributions remain. This also explains why independence matters: without it, the cross-product expectation need not be zero, and the variance formula for independent variables cannot simply be used.

Calculating Component Means and Variances

For a discrete variable with possible values $x_i$ and probabilities $p_i$, its mean is $\sum_i x_i p_i$, and its variance is $\sum_i(x_i-\mu_X)^2p_i$. Equivalently, the variance is $\mathop{\rm E}(X^2)-[\mathop{\rm E}(X)]^2$. Compute these component summaries first; then use the linear-combination formulas.

Worked Example: A Weighted Difference of Two Variables

In a hypothetical scoring exercise, let $X$ have values 0 and 2, with probabilities 0.75 and 0.25. Let $Y$ have values 1 and 3, each with probability 0.50. The exercise stipulates that the variables are independent. Find the mean and variance of $W=3X-2Y$.

Both variables have finite supports, so their means and variances are finite. The stated independence satisfies the condition needed for adding their variance contributions.

For $X$,

\mathop{\rm E}(X)=0(0.75)+2(0.25)=0.50, \qquad \mathop{\rm E}(X^2)=0^2(0.75)+2^2(0.25)=1.00, $$ $$ \mathop{\rm Var}(X)=1.00-(0.50)^2=0.75.

For $Y$,

\mathop{\rm E}(Y)=1(0.50)+3(0.50)=2.00, \qquad \mathop{\rm E}(Y^2)=1^2(0.50)+3^2(0.50)=5.00, $$ $$ \mathop{\rm Var}(Y)=5.00-(2.00)^2=1.00.

The coefficient $-2$ makes the mean contribution from $Y$ negative, while both coefficients are squared for the variance:

\mathop{\rm E}(W)=3(0.50)-2(2.00)=-2.50, \qquad \mathop{\rm Var}(W)=3^2(0.75)+(-2)^2(1.00) =6.75+4.00=10.75.

Thus, $W$ has mean $-2.50$ score units and variance $10.75$ squared score units. The negative coefficient changes the direction of $Y$'s contribution to the mean, not the sign of its contribution to the variance.

As a check, independence gives the four pairs of outcomes probabilities $0.375, 0.375, 0.125,$ and $0.125$. They give $W=-2,-6,4,$ and $0$, respectively. Their weighted mean is $(-2)(0.375)+(-6)(0.375)+4(0.125)+0(0.125)=-2.50$, consistent with the formula.

Coefficients and the Size of a Variance Contribution

If a variable is multiplied by $c$, its variance is multiplied by $c^2$. For example, doubling a component multiplies its variance contribution by four, not two. This follows from squaring the deviation: $[cX-\mathop{\rm E}(cX)]^2=c^2(X-\mathop{\rm E}(X))^2$. Variance therefore uses squared units. If $X$ is measured in points, then $\mathop{\rm Var}(X)$ is measured in squared points; coefficients also account for any scaling or unit conversion in the linear combination.

Worked Example: A Weighted Total

Suppose two independent components of a hypothetical score are $X$, which takes values 0, 1, and 2 with probabilities 0.20, 0.50, and 0.30, and $Y$, which takes values 1 and 2 with probabilities 0.25 and 0.75. Let $W=4X+Y$. Find its mean and variance.

The distributions have finite supports, and the scenario specifies that $X$ and $Y$ are independent. First calculate the component summaries:

\begin{aligned} \mathop{\rm E}(X)&=0(0.20)+1(0.50)+2(0.30)=1.10,\\ \mathop{\rm E}(X^2)&=0^2(0.20)+1^2(0.50)+2^2(0.30)=1.70,\\ \mathop{\rm Var}(X)&=1.70-(1.10)^2=0.49. \end{aligned} $$ $$ \begin{aligned} \mathop{\rm E}(Y)&=1(0.25)+2(0.75)=1.75,\\ \mathop{\rm E}(Y^2)&=1^2(0.25)+2^2(0.75)=3.25,\\ \mathop{\rm Var}(Y)&=3.25-(1.75)^2=0.1875. \end{aligned}

The mean uses the coefficients as written; the variance uses their squares:

\mathop{\rm E}(W)=4(1.10)+1(1.75)=6.15, $$ $$ \mathop{\rm Var}(W)=4^2(0.49)+1^2(0.1875) =7.84+0.1875=8.0275.

The combined score has mean 6.15 score units and variance 8.0275 squared score units. The $X$ contribution is especially large because its variance, 0.49, is multiplied by $4^2=16$.

Worked Example: A Gain Minus a Fee

Consider a hypothetical payment with two independent components. Let $X=1$ if a project earns a $10 gain and $X=0$ otherwise, with $P(X=1)=0.40$. Let $Y=1$ if it incurs a $3 fee and $Y=0$ otherwise, with $P(Y=1)=0.70$. The project’s net amount is $W=10X-3Y$, in dollars. Find its mean, variance, and standard deviation.

Each indicator has a finite distribution, and the scenario stipulates that the gain event and fee event are independent. For an indicator that is 1 with probability $p$ and 0 with probability $1-p$, its mean is $p$, and its variance is $p-p^2=p(1-p)$. Therefore,

\mathop{\rm E}(X)=0.40,\quad \mathop{\rm Var}(X)=0.40(0.60)=0.24, \qquad \mathop{\rm E}(Y)=0.70,\quad \mathop{\rm Var}(Y)=0.70(0.30)=0.21.

The net amount’s mean and variance are

\mathop{\rm E}(W)=10(0.40)-3(0.70)=4.00-2.10=\$1.90, $$ $$ \mathop{\rm Var}(W)=10^2(0.24)+(-3)^2(0.21) =24.00+1.89=25.89\ {\rm dollars}^2.

The standard deviation is the positive square root of the variance:

\mathop{\rm SD}(W)=\sqrt{25.89}\approx \$5.088.

The distribution also offers a check. The outcomes $W=0,-3,10,7$ have probabilities $0.18,0.42,0.12,0.28$, respectively, by independence. The mean is $0(0.18)-3(0.42)+10(0.12)+7(0.28)=1.90$. Also, $\mathop{\rm E}(W^2)=9(0.42)+100(0.12)+49(0.28)=29.50$, so $\mathop{\rm Var}(W)=29.50-(1.90)^2=25.89$ dollars squared.

What Independence Does—and Does Not—Allow

The mean of a linear combination can be calculated from the component means whether or not the variables are independent. Variance is different: the formula that adds the squared-coefficient contributions is justified here by independence. If dependence is known or possible, that independent-variable formula is not warranted without further information about how the variables vary together.

For a simple illustration, if $Y=X$, then $X$ and $Y$ are not independent unless $X$ is constant. The combination $X-Y$ is always zero, so its variance is zero. Adding the component variances as if $X$ and $Y$ were independent would give $2\mathop{\rm Var}(X)$, which is positive whenever $X$ varies. This shows why checking the relationship between the variables is essential before applying the variance rule.

Common Mistakes

Using $a$ instead of $a^2$ for variance. The variance of $aX$ is $a^2\mathop{\rm Var}(X)$, because deviations are squared. If $a=4$, the variance contribution is multiplied by 16.
Making a negative variance contribution. A negative coefficient can make a mean contribution negative, but its squared coefficient is nonnegative. For independent variables, each variance contribution is nonnegative.
Assuming independence is needed for the mean formula. It is not. Linearity gives the mean of a sum or difference without independence; independence is the condition used for the variance formula here.
Adding variances without checking independence. If the variables are dependent, the cross-product in the variance derivation may not be zero. Do not use the independent formula unless independence is justified.
Giving variance the original units. If $W$ is measured in dollars, its variance is measured in dollars squared. The standard deviation, found by taking the square root, is in dollars.

Key takeaway: For independent discrete variables with finite variances, $\mathop{\rm E}(aX+bY)=a\mathop{\rm E}(X)+b\mathop{\rm E}(Y)$ and $\mathop{\rm Var}(aX+bY)=a^2\mathop{\rm Var}(X)+b^2\mathop{\rm Var}(Y)$. Means preserve coefficient signs; variance contributions use squared coefficients.

Check Your Understanding

For each question, identify the role of the coefficients and state any assumption needed for the variance calculation.

Independent variables $X$ and $Y$ have means 2 and 5, and variances 3 and 4. Find the mean and variance of $2X-Y$.
Why is the contribution from $Y$ to the variance of $3X-2Y$ multiplied by 4 rather than by $-2$?
Does the mean formula for $aX+bY$ require independence? Explain briefly.
If $X$ and $Y$ are dependent, what part of the variance derivation may fail to vanish?
A random total is measured in minutes. What are the units of its variance and of its standard deviation?