Bilinearity of Covariance

Why Covariance Has Linear Rules

Covariance measures how two random variables vary together around their means. Its definition uses centered variables, $X-E[X]$ and $Y-E[Y]$, which can make algebraic manipulations look less direct than they are. Once the necessary moments exist, covariance is bilinear: it is linear in either argument when the other is held fixed.

This property is useful whenever a variable is shifted, rescaled, or written as a sum. A constant shift in one argument contributes no covariance, while a scalar multiplier can be pulled outside. Additivity lets us split a covariance involving a sum into simpler pieces. These are algebraic identities; by themselves, they do not say that variables are independent or that any covariance is zero.

Definition and Conditions

Definition: Let $X$ and $Y$ be real-valued random variables on the same probability space with finite second moments, so that $E[X^2]<\infty$ and $E[Y^2]<\infty$. Their covariance is $$ \operatorname{Cov}(X,Y)=E\big[(X-E[X])(Y-E[Y])\big]. $$ The finite-second-moment conditions ensure that the expectation in this definition exists and is finite.

The conditions matter. A covariance uses a product of centered variables, so its expectation must be defined. Finite second moments are a convenient standard sufficient condition for that product to be integrable. Indeed, for real numbers $u,v$, $2|uv|\leq u^2+v^2$. Applying this inequality to $U=X-E[X]$ and $V=Y-E[Y]$ gives

E[|UV|]\leq \frac{E[U^2]+E[V^2]}{2}<\infty.

Here $E[X]$ and $E[Y]$ are finite because finite second moments imply finite first absolute moments; for example, $|x|\leq 1+x^2$. Also, $E[(X-E[X])^2]$ is finite: expanding the square gives $E[X^2]-(E[X])^2$. Thus the centered product is integrable and the covariance is well-defined.

A Useful Product-Moment Formula

The centered definition has a frequently useful equivalent form. Under the finite-second-moment conditions above, $XY$ is integrable: the same inequality gives $E[|XY|]\leq (E[X^2]+E[Y^2])/2<\infty$. We can therefore expand the product and use linearity of expectation without undefined terms.

Theorem: If $X$ and $Y$ have finite second moments, then $$ \operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]. $$

Proof. Since all terms are integrable, expand the product inside the expectation and apply linearity:

\begin{aligned} \operatorname{Cov}(X,Y) &=E\big[(X-E[X])(Y-E[Y])\big]\\ &=E[XY-E[X]Y-E[Y]X+E[X]E[Y]]\\ &=E[XY]-E[X]E[Y]-E[Y]E[X]+E[X]E[Y]\\ &=E[XY]-E[X]E[Y]. \end{aligned}

The quantities $E[X]$ and $E[Y]$ in the expansion are constants, so, for example, $E[E[X]Y]=E[X]E[Y]$. This proves the identity. $\square$

Bilinearity: The Main Result

Theorem: Suppose $X$, $Z$, and $Y$ have finite second moments, and let $a,b$ be real constants. Then $$ \operatorname{Cov}(aX+bZ,Y) =a\operatorname{Cov}(X,Y)+b\operatorname{Cov}(Z,Y). $$ By the symmetry $\operatorname{Cov}(U,V)=\operatorname{Cov}(V,U)$, covariance is linear in its second argument as well: $$ \operatorname{Cov}(X,aY+bW) =a\operatorname{Cov}(X,Y)+b\operatorname{Cov}(X,W) $$ whenever $X,Y,W$ have finite second moments.

The assumptions also ensure that the new variables are eligible arguments of covariance. For instance, $(aX+bZ)^2\leq 2a^2X^2+2b^2Z^2$, whose expectation is finite. The proof now follows directly from centering and expectation linearity.

Proof. First,

$$ (aX+bZ)-E[aX+bZ] =a(X-E[X])+b(Z-E[Z]), $$

because expectation is linear and $a,b$ are constants. Multiply both sides by $Y-E[Y]$, take expectations, and distribute:

\begin{aligned} \operatorname{Cov}(aX+bZ,Y) &=E\big[\{a(X-E[X])+b(Z-E[Z])\}(Y-E[Y])\big]\\ &=aE[(X-E[X])(Y-E[Y])] +bE[(Z-E[Z])(Y-E[Y])]\\ &=a\operatorname{Cov}(X,Y)+b\operatorname{Cov}(Z,Y). \end{aligned}

The expectations are legitimate because each centered product is integrable under the stated moment conditions. Covariance is symmetric for real-valued variables: the products $(U-E[U])(V-E[V])$ and $(V-E[V])(U-E[U])$ are equal pointwise. Applying symmetry to the proved identity gives linearity in the second argument. $\square$

Immediate Consequences

Setting one coefficient to zero or one gives the basic rules for scalar multiples and sums. Setting an argument to a constant gives zero covariance, because a constant has no centered variation.

Consequences: For finite-second-moment variables $X,Y,Z$, real constants $a,b,c$, and a constant random variable $c$, $$ \begin{aligned} \operatorname{Cov}(aX,Y)&=a\operatorname{Cov}(X,Y),\\ \operatorname{Cov}(X,aY)&=a\operatorname{Cov}(X,Y),\\ \operatorname{Cov}(X+Z,Y)&=\operatorname{Cov}(X,Y)+\operatorname{Cov}(Z,Y),\\ \operatorname{Cov}(X,Y+Z)&=\operatorname{Cov}(X,Y)+\operatorname{Cov}(X,Z),\\ \operatorname{Cov}(X,c)&=\operatorname{Cov}(c,X)=0. \end{aligned} $$

A related identity combines scaling and shifting: $\operatorname{Cov}(aX+b,cY+d)=ac\operatorname{Cov}(X,Y)$ for constants $a,b,c,d$. The shifts $b,d$ disappear because the centered versions are $a(X-E[X])$ and $c(Y-E[Y])$. These rules apply to one covariance at a time; using them does not require any assumption of independence.

Worked Example: Scaling and Shifting a Joint Distribution

Suppose $(X,Y)$ has the following distribution: $(X,Y)=(0,1)$ with probability $0.2$, $(1,0)$ with probability $0.3$, and $(2,2)$ with probability $0.5$. Find $\operatorname{Cov}(X,Y)$, then find $\operatorname{Cov}(3X-2,-2Y+5)$.

First calculate the means and product moment:

\begin{aligned} E[X]&=0(0.2)+1(0.3)+2(0.5)=1.3,\\ E[Y]&=1(0.2)+0(0.3)+2(0.5)=1.2,\\ E[XY]&=(0)(1)(0.2)+(1)(0)(0.3)+(2)(2)(0.5)=2. \end{aligned}

The product-moment formula gives

\operatorname{Cov}(X,Y)=2-(1.3)(1.2)=2-1.56=0.44.

The transformed variables have scale factors $3$ and $-2$; their additive constants do not affect covariance. Therefore

\operatorname{Cov}(3X-2,-2Y+5) =3(-2)\operatorname{Cov}(X,Y) =-6(0.44)=-2.64.

The negative sign is consistent with reversing the direction of one variable through multiplication by $-2$.

Worked Example: Covariance With a Sum

A random experiment has three equally likely outcomes. The values of $(X,Z,Y)$ in those outcomes are $(0,1,2)$, $(1,3,0)$, and $(2,-1,1)$. Calculate $\operatorname{Cov}(X+Z,Y)$ by splitting it into two covariances, and check the answer directly.

Each outcome has probability $1/3$. The means are

E[X]=\frac{0+1+2}{3}=1,\qquad E[Z]=\frac{1+3-1}{3}=1,\qquad E[Y]=\frac{2+0+1}{3}=1.

For the first component, $XY$ takes the values $0,0,2$, so $E[XY]=2/3$ and $\operatorname{Cov}(X,Y)=2/3-(1)(1)=-1/3$. For the second, $ZY$ takes the values $2,0,-1$, so $E[ZY]=1/3$ and $\operatorname{Cov}(Z,Y)=1/3-(1)(1)=-2/3$. By additivity,

\operatorname{Cov}(X+Z,Y)=-\frac13-\frac23=-1.

For a direct check, $X+Z$ takes values $1,4,1$, so $E[X+Z]=2$. The product $(X+Z)Y$ takes values $2,0,1$, so its expectation is $1$. Hence $\operatorname{Cov}(X+Z,Y)=1-(2)(1)=-1$, agreeing with the split calculation.

Worked Example: A Constant Argument

Let $X$ take the values $-1,0,2$ with probabilities $1/4,1/4,1/2$, respectively. Calculate its covariance with the constant random variable $C=7$.

The mean of $X$ is

E[X]=(-1)\frac14+0\frac14+2\frac12=\frac34.

Since $E[C]=7$, the centered constant is $C-E[C]=7-7=0$ in every outcome. Thus

\operatorname{Cov}(X,C) =E[(X-\tfrac34)(7-7)] =E[0]=0.

As a check using the product-moment formula, $E[XC]=7E[X]=21/4$ and $E[X]E[C]=(3/4)(7)=21/4$, so their difference is also zero.

What Bilinearity Does—and Does Not—Say

The sum rule says that a covariance can be decomposed into covariances of its summands. It does not say those pieces are equal, positive, or zero. Their values depend on the joint distribution. Likewise, a zero covariance with a constant is guaranteed by the definition, but zero covariance with a nonconstant variable is not guaranteed merely because a sum or scalar multiple appears.

A useful way to avoid overinterpreting the formulas is to separate algebra from distributional claims. Bilinearity manipulates covariance once the relevant joint moments exist. Additional conclusions about the numerical value of a covariance need additional information about the variables' joint behavior.

Common Mistake

A frequent error is to treat covariance like a product and write $\operatorname{Cov}(X+Z,Y)=\operatorname{Cov}(X,Y)$, as if the $Z$ term could simply be ignored. The correction is to apply additivity:

\operatorname{Cov}(X+Z,Y)=\operatorname{Cov}(X,Y)+\operatorname{Cov}(Z,Y).

The second term disappears only if there is a valid reason for it to be zero, such as $Z$ being constant. Another common error is to claim that $\operatorname{Cov}(X,Y)=0$ whenever $X$ and $Y$ are not known to be related. Bilinearity gives no such claim: covariance is determined by their joint distribution, and absence of information is not an algebraic condition.

Key takeaway: For variables with finite second moments, covariance distributes over sums and scalar factors in either argument. Constants contribute zero. These identities are consequences of centered-variable algebra and linearity of expectation, not assumptions about independence.

Check Your Understanding

Use the bilinearity rules and the stated moment conditions to answer these questions.

If $\operatorname{Cov}(X,Y)=-0.6$, what is $\operatorname{Cov}(4X,Y)$?
Write $\operatorname{Cov}(X+Z,Y+W)$ as a sum of four covariances.
Why is $\operatorname{Cov}(X,c)=0$ for a constant $c$?
State the product-moment formula for covariance and identify the moment conditions used here.
Does bilinearity alone imply that $\operatorname{Cov}(X,Z)=0$? Explain briefly.

What You'll Learn