Independence of the Sum and Difference

Why Compare a Sum With a Difference?

For a pair of measurements, the sum \(X+Y\) records their combined level, while the difference \(X-Y\) records their contrast. These two summaries can be dependent even when \(X\) and \(Y\) themselves are independent. In the bivariate normal model, however, there is a simple exact test: calculate the covariance of the sum and difference.

The test depends on the variances of the original variables, not on whether their covariance is zero. This distinction is important in exam problems: a calculation about \(X\) and \(Y\) cannot be substituted for a calculation about \(X+Y\) and \(X-Y\).

Definition: Let \((X,Y)\) be a jointly bivariate normal pair, with means \(\mu_X,\mu_Y\), variances \(\sigma_X^2,\sigma_Y^2\), and covariance \(\operatorname{Cov}(X,Y)\). Define the sum and difference by \(S=X+Y\) and \(D=X-Y\). Then \(S\) and \(D\) are independent if and only if \(\sigma_X^2=\sigma_Y^2\). This criterion applies whether or not \(X\) and \(Y\) are independent.

Calculate the Covariance First

Because \(S\) and \(D\) are linear combinations of a jointly normal pair, they are jointly normal as well. Their covariance follows directly from the rules for covariance: covariance is linear in each argument, and \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(Y,X)\).

\begin{aligned} \operatorname{Cov}(S,D) &=\operatorname{Cov}(X+Y,X-Y)\\ &=\operatorname{Var}(X)-\operatorname{Cov}(X,Y) +\operatorname{Cov}(Y,X)-\operatorname{Var}(Y)\\ &=\sigma_X^2-\sigma_Y^2. \end{aligned}

The two cross-covariance terms cancel. The covariance between \(X\) and \(Y\) still affects the separate variances of \(S\) and \(D\), but it does not affect their covariance. The corresponding means and variances are

\begin{aligned} E[S]&=\mu_X+\mu_Y, & \operatorname{Var}(S)&=\sigma_X^2+\sigma_Y^2+2\operatorname{Cov}(X,Y),\\ E[D]&=\mu_X-\mu_Y, & \operatorname{Var}(D)&=\sigma_X^2+\sigma_Y^2-2\operatorname{Cov}(X,Y). \end{aligned}

For any two random variables, independence implies zero covariance when the relevant expectations exist. The converse is not generally true. The extra fact that makes the converse valid here is joint normality: for a jointly normal pair, zero covariance implies independence. Since \((S,D)\) is jointly normal, \(\operatorname{Cov}(S,D)=0\) is equivalent to independence of \(S\) and \(D\). Combining this with the covariance formula gives the criterion in the definition.

Why Zero Covariance Works Here

The zero-correlation criterion is a property of jointly normal variables, not a general rule for all random variables. One way to see why it holds is to use the joint characteristic function. For a jointly normal pair \((S,D)\), its characteristic function at real numbers \(t,u\) is

E\!\left[e^{\mathrm{i}(tS+uD)}\right] = \exp\!\left( \mathrm{i}(t\mu_S+u\mu_D) -\frac12\left[t^2\operatorname{Var}(S)+u^2\operatorname{Var}(D) +2tu\operatorname{Cov}(S,D)\right] \right),

where \(\mathrm{i}\) denotes the imaginary unit. If \(\operatorname{Cov}(S,D)=0\), the exponent separates into a term involving only \(t\) and a term involving only \(u\). The characteristic function therefore factors into the product of the marginal characteristic functions of \(S\) and \(D\). That factorization is the independence criterion. This argument also covers degenerate jointly normal pairs, where one of the variances may be zero.

Worked Examples

Worked Example: Equal Variances With Dependent Components

Suppose \((X,Y)\) is jointly bivariate normal with \(E[X]=10\), \(E[Y]=4\), \(\operatorname{Var}(X)=9\), \(\operatorname{Var}(Y)=9\), and correlation \(0.4\). Find the distributions of \(S=X+Y\) and \(D=X-Y\), and determine whether they are independent.

Method. First convert the correlation to a covariance, then calculate the means and variances of the two linear combinations. Finally, use their covariance and joint normality to test independence.

The standard deviations of \(X\) and \(Y\) are both \(3\). Thus

\operatorname{Cov}(X,Y)=0.4(3)(3)=3.6.

The means are \(E[S]=10+4=14\) and \(E[D]=10-4=6\). Their variances are

\begin{aligned} \operatorname{Var}(S)&=9+9+2(3.6)=25.2,\\ \operatorname{Var}(D)&=9+9-2(3.6)=10.8. \end{aligned}

The covariance is

\operatorname{Cov}(S,D)=9-9=0.

Since the transformed pair is jointly normal, \(S\) and \(D\) are independent. In the course’s normal notation, \(S\sim N(14,25.2)\) and \(D\sim N(6,10.8)\), where the second parameter is the variance. As a check, the original correlation \(0.4\) affects both transformed variances, but it cancels from their covariance.

Worked Example: Unequal Variances Despite a Negative Correlation

Let \((X,Y)\) be jointly bivariate normal with \(E[X]=2\), \(E[Y]=-1\), \(\operatorname{Var}(X)=16\), \(\operatorname{Var}(Y)=9\), and correlation \(-0.5\). Find the means, variances, and covariance of \(S=X+Y\) and \(D=X-Y\). Are they independent?

Method. The covariance of the original pair is needed for the two transformed variances. The covariance of the transformed pair can then be found either from its definition or from the difference of the original variances.

The standard deviations are \(4\) and \(3\), so

\operatorname{Cov}(X,Y)=(-0.5)(4)(3)=-6.

The transformed means are \(E[S]=2+(-1)=1\) and \(E[D]=2-(-1)=3\). Their variances are

\begin{aligned} \operatorname{Var}(S)&=16+9+2(-6)=13,\\ \operatorname{Var}(D)&=16+9-2(-6)=37. \end{aligned}

Their covariance is \(\operatorname{Cov}(S,D)=16-9=7\), which is not zero. Therefore, \(S\) and \(D\) are not independent. Their correlation is \(7/\sqrt{13\cdot37}=7/\sqrt{481}\), approximately \(0.3192\) to four decimal places. Thus \(S\sim N(1,13)\) and \(D\sim N(3,37)\), but their joint distribution does not factor into independent marginals.

Check. The variances are positive, and the original covariance is compatible with the stated variances: \( (-6)^2=36<16\cdot9=144\). The bivariate normal parameters are valid.

Worked Example: Find an Unknown Variance From Independence

Suppose \((X,Y)\) is jointly bivariate normal with \(E[X]=3\), \(E[Y]=-2\), \(\operatorname{Var}(X)=25\), and \(\operatorname{Cov}(X,Y)=6\). The sum \(S=X+Y\) and difference \(D=X-Y\) are independent. Find \(\operatorname{Var}(Y)\), the correlation of \(X\) and \(Y\), and the distributions of \(S\) and \(D\).

Method. Independence of the jointly normal transformed pair requires zero covariance. Set the difference of the original variances equal to zero, then use the resulting variance to calculate the remaining parameters.

Since \(\operatorname{Cov}(S,D)=\operatorname{Var}(X)-\operatorname{Var}(Y)\), independence requires

25-\operatorname{Var}(Y)=0, \qquad \operatorname{Var}(Y)=25.

Both standard deviations are \(5\), so the original correlation is

\operatorname{Corr}(X,Y)=\frac{6}{(5)(5)}=0.24.

The means of \(S\) and \(D\) are \(3+(-2)=1\) and \(3-(-2)=5\). Their variances are

\begin{aligned} \operatorname{Var}(S)&=25+25+2(6)=62,\\ \operatorname{Var}(D)&=25+25-2(6)=38. \end{aligned}

Thus \(S\sim N(1,62)\) and \(D\sim N(5,38)\), and they are independent. As a parameter check, the correlation \(0.24\) lies between \(-1\) and \(1\); equivalently, \(6^2=36\leq25\cdot25=625\).

Common Mistake: Using the Wrong Covariance

A frequent error is to conclude that \(S\) and \(D\) are independent because \(X\) and \(Y\) have zero covariance. That conclusion does not follow. Consider independent normal variables with \(\operatorname{Var}(X)=4\) and \(\operatorname{Var}(Y)=1\). The incorrect calculation is: \(\operatorname{Cov}(X,Y)=0\), therefore the sum and difference are independent.

The correction is to calculate the covariance of the quantities being tested:

\operatorname{Cov}(X+Y,X-Y) =\operatorname{Var}(X)-\operatorname{Var}(Y) =4-1=3.

The transformed pair is jointly normal, but its covariance is not zero, so the sum and difference are not independent. The original variables being independent is not enough; their variances must also be equal for the sum and difference to be independent.

Degenerate Cases and a Reliable Test

The criterion does not require the original pair to have a nonsingular covariance matrix. For example, if \(\operatorname{Var}(X)=\operatorname{Var}(Y)\) and \(\operatorname{Cov}(X,Y)=\operatorname{Var}(X)\), then \(\operatorname{Var}(D)=0\). The difference is constant with probability one. A constant random variable is independent of every random variable, consistent with the zero-covariance criterion. The same reasoning applies if the sum is constant.

For a nondegenerate transformed pair, the correlation can also be calculated as

\operatorname{Corr}(S,D) = \frac{\sigma_X^2-\sigma_Y^2} {\sqrt{\bigl(\sigma_X^2+\sigma_Y^2+2\operatorname{Cov}(X,Y)\bigr) \bigl(\sigma_X^2+\sigma_Y^2-2\operatorname{Cov}(X,Y)\bigr)}}.

This expression is defined only when both transformed variances are positive. When either variance is zero, use covariance and the degenerate-case argument rather than trying to calculate a correlation with a zero denominator.

A reliable exam method is to name the transformed variables, calculate \(\operatorname{Cov}(S,D)\), and then invoke the jointly normal zero-correlation criterion. The means do not determine independence, and the original correlation alone does not determine it. The deciding quantity is the difference \(\operatorname{Var}(X)-\operatorname{Var}(Y)\). This treatment is aligned with the SOA Exam P (Probability) syllabus.

Check Your Understanding

For each problem, identify the relevant transformed covariance before deciding whether the sum and difference are independent.

Let \((X,Y)\) be jointly normal with means \(2\) and \(5\), variances \(4\) and \(4\), and covariance \(-1\). Find the distributions of \(X+Y\) and \(X-Y\), and decide whether they are independent. (Answer: \(X+Y\sim N(7,6)\), \(X-Y\sim N(-3,10)\), and they are independent.)
Let \((X,Y)\) be jointly normal with means \(1\) and \(0\), variances \(9\) and \(4\), and covariance \(3\). Find \(\operatorname{Cov}(X+Y,X-Y)\) and decide whether the transformed variables are independent. (Answer: the covariance is \(5\), so they are not independent.)
For a jointly normal pair, \(\operatorname{Var}(X)=16\), \(\operatorname{Cov}(X,Y)=-4\), and \(X+Y\) is independent of \(X-Y\). Find \(\operatorname{Var}(Y)\) and \(\operatorname{Corr}(X,Y)\). (Answer: \(\operatorname{Var}(Y)=16\) and \(\operatorname{Corr}(X,Y)=-0.25\).)
Suppose \(X\) and \(Y\) are independent normal variables, each with mean \(0\) and variance \(4\). Find the distributions of their sum and difference and state their dependence status. (Answer: \(X+Y\sim N(0,8)\), \(X-Y\sim N(0,8)\), and they are independent.)
Let \((X,Y)\) be jointly normal with means \(2\) and \(-1\), variances \(9\) and \(9\), and covariance \(9\). Determine whether \(X+Y\) and \(X-Y\) are independent, including any degeneracy. (Answer: they are independent; \(X-Y=3\) with probability one, so the difference is constant.)

What You'll Learn