Worked Problems with Linear Combinations and Dependence

Why Linear Combinations Require Dependence Checks

In a bivariate normal problem, a new quantity is often a weighted sum or difference of the two original variables. Its distribution can be found from means, variances, and covariance. But when a question involves two such quantities at once, their dependence matters: knowing both marginal distributions does not generally determine a joint probability.

The key sequence is to identify the original covariance, carry it into every variance and covariance calculation, and then decide whether the resulting variables are independent. A special boundary case occurs when the original correlation is exactly \(1\) or \(-1\): the pair is confined to a line, so some apparent combinations are deterministic and some events that look possible from marginal distributions cannot occur together.

Definition: Suppose \((X,Y)\) is a jointly normal pair with means \(\mu_X,\mu_Y\), variances \(\sigma_X^2,\sigma_Y^2\), and correlation \(\rho\). Its covariance is \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\). For constants \(a,b,c,d\), define \(L=aX+bY\) and \(M=cX+dY\). These are linear combinations of the pair.

Formulas for Two Linear Combinations

Linearity of expectation and expansion of the variance give the distribution parameters needed for a linear combination. No independence assumption between \(X\) and \(Y\) is required. In particular, the covariance term must be included unless it is known to be zero.

\begin{aligned} E[L]&=a\mu_X+b\mu_Y,\\ \operatorname{Var}(L)&=a^2\sigma_X^2+b^2\sigma_Y^2+2ab\operatorname{Cov}(X,Y),\\ \operatorname{Cov}(L,M)&=ac\sigma_X^2+bd\sigma_Y^2+(ad+bc)\operatorname{Cov}(X,Y). \end{aligned}

Because \((X,Y)\) is jointly normal, any linear combinations \(L\) and \(M\) are jointly normal as well. If \(\operatorname{Var}(L)>0\), then standardizing \(L\) gives a standard normal variable: \(Z=(L-E[L])/\sqrt{\operatorname{Var}(L)}\). This lets us use standard normal probabilities for a one-variable event involving \(L\).

Why the Covariance Term Appears

The variance formula follows directly by centering each variable and expanding the square. Write \(X-\mu_X\) and \(Y-\mu_Y\) for the centered variables. Then \(L-E[L]=a(X-\mu_X)+b(Y-\mu_Y)\). Squaring this expression produces a cross-product, whose expectation is the covariance.

\begin{aligned} \operatorname{Var}(L) &=E\!\left[\{a(X-\mu_X)+b(Y-\mu_Y)\}^2\right]\\ &=a^2\operatorname{Var}(X)+b^2\operatorname{Var}(Y) +2abE[(X-\mu_X)(Y-\mu_Y)]\\ &=a^2\sigma_X^2+b^2\sigma_Y^2+2ab\operatorname{Cov}(X,Y). \end{aligned}

The same expansion using \(L-E[L]\) and \(M-E[M]\) gives the covariance formula above. The sign of the cross-term depends on the coefficients: for example, subtracting \(Y\) makes its coefficient negative. A positive original covariance can therefore decrease the variance of a difference.

Theorem: If \(L\) and \(M\) are jointly normal and both have positive variance, then \(L\) and \(M\) are independent exactly when \(\operatorname{Cov}(L,M)=0\). This equivalence relies on joint normality; zero covariance alone does not imply independence for arbitrary random variables. If either variable is constant, it is independent of the other, although its correlation is undefined.

For a nonsingular jointly normal pair, zero covariance means zero correlation. The bivariate normal density then factors into the product of its two marginal normal densities, which is the defining condition for independence. Since linear combinations of a jointly normal pair remain jointly normal, this test applies to \(L\) and \(M\). At perfect correlation the pair is singular, so it has no ordinary two-dimensional density; check the actual linear relationship rather than relying on a nonsingular density formula.

Worked Example: Comparing Two Dependent Combinations

Suppose \((X,Y)\) is jointly normal with \(X\sim N(8,9)\), \(Y\sim N(3,4)\), and correlation \(\rho=0.5\). Let \(A=2X-Y\) and \(B=X+Y\). Find their means, variances, covariance, and correlation. Then calculate \(P(A>13+\sqrt{28})\).

Method and covariance. First convert the correlation to covariance. The standard deviations are \(\sigma_X=3\) and \(\sigma_Y=2\), so \(\operatorname{Cov}(X,Y)=0.5(3)(2)=3\).

The means are \(E[A]=2(8)-3=13\) and \(E[B]=8+3=11\).

For \(A\), use coefficients \(2\) and \(-1\): \[ \operatorname{Var}(A)=2^2(9)+(-1)^2(4)+2(2)(-1)(3) =36+4-12=28. \]

For \(B\), both coefficients are \(1\): \[ \operatorname{Var}(B)=9+4+2(1)(1)(3)=19. \]

Their covariance is \[ \begin{aligned} \operatorname{Cov}(A,B) &=(2)(1)(9)+(-1)(1)(4)+\{(2)(1)+(-1)(1)\}(3)\\ &=18-4+3=17. \end{aligned} \] Thus their correlation is \(17/(\sqrt{28}\sqrt{19})=17/\sqrt{532}\), approximately \(0.7370\). It is not zero, so the jointly normal variables \(A\) and \(B\) are dependent.

Since \(A\) is normal with mean \(13\) and variance \(28\), \[ P(A>13+\sqrt{28}) =P\!\left(\frac{A-13}{\sqrt{28}}>1\right) =1-\Phi(1)=0.1587. \] The final probability is rounded to four decimals. As a check, the threshold is exactly one standard deviation above \(A\)'s mean, so the upper-tail probability should be about \(0.16\).

Common mistake avoided: Omitting the covariance term would give \(36+4=40\) for \(\operatorname{Var}(A)\). The coefficient of \(Y\) is negative, so the correct cross-term is \(-12\), giving variance \(28\), not \(40\).

Zero Covariance Can Make the New Pair Independent

Two original variables can be dependent while particular linear combinations of them are independent. The test is not whether the original correlation is zero: calculate the covariance of the specific combinations. If the combinations are jointly normal and their covariance is zero, they are independent.

Worked Example: Independent Sum and Difference Combinations

Let \((X,Y)\) be jointly normal with \(X\sim N(2,4)\), \(Y\sim N(-1,4)\), and \(\rho=-0.25\). Define \(U=X+Y\) and \(V=X-Y\). Find their distributions and calculate \(P(U>1,\ V<3)\).

Method and covariance check. Both original standard deviations are \(2\), so \(\operatorname{Cov}(X,Y)=(-0.25)(2)(2)=-1\). The means are \(E[U]=2+(-1)=1\) and \(E[V]=2-(-1)=3\).

The variances are \[ \operatorname{Var}(U)=4+4+2(-1)=6 \quad\text{and}\quad \operatorname{Var}(V)=4+4-2(-1)=10. \] For the covariance, \[ \operatorname{Cov}(U,V) =\operatorname{Cov}(X+Y,X-Y) =\operatorname{Var}(X)-\operatorname{Var}(Y)=4-4=0. \] The two combinations are jointly normal, and both have positive variance. They are therefore independent, even though \(X\) and \(Y\) have correlation \(-0.25\).

Now calculate the two marginal probabilities: \[ P(U>1)=0.5 \] because the threshold is \(U\)'s mean, and \[ P(V<3)=0.5 \] because that threshold is \(V\)'s mean. Independence permits multiplication: \[ P(U>1,\ V<3)=P(U>1)P(V<3)=(0.5)(0.5)=0.2500. \] The probability is given to four decimals. Each marginal probability is one-half by symmetry of a continuous normal distribution about its mean, which checks the calculation.

Common mistake avoided: Do not assume the combinations are dependent just because \(X\) and \(Y\) are dependent. Conversely, do not multiply the marginal probabilities until independence of the combinations has been established.

Check the Support at Perfect Correlation

When \(\rho=1\) or \(\rho=-1\), standardized variables have a deterministic relationship: \[ \frac{Y-\mu_Y}{\sigma_Y} =\frac{X-\mu_X}{\sigma_X}\quad(\rho=1), \qquad \frac{Y-\mu_Y}{\sigma_Y} =-\frac{X-\mu_X}{\sigma_X}\quad(\rho=-1). \] Thus, the pair lies on a line rather than ranging freely over the plane. A calculation that treats the variables as if every combination of their marginal values were possible can assign positive probability to an impossible event. Strict versus non-strict inequalities do not change normal tail probabilities at a single point, but event intersections must still be checked against this support.

Worked Example: A Joint Event on a Singular Support

Suppose \((X,Y)\) is jointly normal with \(X\sim N(5,4)\), \(Y\sim N(1,9)\), and \(\rho=-1\). Find the constant value of \(R=3X+2Y\), and calculate \(P(X>7,\ Y<-2)\).

Method and support check. The standard deviations are \(2\) and \(3\). Let \(Z=(X-5)/2\), so \(Z\sim N(0,1)\). Perfect negative correlation gives \[ X=5+2Z,\qquad Y=1-3Z. \] Consequently, \[ R=3(5+2Z)+2(1-3Z)=15+6Z+2-6Z=17. \] The random combination \(R\) is the constant \(17\).

Translate both event conditions using the same \(Z\): \[ X>7\ \Longleftrightarrow\ 5+2Z>7\ \Longleftrightarrow\ Z>1, \] and \[ Y<-2\ \Longleftrightarrow\ 1-3Z<-2\ \Longleftrightarrow\ Z>1. \] The two conditions are the same restriction on the support. Therefore, \[ P(X>7,\ Y<-2)=P(Z>1)=1-\Phi(1)=0.1587. \] The probability is rounded to four decimals. Both thresholds reduce to \(Z>1\), which verifies that the intersection is not the product of two separate tail events.

Common mistake avoided: Multiplying the marginal tail probabilities would treat \(X>7\) and \(Y<-2\) as independent. They are perfectly dependent here, and both conditions hold for exactly the same values of \(Z\).

A Reliable Exam Method

For a problem combining linear combinations and dependence, use a fixed sequence. It helps prevent errors in signs, units of spread, and event boundaries.

Record the original model.
Write down both means, both variances, the correlation, and the covariance. Convert correlation using \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\); do not substitute variances for standard deviations in this conversion.

Translate each combination.
Identify its coefficients and calculate the mean and variance. Keep the signed coefficients in the cross-term.

Test dependence between combinations.
Calculate their covariance. For jointly normal combinations, zero covariance establishes independence; a nonzero value rules it out.

Check for a boundary case.
If the original correlation is \(1\) or \(-1\), express both variables using one standard normal variable. Test event conditions on this common support.

Standardize and calculate.
For a one-variable normal probability, subtract its mean and divide by its standard deviation. Multiply probabilities of events only after independence has been justified.

Common Traps and Final Checks

Using correlation where covariance is needed. The cross-term uses \(\operatorname{Cov}(X,Y)=\rho\sigma_X\sigma_Y\), not \(\rho\) by itself.
Using the variance as the standard deviation. If \(X\sim N(\mu,\sigma^2)\), standardization divides by \(\sigma\), the square root of the second parameter.
Dropping a negative coefficient. In \(X-Y\), the cross-term in the variance is \(-2\operatorname{Cov}(X,Y)\).
Assuming zero covariance always means independence. That conclusion is justified here by joint normality; it is not a general rule for arbitrary random variables.
Ignoring a singular support. At perfect correlation, check whether the event inequalities can hold for the same value of the underlying standard normal variable.

These checks are especially useful in multi-step questions: a correct marginal distribution can still lead to a wrong answer if the dependence between transformed variables is mishandled. The calculations here are aligned with the SOA Exam P (Probability) syllabus.

Check Your Understanding

For each problem, identify the covariance or dependence issue before calculating the requested probability.

Let \(X\sim N(4,9)\) and \(Y\sim N(2,1)\) be jointly normal with correlation \(-1/3\). Find \(P(X+2Y>11)\). (Answer: \(0.1587\))
Let \(X\sim N(1,4)\) and \(Y\sim N(2,4)\) be jointly normal with correlation \(0.5\). Set \(U=X-Y\) and \(V=X+Y\). Find \(P(U\leq -1,\ V>3)\). (Answer: \(0.2500\))
Let \(X\sim N(10,4)\) and \(Y\sim N(5,1)\) be jointly normal with correlation \(-1\). Find \(P(X>12,\ Y<4)\) and the value of \(2X+4Y\). (Answer: \(0.1587\); \(2X+4Y=40\) with probability 1)
Let \(X\sim N(0,1)\) and \(Y\sim N(0,4)\) be jointly normal with correlation \(0.5\). For \(U=2X+Y\) and \(V=X-\tfrac12Y\), determine whether they are independent and find \(P(U>0,\ V>0)\). (Answer: They are independent; \(0.2500\))
Let \(X\sim N(3,1)\) and \(Y\sim N(-2,9)\) be jointly normal with correlation \(1\). Find \(P(X\leq2,\ Y>1)\) and the value of \(3X-Y\). (Answer: \(0\); \(3X-Y=11\) with probability 1)

What You'll Learn