Correlation as Standardized Covariance

Why Standardize Covariance?

Covariance describes how two random variables vary together, but its numerical value depends on the units used for each variable. Changing dollars to thousands of dollars, for example, changes the numerical covariance. Pearson correlation removes those units by dividing covariance by the product of the two standard deviations. The result is a standardized measure that can be compared across models with different measurement scales.

Definition: Let $X$ and $Y$ be random variables with finite, positive variances. Their Pearson correlation coefficient is $$ \rho_{X,Y} = \frac{\operatorname{Cov}(X,Y)} {\sqrt{\operatorname{Var}(X)}\sqrt{\operatorname{Var}(Y)}}. $$ The denominator is the product of the standard deviations, not the product of the variances. Equivalently, $\rho_{X,Y}$ is the covariance of the standardized variables $(X-E[X])/\sqrt{\operatorname{Var}(X)}$ and $(Y-E[Y])/\sqrt{\operatorname{Var}(Y)}$.

The positive-variance condition matters. If either variable has variance zero, that variable is constant with probability one, its standard deviation is zero, and the formula divides by zero. In that case, Pearson correlation is undefined. Finite variances ensure that the covariance and the standard deviations in the definition are finite.

Correlation has no units: the units in the covariance cancel with those in the standard deviations. Its value also cannot exceed 1 or be less than -1. This bound is a useful model check, not an optional interpretation.

Calculating Correlation From a Discrete Joint Model

For a discrete joint distribution, first find the marginal probabilities and the expectations needed for the variances and covariance. A reliable calculation uses $\operatorname{Var}(X)=E[X^2]-(E[X])^2$ and $\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]$. Then divide the covariance by the product of the standard deviations.

Worked Example: Correlation From a Joint Probability Table

A hypothetical model records two binary indicators, $X$ and $Y$, for whether each of two specified events occurs. The joint probabilities are $P(X=0,Y=0)=0.25$, $P(X=0,Y=1)=0.15$, $P(X=1,Y=0)=0.15$, and $P(X=1,Y=1)=0.45$. Calculate the Pearson correlation.

Method. Calculate the means, second moments, and cross-moment from the joint probabilities, then standardize the covariance. This method avoids treating the covariance itself as the correlation.

Step 1: Check the joint probabilities and find the marginals. The four probabilities sum to $0.25+0.15+0.15+0.45=1$. The probability that $X=1$ is $0.15+0.45=0.60$, and the probability that $Y=1$ is also $0.15+0.45=0.60$. Because each variable is an indicator, $E[X]=0.60$, $E[Y]=0.60$, $E[X^2]=0.60$, and $E[Y^2]=0.60$.

Step 2: Calculate the variances.

\operatorname{Var}(X)=0.60-(0.60)^2=0.24, \qquad \operatorname{Var}(Y)=0.60-(0.60)^2=0.24.

Step 3: Calculate the covariance. The product $XY$ is 1 only when both indicators equal 1, so $E[XY]=0.45$. Therefore, $\operatorname{Cov}(X,Y)=0.45-(0.60)(0.60)=0.09$.

Step 4: Standardize. Each standard deviation is $\sqrt{0.24}$, so their product is $\sqrt{0.24}\sqrt{0.24}=0.24$. Thus $\rho_{X,Y}=0.09/0.24=0.3750$.

Check. Both variances are positive, and $0.3750$ lies between -1 and 1. The calculation is consistent with the required correlation bound.

Calculating Correlation From a Continuous Joint Model

For continuous variables, expectations are calculated by integrating against the joint density. A joint density must be nonnegative on its support and integrate to 1. Marginal densities can be found by integrating out the other variable; they help establish each mean and variance before the cross-moment is calculated.

Worked Example: Correlation From a Joint Density

Let $X$ and $Y$ have joint density $f_{X,Y}(x,y)=1+\frac{1}{2}(2x-1)(2y-1)$ for $0<x<1$ and $0<y<1$, and zero elsewhere. Find their Pearson correlation.

Method. Verify the density, obtain the marginal distributions, and calculate the moments by integration. This avoids assuming that a density that looks uniform has independent coordinates.

Step 1: Verify the density. Since each of $2x-1$ and $2y-1$ lies between -1 and 1 on the support, their product is at least -1. Consequently $f_{X,Y}(x,y)\geq 1-\frac12=\frac12$, so the density is nonnegative. Also, the integral of the product term over the unit square is zero, because $\int_0^1(2x-1)\,dx=0$. The integral of the constant term over the unit square is 1. Thus the density integrates to 1.

Step 2: Find the marginals. Integrating over $y$, the product term vanishes because $\int_0^1(2y-1)\,dy=0$. Hence $f_X(x)=1$ on $0<x<1$. By the same calculation, $f_Y(y)=1$ on $0<y<1$. Each marginal is uniform on $(0,1)$, giving $E[X]=E[Y]=1/2$ and $\operatorname{Var}(X)=\operatorname{Var}(Y)=1/12$.

Step 3: Calculate the cross-moment. The integral of $xy$ against the constant part of the density is $\int_0^1x\,dx\int_0^1y\,dy=1/4$. For the additional part, separate the two integrals:

\frac12 \left(\int_0^1x(2x-1)\,dx\right) \left(\int_0^1y(2y-1)\,dy\right) = \frac12\left(\frac16\right)\left(\frac16\right) = \frac1{72}.

Here each integral equals $2/3-1/2=1/6$. Therefore $E[XY]=1/4+1/72=19/72$.

Step 4: Calculate and standardize the covariance. Since $E[X]E[Y]=1/4=18/72$, $\operatorname{Cov}(X,Y)=19/72-18/72=1/72$. The standard-deviation product is $\sqrt{1/12}\sqrt{1/12}=1/12$. It follows that $\rho_{X,Y}=(1/72)/(1/12)=1/6$, or approximately $0.1667$ to four decimal places.

Check. The result is between -1 and 1, and both marginal variances are positive. The nonzero correlation was calculated from the joint density rather than inferred from the marginal distributions.

Why the Correlation Bound Holds

The bound $-1\leq \rho_{X,Y}\leq 1$ follows directly from standardizing the variables and applying the Cauchy–Schwarz inequality. This derivation also explains why dividing by both standard deviations is essential.

Standardize both variables.
Set $U=(X-E[X])/\sqrt{\operatorname{Var}(X)}$ and $V=(Y-E[Y])/\sqrt{\operatorname{Var}(Y)}$. Positive, finite variances make these definitions valid, and $E[U]=E[V]=0$ with $E[U^2]=E[V^2]=1$.

Identify their expected product.
Using the definition of covariance, $E[UV]=\operatorname{Cov}(X,Y)/(\sqrt{\operatorname{Var}(X)}\sqrt{\operatorname{Var}(Y)})=\rho_{X,Y}$.

Apply Cauchy–Schwarz.
The inequality $|E[UV]|\leq\sqrt{E[U^2]E[V^2]}$ gives $|\rho_{X,Y}|\leq\sqrt{1\cdot1}=1$. Therefore $-1\leq\rho_{X,Y}\leq1$.

This is also a practical consistency check. If reported moments give a quotient outside that interval, the quotient is not a valid correlation: at least one reported moment is incompatible with a joint distribution having those moments. Do not report an out-of-range quotient as though it were the correlation.

Using the Formula With Reported Moments

Sometimes a problem supplies moments rather than a complete joint distribution. In that case, use the same definition: calculate the covariance, calculate each standard deviation, and form the quotient. If the covariance is supplied directly, there is no need to reconstruct it from $E[XY]$. Still, check that both variances are finite and positive and that the quotient satisfies the bound.

Worked Example: Correlation From Reported Moments

A hypothetical joint model for two measurements $X$ and $Y$ reports $E[X]=12$, $E[Y]=5$, $\operatorname{Var}(X)=16$, $\operatorname{Var}(Y)=81$, and $E[XY]=48$. Calculate the correlation and check whether the moments pass the correlation-bound check.

Method. Recover covariance from the shortcut formula and divide by the product of standard deviations. This avoids dividing by the product of the variances.

Step 1: Find the covariance. $\operatorname{Cov}(X,Y)=E[XY]-E[X]E[Y]=48-(12)(5)=48-60=-12$.

Step 2: Find the standard deviations. They are $\sqrt{16}=4$ and $\sqrt{81}=9$, so their product is $4\cdot9=36$.

Step 3: Calculate the correlation. $\rho_{X,Y}=-12/36=-1/3$, or approximately $-0.3333$ to four decimal places.

Check. Both variances are positive, and $-1/3$ is within the required interval. The supplied moments pass this necessary consistency check.

Common Mistake: Dividing by the Variances

The denominator in the correlation formula is $\sqrt{\operatorname{Var}(X)}\sqrt{\operatorname{Var}(Y)}$, not $\operatorname{Var}(X)\operatorname{Var}(Y)$. For example, if the covariance is 7 and the variances are 4 and 9, the incorrect calculation divides by $4\cdot9=36$, producing $7/36$. That is not the correlation formula.

The standard deviations are $\sqrt4=2$ and $\sqrt9=3$, so the formal quotient is $7/(2\cdot3)=7/6$. Since $7/6>1$, these moments cannot describe a valid joint model. The formal quotient is $7/6$, which violates the correlation bound, so no valid joint model has all the reported moments and no correlation can be assigned to them. The correction is not to label $7/6$ as a correlation; it is to identify the inconsistency.

Key takeaway: Pearson correlation is covariance divided by the product of the standard deviations. Calculate that denominator carefully, require finite positive variances, and use the bound $-1\leq\rho_{X,Y}\leq1$ to detect impossible moment combinations.

Check Your Understanding

For each problem, calculate Pearson correlation when the moments are valid, and check the bound before reporting a result.

A joint model has $\operatorname{Var}(X)=16$, $\operatorname{Var}(Y)=25$, and $\operatorname{Cov}(X,Y)=-10$. Find the correlation. (Answer: $-0.5000$)
A joint model has $E[X]=3$, $E[Y]=2$, $E[X^2]=13$, $E[Y^2]=5$, and $E[XY]=4$. Calculate the variances, covariance, and correlation. (Answer: $\operatorname{Var}(X)=4$, $\operatorname{Var}(Y)=1$, $\operatorname{Cov}(X,Y)=-2$, $\rho_{X,Y}=-1$)
Binary variables have $P(X=1)=0.4$, $P(Y=1)=0.5$, and $P(X=1,Y=1)=0.3$. Find their correlation. (Answer: $0.4082$ to four decimal places)
A pair of variables has finite variances $4$ and $16$, and reported covariance $18$. What does the standardized quotient imply? (Answer: The formal quotient is $18/(\sqrt4\sqrt{16})=18/8=2.25$, which exceeds 1; no valid joint model has all the reported moments, so no correlation can be assigned.)
A variable $X$ has variance $9$, and $Y$ has variance $25$. If their covariance is $-12$, calculate the correlation and state whether the moments pass the bound check. (Answer: $-12/(3\cdot5)=-0.8000$; the moments pass the bound check.)

What You'll Learn