Why Standardize Covariance?
Covariance describes how two random variables vary together, but its numerical size depends on the units in which they are measured. A covariance of 12 could be large or small depending on those units, so it is not generally meaningful to compare covariance values across different pairs of variables. Correlation addresses this issue by dividing covariance by the product of the two standard deviations. The result is dimensionless and bounded.
Correlation is defined only when both variables have finite, strictly positive variances. The finite-moment condition ensures that the quantities in the definition exist; the positive-variance condition ensures that the standard deviations used as divisors are not zero.
Standardization centers each variable at zero and measures its values in standard-deviation units. Correlation is the expected product of these standardized variables. This representation gives a direct interpretation: it summarizes whether values above their respective means tend to occur together, or whether one tends to be above its mean when the other is below.
Standardized Variables and the Range of Correlation
The Cauchy–Schwarz bound for covariance gives \(|\operatorname{Cov}(X,Y)|\leq \sqrt{\operatorname{Var}(X)\operatorname{Var}(Y)}\) when the variances are finite. Dividing by the positive denominator in the definition of correlation gives its range. The standardized-variable representation also follows directly from the definition.
Proof. Write \(\mu_X=E[X]\), \(\mu_Y=E[Y]\), \(\sigma_X=\sqrt{\operatorname{Var}(X)}\), and \(\sigma_Y=\sqrt{\operatorname{Var}(Y)}\), where both standard deviations are positive. Then
The same calculations give \(E[Z_Y]=0\) and \(\operatorname{Var}(Z_Y)=1\). Since both standardized variables have mean zero,
Finally, the covariance bound applied to \(Z_X,Z_Y\) gives \(|\operatorname{Cov}(Z_X,Z_Y)|\leq\sqrt{1\cdot1}=1\). Their covariance is their correlation, so \(-1\leq\operatorname{Corr}(X,Y)\leq1\). \(\square\)
The endpoints have a precise meaning: correlation equals \(1\) or \(-1\) exactly when the standardized variables agree or are negatives of one another with probability one. This is stronger than saying that the variables usually move in the same or opposite directions; it describes an exact linear relation after standardization.
Proof. Let \(\rho=\operatorname{Corr}(X,Y)=E[Z_XZ_Y]\). The standardized variables each have second moment one. Therefore,
and
If \(\rho=1\), the first expectation is zero. A nonnegative random variable with expectation zero equals zero with probability one, so \(Z_X-Z_Y=0\) with probability one. Conversely, if \(Z_X=Z_Y\) with probability one, then \(E[Z_XZ_Y]=E[Z_X^2]=1\), so \(\rho=1\). If \(\rho=-1\), the second expectation is zero, which implies \(Z_X+Z_Y=0\) with probability one. Conversely, if \(Z_X=-Z_Y\) with probability one, then \(E[Z_XZ_Y]=-E[Z_X^2]=-1\), so \(\rho=-1\). \(\square\)
Calculating Correlation From a Joint Law
For a discrete joint distribution, the calculation begins with the marginal means and variances and the joint product moment \(E[XY]\). The covariance can then be found as \(E[XY]-E[X]E[Y]\). For a continuous joint density, the same moment identities apply, with sums replaced by integrals. In either case, keep the covariance and the two standard deviations distinct until the final division.
Worked Example: A Discrete Joint Probability Table
Suppose \(X\) and \(Y\) each take values 0 and 1, with joint probabilities shown below. Find their correlation.
| \(X\) | \(Y\) | \(P(X,Y)\) |
|---|---|---|
| 0 | 0 | 0.4 |
| 0 | 1 | 0.1 |
| 1 | 0 | 0.1 |
| 1 | 1 | 0.4 |
The probabilities sum to \(0.4+0.1+0.1+0.4=1\). The marginal probability that \(X=1\) is \(0.1+0.4=0.5\); likewise \(P(Y=1)=0.5\). Thus both variables are Bernoulli with success probability \(0.5\), so
The product \(XY\) is 1 only in the outcome \((1,1)\). Hence \(E[XY]=0.4\), and
The product of the standard deviations is \(\sqrt{0.25}\sqrt{0.25}=0.5(0.5)=0.25\). Therefore,
The positive value indicates that, in this joint law, the two variables tend to be on the same side of their means. It is not an endpoint, so the standardized variables do not agree exactly with probability one.
Worked Example: A Joint Law on Four Outcomes
Let \((X,Y)\) have probabilities \(3/8\) at \((1,1)\), \(3/8\) at \((-1,-1)\), \(1/8\) at \((1,-1)\), and \(1/8\) at \((-1,1)\). Find \(\operatorname{Corr}(X,Y)\).
For each variable, the probabilities of \(1\) and \(-1\) are \(1/2\), so \(E[X]=E[Y]=0\) and \(E[X^2]=E[Y^2]=1\). It follows that both variances are 1. The product \(XY\) equals 1 at the two outcomes with equal signs and \(-1\) at the two outcomes with opposite signs. Consequently,
Since the means are zero, \(\operatorname{Cov}(X,Y)=E[XY]=1/2\). The standard deviations are both 1, so
This calculation also illustrates why \(E[XY]\) alone is not always correlation: it equals correlation here because both means are zero and both standard deviations are one.
Worked Example: A Continuous Joint Density
Suppose \((X,Y)\) has joint density \(f(x,y)=2\) on \(0<y<x<1\), and density zero elsewhere. First check that it is a probability density:
Integrating against the density gives
The second moments are
Therefore,
The joint product moment is
Thus \(\operatorname{Cov}(X,Y)=1/4-(2/3)(1/3)=1/36\), while the product of the standard deviations is \((1/\sqrt{18})(1/\sqrt{18})=1/18\). The correlation is
The support restriction \(Y<X\) is reflected in a positive, but non-perfect, correlation.
Interpretation and Common Pitfalls
Correlation is a standardized measure of linear co-variation. Its sign records the direction of the association: positive correlation means the centered variables tend to have products of the same sign, while negative correlation means products of opposite signs tend to contribute. Its magnitude is bounded by one and reaches an endpoint only for an exact relation between the standardized variables.
A correlation near zero indicates little linear association as measured by the expected product of standardized values. It does not, by itself, establish independence or rule out other forms of dependence. Nor is there a universal cutoff that makes a particular correlation “strong”: interpretation depends on the application, the distributions, and the intended use.
- Using covariance as though it were correlation. Covariance is not generally bounded by one and retains units. Divide by both standard deviations to obtain correlation.
- Forgetting the positive-variance condition. If either variable is constant with probability one, its standard deviation is zero and the correlation is undefined, not zero.
- Using only the product moment. In general, \(E[XY]\) is not covariance; subtract \(E[X]E[Y]\) first. It equals covariance only when at least one mean is zero.
- Interpreting the range as a strength scale with fixed cutoffs. The bounds \([-1,1]\) are universal, but labels such as “weak” or “strong” require context.
Check Your Understanding
Use the definitions and calculations above to answer these questions.
- What conditions on \(X\) and \(Y\) are required for \(\operatorname{Corr}(X,Y)\) to be defined?
- If \(E[X]=0\), \(E[Y]=0\), and both variances equal 1, how does \(E[XY]\) relate to correlation?
- For standardized variables with correlation \(-1\), what equality holds with probability one?
- A joint law gives \(E[XY]=0.7\), \(E[X]=0.2\), \(E[Y]=0.5\), and standard deviations 2 and 3. Calculate the correlation.
- Why does a correlation near zero not, by itself, show that two variables are independent?