Why Covariance Starts With the Joint Table
A joint probability table describes the probabilities of pairs of values for two random variables. To calculate their covariance, the table must be used in two different ways: its row and column totals give the marginal distributions, while its individual cells give the probabilities of value pairs. Keeping those jobs separate helps prevent a common error: using a marginal probability where a joint probability is needed, or overlooking a product in one cell.
Covariance summarizes whether values of two variables tend to lie above or below their respective means together. It is calculated from deviations from those means, so its sign depends on how the variables vary as pairs. A positive value indicates an overall tendency for deviations to have the same sign; a negative value indicates an overall tendency for them to have opposite signs. Covariance is not a probability and is not restricted to the interval from zero to one.
A Table-Based Calculation
Find the marginal distribution of \(X\) by summing each row when \(X\) labels the rows. Find the marginal distribution of \(Y\) by summing each column when \(Y\) labels the columns. Then calculate \(E[X]\) and \(E[Y]\) from their marginals. Separately, calculate \(E[XY]\) by multiplying \(x_i y_j\) by the joint probability in each cell and adding the results.
The centered-product definition leads to a convenient calculation for finite tables. Write the expectation as a sum over every possible pair and expand the product inside that sum:
In the last line, the first sum is \(E[XY]\); the next two sums reduce to \(E[X]\) and \(E[Y]\) because the table margins are the marginal distributions; and the final sum is 1 because all joint probabilities sum to 1. Thus the table calculation has three main parts: find both marginal means, find \(E[XY]\), and subtract the product of the means.
Confirm that each entry is nonnegative and that all cells sum to 1.
Sum across rows or columns to obtain the probabilities needed for each marginal mean.
For every cell, multiply its row value by its column value and by the cell probability; then add.
Subtract \(E[X]E[Y]\) from \(E[XY]\), and interpret the sign in the context of the paired values.
Worked Example: Negative Covariance From Three Possible Pairs
A hypothetical monitoring device records a pair of integer readings \((X,Y)\). The three possible pairs are \((0,5)\), \((2,1)\), and \((4,3)\), each with probability \(1/3\). Find both means, the expected product, and the covariance.
Method. Treat each pair as one joint outcome. The only nonzero cells in the table have probability \(1/3\); all other pairs have probability zero.
| \(X\) | \(Y=1\) | \(Y=3\) | \(Y=5\) | Row total |
|---|---|---|---|---|
| 0 | 0 | 0 | \(1/3\) (0.3333) | \(1/3\) |
| 2 | \(1/3\) (0.3333) | 0 | 0 | \(1/3\) |
| 4 | 0 | \(1/3\) (0.3333) | 0 | \(1/3\) |
| Column total | \(1/3\) | \(1/3\) | \(1/3\) | 1 |
The row totals give \(P(X=0)=P(X=2)=P(X=4)=1/3\). Therefore,
The column totals give \(P(Y=1)=P(Y=3)=P(Y=5)=1/3\), so
For the expected product, use each joint pair once: \(0\cdot5=0\), \(2\cdot1=2\), and \(4\cdot3=12\). Thus
It follows that
Check. The probabilities total 1, and the three products total \(0+2+12=14\), giving \(E[XY]=14/3\). The negative covariance indicates that, across these possible pairs, larger-than-average values of one reading tend to occur with smaller-than-average values of the other. The method avoids the common mistake of omitting the product \(4\cdot3=12\).
Worked Example: Positive Covariance in a Two-by-Two Table
A hypothetical service platform records whether a request is in the higher-volume category, represented by \(X=1\) rather than \(X=0\), and whether its processing-time index is high, represented by \(Y=2\) rather than \(Y=0\). The joint probabilities are shown below. Find the covariance and interpret its sign.
| \(X\) | \(Y=0\) | \(Y=2\) | Row total |
|---|---|---|---|
| 0 | 0.4000 | 0.1000 | 0.5000 |
| 1 | 0.1000 | 0.4000 | 0.5000 |
| Column total | 0.5000 | 0.5000 | 1.0000 |
Method. Use the margins for the means and use all four joint cells for \(E[XY]\). First, \(P(X=0)=P(X=1)=0.5000\), so
Also, \(P(Y=0)=P(Y=2)=0.5000\), giving
For \(E[XY]\), the products in the cells are \(0\cdot0=0\), \(0\cdot2=0\), \(1\cdot0=0\), and \(1\cdot2=2\). Therefore,
The covariance is
Check and interpretation. The table sums to \(0.4000+0.1000+0.1000+0.4000=1.0000\). The positive covariance indicates that the higher-volume category tends to pair with the high processing-time index in this model. It describes an overall association in the listed pairs; by itself, it does not establish a cause-and-effect relationship.
Worked Example: A Negative Result Despite Positive Values
In a hypothetical quality study, \(X\) is a product grade coded as 1 or 3, and \(Y\) is a rework score coded as 2 or 5. The joint table is:
| \(X\) | \(Y=2\) | \(Y=5\) | Row total |
|---|---|---|---|
| 1 | 0.1000 | 0.4000 | 0.5000 |
| 3 | 0.4000 | 0.1000 | 0.5000 |
| Column total | 0.5000 | 0.5000 | 1.0000 |
Find \(\operatorname{Cov}(X,Y)\). The positive values of \(X\) and \(Y\) do not, by themselves, imply positive covariance.
Marginal means. Each value of \(X\) has probability \(0.5000\), so
Each value of \(Y\) also has probability \(0.5000\), so
Expected product. Multiply the two values and the cell probability in each cell:
Covariance.
Check and interpretation. The probabilities sum to 1.0000, and the lower grade value occurs mostly with the higher rework score, while the higher grade value occurs mostly with the lower score. That pattern agrees with the negative sign. The method avoids judging covariance from whether the variable values themselves are positive or negative; covariance concerns deviations from their means.
Interpreting and Checking the Result
Covariance is expressed in the product of the units of \(X\) and \(Y\). If \(X\) is measured in dollars and \(Y\) in days, the covariance has units of dollar-days. Its numerical magnitude depends on the scales used for both variables, so a value such as \(0.3\) is not meaningful in isolation. Interpret the sign and magnitude with the variable definitions and units in view.
For finite-support variables, the sum in the definition is well-defined as long as the table is a valid probability distribution. The table formula also makes a useful arithmetic check possible: calculate the covariance both as the sum of centered products and as \(E[XY]-E[X]E[Y]\) if there is concern about an error. In either calculation, the pair probabilities—not just the row or column totals—are essential.
Common Exam Traps
- Using a marginal probability in the product sum. The contribution to \(E[XY]\) from a cell is \(x_i y_j p_{ij}\), where \(p_{ij}\) is the joint probability of that pair.
- Leaving out a cell product. A cell with a zero value for either variable contributes zero, but every nonzero cell must be included.
- Multiplying the wrong quantities. Covariance is \(E[XY]-E[X]E[Y]\), not \(E[X]E[Y]-E[XY]\). Keep the order consistent with the centered-product definition.
- Reading sign from the values alone. Covariance concerns how values vary around their means. Two variables can both have only positive values and still have negative covariance.
- Forgetting to check the table. If the entries do not sum to 1, they cannot be used as the stated joint probability distribution without first resolving the problem.
Check Your Understanding
For each problem, use the joint table or listed pairs to find the requested covariance. Keep joint probabilities attached to their corresponding value pairs.
- A joint table has rows \(X=0,2\) and columns \(Y=1,4\). Its rows are \((0.2000,0.3000)\) and \((0.1000,0.4000)\), respectively. Find \(E[X]\), \(E[Y]\), and \(\operatorname{Cov}(X,Y)\). (Answer: \(E[X]=1,\ E[Y]=3.1,\ \operatorname{Cov}(X,Y)=0.3\))
- Three equally likely recorded pairs are \((1,4)\), \((2,2)\), and \((5,1)\). Find both marginal means, \(E[XY]\), and the covariance. (Answer: \(E[X]=8/3,\ E[Y]=7/3,\ E[XY]=13/3,\ \operatorname{Cov}(X,Y)=-17/9\))
- A table has rows \(X=0,1\) and columns \(Y=0,1\); its rows are \((0.1000,0.2000)\) and \((0.3000,0.4000)\). Find the covariance. (Answer: \(\operatorname{Cov}(X,Y)=-0.0200\))
- A device produces \((X,Y)=(0,2)\) with probability 0.2500, \((1,0)\) with probability 0.2500, and \((2,1)\) with probability 0.5000. Find \(E[X]\), \(E[Y]\), \(E[XY]\), and the covariance. (Answer: \(E[X]=1.25,\ E[Y]=1,\ E[XY]=1,\ \operatorname{Cov}(X,Y)=-0.25\))
- A joint table has rows \(X=1,2\) and columns \(Y=0,3\); its rows are \((0.2000,0.3000)\) and \((0.3000,0.2000)\). Find the two marginal means, the expected product, and the covariance. (Answer: \(E[X]=1.5,\ E[Y]=1.5,\ E[XY]=2.1,\ \operatorname{Cov}(X,Y)=-0.15\))