Why Standardized Scores Clarify Correlation
Pearson’s correlation summarizes the direction and strength of a linear pattern between two quantitative variables. Its value is easier to interpret when we connect it to the positions of individual observations: is each value above or below its own variable’s mean, and do the two values tend to be on the same side of their means? Standardized scores make those positions comparable, even when the original variables use different units.
For example, suppose a hypothetical set of students has paired measurements of study time and quiz score. A student who studies more than the sample average has a positive standardized study-time score; a student who scores below the quiz average has a negative standardized quiz-score score. Multiplying the two scores gives a negative contribution because the student is above average on one variable and below average on the other.
The key is that the scores must be standardized within the sample using the sample standard deviations. This specific convention leads to an average whose divisor is \(n-1\), not \(n\).
Sample Z-Scores and Their Products
A positive z-score means the observation is above its sample mean; a negative z-score means it is below. A score of zero means it equals the sample mean. The two scores in a pair must come from the same individual or observational unit. Pairing a study-time score from one student with a quiz-score score from another would not describe how the variables vary together.
The product \(z_{x,i}z_{y,i}\) records whether the paired deviations point in the same direction. If both scores are positive, their product is positive. If both are negative, their product is also positive. If one is positive and the other negative, the product is negative. If either score is zero, the product is zero. The size of the product also matters: a pair of larger-magnitude scores can make a larger positive or negative contribution.
Correlation as an Average Product
This formula is a direct consequence of the definitions. Write \(S_{xy}=\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})\), \(S_{xx}=\sum_{i=1}^{n}(x_i-\bar{x})^2\), and \(S_{yy}=\sum_{i=1}^{n}(y_i-\bar{y})^2\). Pearson’s correlation is \(r=S_{xy}/\sqrt{S_{xx}S_{yy}}\). The sample standard deviations satisfy \(s_x=\sqrt{S_{xx}/(n-1)}\) and \(s_y=\sqrt{S_{yy}/(n-1)}\).
Now substitute the sample z-score definitions into the sum of products. Since \(s_xs_y=\sqrt{S_{xx}S_{yy}}/(n-1)\), the sum is
Dividing both sides by \(n-1\) gives the stated result. This derivation requires that both variables vary in the sample, so that \(s_x\) and \(s_y\) are nonzero and their z-scores are defined. It also relies on pairing the \(x\) and \(y\) values correctly.
This is a descriptive identity, not a procedure that requires normally distributed data. It explains the calculation and interpretation of sample correlation for paired quantitative data. Whether correlation is a useful summary in a particular situation still depends on the pattern: a curved relationship, an unusual observation, or distinct clusters can make one number incomplete or misleading.
Worked Example: Perfect Same-Direction Standardized Scores
Suppose four hypothetical students have sample z-scores for study time and quiz score as shown. Let \(a=\sqrt{3}/2\), which is approximately \(0.8660\).
| Student | Study-time z-score | Quiz-score z-score | Product |
|---|---|---|---|
| A | \(-a\) | \(-a\) | \(a^2=0.75\) |
| B | \(-a\) | \(-a\) | \(a^2=0.75\) |
| C | \(a\) | \(a\) | \(a^2=0.75\) |
| D | \(a\) | \(a\) | \(a^2=0.75\) |
These columns are valid sample z-scores. Each column sums to zero, and for each column the squared scores sum to \(4a^2=4(3/4)=3=n-1\). Those checks matter: values should not be labelled sample z-scores unless they are consistent with the sample mean and standard deviation used to standardize them.
The sum of paired products is \(0.75+0.75+0.75+0.75=3\). Therefore,
For each student, the study-time and quiz-score scores are on the same side of their respective sample means. Students below average on study time are also below average on quiz score, and students above average on study time are also above average on quiz score. The perfect alignment produces a correlation of \(1\).
Worked Example: Positive Correlation With Mixed Contributions
Now use four hypothetical pairs with \(a=\sqrt{3}/2\). The study-time scores are \((-a,-a,a,a)\), and the quiz-score scores are \((0.2a,-1.4a,1.4a,-0.2a)\). The second column is not identical to the first, so some paired products will be negative.
| Student | Study-time z-score | Quiz-score z-score | Product |
|---|---|---|---|
| A | \(-a\) | \(0.2a\) | \(-0.2a^2=-0.15\) |
| B | \(-a\) | \(-1.4a\) | \(1.4a^2=1.05\) |
| C | \(a\) | \(1.4a\) | \(1.4a^2=1.05\) |
| D | \(a\) | \(-0.2a\) | \(-0.2a^2=-0.15\) |
The quiz-score column sums to zero because \(0.2-1.4+1.4-0.2=0\). Its squared scores sum to \((0.2^2+1.4^2+1.4^2+0.2^2)a^2=4a^2=3=n-1\). The study-time column also sums to zero and has squared scores summing to \(4a^2=3\). Both columns therefore meet the sample z-score checks.
The product total is \(-0.15+1.05+1.05-0.15=1.8\). Thus,
Two students make negative contributions because their scores are on opposite sides of the means. The other two make larger positive contributions, so the total is positive. The correlation is \(0.6000\): the paired observations have an overall tendency toward same-direction deviations, but they do not line up perfectly.
Worked Example: Perfect Opposite-Direction Standardized Scores
Consider another hypothetical set of four students. Let the study-time sample z-scores again be \((-a,-a,a,a)\), where \(a=\sqrt{3}/2\), and let the quiz-score z-scores be \((a,a,-a,-a)\). Each quiz-score z-score is the negative of the paired study-time z-score.
| Student | Study-time z-score | Quiz-score z-score | Product |
|---|---|---|---|
| A | \(-a\) | \(a\) | \(-a^2=-0.75\) |
| B | \(-a\) | \(a\) | \(-a^2=-0.75\) |
| C | \(a\) | \(-a\) | \(-a^2=-0.75\) |
| D | \(a\) | \(-a\) | \(-a^2=-0.75\) |
Each column sums to zero and has squared scores summing to \(4a^2=3=n-1\), so these are valid sample z-score columns. The sum of products is \(-0.75-0.75-0.75-0.75=-3\), giving
Every student’s scores are on opposite sides of the two means, and the standardized scores match exactly in magnitude with opposite signs. Hence every product is negative, and the correlation is \(-1\).
How to Read the Contributions
The product formula makes the role of the individual pairs visible. A positive product supports a positive overall pattern; a negative product supports a negative overall pattern. Correlation combines all these contributions into one average. It is therefore possible for a positive correlation to include some negative products, as in the second example: what matters is the net sum, not whether every pair agrees.
A product’s sign describes the directions of the two deviations, not a causal effect. For example, a positive product does not show that being above average on one measurement caused an observation to be above average on the other. Correlation describes how paired values vary together; it does not establish why they do so.
The standardized scores are unitless, so their products and their average are also unitless. Standardization expresses each observation’s position relative to its own sample mean in units of its own sample standard deviation. This lets the paired pattern be summarized without mixing, for example, hours with points.
Common Mistakes and a Practical Check
- Dividing by \(n\). The sample z-scores use sample standard deviations, whose squared deviations are scaled by \(n-1\). Their paired products are therefore divided by \(n-1\) to obtain \(r\), not by \(n\).
- Using scores that are not valid sample z-scores. For a column of \(n\) sample z-scores, the scores sum to zero and their squared scores sum to \(n-1\). If a proposed column fails these checks, either it is not a set of sample z-scores or the stated calculation needs correction.
- Looking only at the signs of the products. A few negative products do not necessarily make the correlation negative. The magnitudes matter too, because correlation uses the sum of all products.
- Breaking the pairing. Each product must use the two standardized measurements from the same observational unit. Multiplying scores from unrelated rows changes the quantity being summarized.
- Treating a product as a causal explanation. A positive or negative contribution describes the direction of a paired deviation, not what produced it.
When interpreting a correlation through standardized scores, first identify the paired observations, then check that standardization is within each sample column, and finally examine the signs and sizes of the products. This connects the numerical summary to the observed relationship without confusing an individual contribution with the overall pattern.
Check Your Understanding
Use the sample z-score product interpretation to answer these questions.
- If an observation has a positive \(x\) z-score and a negative \(y\) z-score, what is the sign of its product, and what does that sign indicate?
- For \(n=6\), what should the sum of squares of a valid column of sample z-scores equal?
- Why does the formula for \(r\) divide the sum of paired sample z-score products by \(n-1\), rather than \(n\)?
- A set of paired observations has both positive and negative products. What determines whether its correlation is positive or negative?
- Why must the two z-scores in each product come from the same observational unit?