Deriving The t Test From The Sample Correlation

Why a Correlation Test Has a t Statistic

A sample correlation $r$ summarizes the direction and strength of a linear relationship in observed paired data. To test whether the population correlation is zero, we need more than the value of $r$: we need to account for sample size and the variability expected when the population relationship is absent. Under a specific normal model, the required test statistic has a $t$ distribution with $n-2$ degrees of freedom.

The result is closely connected to the test of a zero regression slope. In simple linear regression, the sample slope and the sample correlation have the same sign, and their test statistics are algebraically identical. That connection gives both a derivation and a practical way to understand the degrees of freedom.

Definition: For $n$ paired observations $(X_i,Y_i)$, let $S_{xx}=\sum_{i=1}^n(X_i-\bar X)^2$, $S_{yy}=\sum_{i=1}^n(Y_i-\bar Y)^2$, and $S_{xy}=\sum_{i=1}^n(X_i-\bar X)(Y_i-\bar Y)$. When $S_{xx}>0$ and $S_{yy}>0$, the sample correlation is $r=S_{xy}/\sqrt{S_{xx}S_{yy}}$.

For inference about a population correlation, the standard exact model assumes the pairs are independent observations from a bivariate normal population with finite, positive variances. Write the population correlation as $\rho$. Under this model, testing $H_0:\rho=0$ is equivalent to testing for a zero slope in the conditional regression of $Y$ on $X$. The exact $t$ result depends on the normal model; a linear pattern alone does not guarantee it.

Theorem: Under independent sampling from a bivariate normal population, if $n\geq 3$, $S_{xx}>0$, and $S_{yy}>0$, then under $H_0:\rho=0$, $$ t=r\sqrt{\frac{n-2}{1-r^2}}\sim t_{n-2}. $$ The sign of $t$ is the sign of $r$. The statistic is used for a one-sided or two-sided test according to the alternative hypothesis.

Deriving the Statistic from Regression

Fit the simple linear regression of $Y$ on $X$, including an intercept. The estimated slope is $b_1=S_{xy}/S_{xx}$, and the residual sum of squares is $SSE$. In a regression model with independent normal errors of constant variance, the usual slope statistic for testing a zero slope is $b_1/SE(b_1)$, where $SE(b_1)=\sqrt{(SSE/(n-2))/S_{xx}}$. The $n-2$ occurs because two regression coefficients, the intercept and slope, are estimated.

The algebra below shows why this slope statistic is exactly the statistic expressed in terms of $r$. It also shows why the sign must be retained: squaring the statistic would lose the direction of the relationship.

Express the slope using the correlation.
From the definitions, $S_{xy}=r\sqrt{S_{xx}S_{yy}}$, so $b_1=S_{xy}/S_{xx}=r\sqrt{S_{yy}/S_{xx}}$.

Express the residual variation using the correlation.
The regression identity is $SSE=S_{yy}-S_{xy}^2/S_{xx}$. Substituting $S_{xy}^2=r^2S_{xx}S_{yy}$ gives $SSE=S_{yy}(1-r^2)$.

Form the slope standard error.
Using $SE(b_1)=\sqrt{SSE/((n-2)S_{xx})}$, substitute the expression for $SSE$: $SE(b_1)=\sqrt{S_{yy}(1-r^2)/((n-2)S_{xx})}$.

Divide and simplify.
The ratio of $b_1$ to its standard error is $r\sqrt{S_{yy}/S_{xx}}\big/\sqrt{S_{yy}(1-r^2)/((n-2)S_{xx})}=r\sqrt{(n-2)/(1-r^2)}$.

This derivation uses the regression slope statistic, but the exact null distribution can also be seen from the regression model. Conditional on the observed predictor values, normally distributed independent errors imply that the estimated slope, when the null slope is zero, is normal with mean zero. The residual sum of squares divided by the error variance has a chi-square distribution with $n-2$ degrees of freedom and is independent of the estimated slope. Dividing the standardized normal slope by the square root of this independent chi-square quantity divided by $n-2$ gives a $t_{n-2}$ variable. Under bivariate normal sampling, the conditional regression model has these properties, and $\rho=0$ implies the conditional slope is zero.

Worked Example: A Two-Sided Test with a Positive Correlation

A hypothetical study records paired measurements on $n=10$ independent individuals. The population is assumed bivariate normal, and the observed sample correlation is $r=0.60$. Test $H_0:\rho=0$ against $H_A:\rho\ne 0$ at significance level $0.05$.

Method. Use the correlation-based $t$ statistic and compare its absolute value with the two-sided critical value. The model assumptions are stipulated in the problem; the degrees of freedom are $n-2=10-2=8$.

First, $1-r^2=1-(0.60)^2=1-0.36=0.64$. Therefore, $$ t=0.60\sqrt{\frac{8}{0.64}} =0.60\sqrt{12.5} =0.60(3.5355) =2.1213. $$ The statistic is positive because the sample correlation is positive.

For a two-sided test with $8$ degrees of freedom and $\alpha=0.05$, the critical values are approximately $-2.306$ and $2.306$. Since $|2.1213|<2.306$, do not reject $H_0$. At the 5% level, the sample does not provide sufficient evidence that the population correlation differs from zero.

Check. The statistic is less extreme than the positive critical value, and the test is two-sided, so the conclusion is consistent with the comparison. The common mistake to avoid is comparing $t$ with the one-sided 5% critical value; that would use the wrong rejection region.

Degrees of Freedom, Direction, and Interpretation

The degrees of freedom are $n-2$, not $n-1$. A simple regression with an intercept estimates two parameters: the population intercept and slope. The residual variation used to estimate error variance therefore has $n-2$ degrees of freedom. The correlation formula inherits this same $n-2$ through its equivalence to the slope test.

The hypotheses determine how to use the statistic. For $H_A:\rho\ne 0$, compare $|t|$ with the two-sided critical value, or use both tails to obtain the p-value. For $H_A:\rho>0$, use the upper tail; for $H_A:\rho<0$, use the lower tail. A negative $r$ produces a negative $t$, not a positive one.

The test concerns evidence about a population linear association under the stated model. It does not establish causation. Also, a failure to reject $H_0$ is not proof that $\rho=0$; it means the data do not give sufficient evidence against zero at the chosen significance level.

Worked Example: A One-Sided Test for Negative Correlation

A hypothetical quality-control investigation collects $n=12$ independent pairs of measurements. Assume a bivariate normal population, and suppose the sample correlation is $r=-0.50$. Test $H_0:\rho=0$ against $H_A:\rho<0$ at level $0.05$.

Method. Calculate the signed statistic and compare it with the lower-tail critical value. The degrees of freedom are $12-2=10$.

Here $r^2=0.25$, so $1-r^2=0.75$. Thus $$ t=-0.50\sqrt{\frac{10}{0.75}} =-0.50\sqrt{\frac{40}{3}} =-1.8257. $$ The one-sided 5% lower critical value for $t_{10}$ is approximately $-1.812$.

Because $-1.8257<-1.812$, reject $H_0$. The result provides evidence at the 5% level that the population correlation is negative.

Check. The sign of the statistic agrees with the negative-direction alternative, and its magnitude passes the lower-tail cutoff. A common mistake is to replace $r$ by $|r|$ and test in the upper tail; that would discard the direction specified in the alternative.

When the Exact Test Does Not Apply

The statistic can always be calculated when $n\geq3$, the sample variances are positive, and $|r|<1$. However, its exact $t_{n-2}$ null distribution requires the model assumptions. In particular, strongly nonnormal paired data, influential outliers, dependent observations, or a relationship that is not adequately represented by the model can make the exact test unreliable. With small samples, these concerns matter especially because there is little data to average out unusual observations.

If $|r|=1$, the denominator $1-r^2$ is zero. The formula then gives no finite test statistic: the sample points lie perfectly on a line, and ordinary residual variation is zero. This boundary case should not be handled by substituting into the formula as though the denominator were positive. If $n=2$, there are no residual degrees of freedom and the test is not defined.

The connection to regression is useful for checking work, but it does not remove the assumptions. Under the bivariate normal model, testing zero population correlation is equivalent to testing zero regression slope. Outside that model, a slope test and a correlation test need not support the same exact inference.

Worked Example: Recovering the Test from Regression Sums

For $n=7$ independent paired observations from a bivariate normal population, suppose $S_{xx}=20$, $S_{yy}=45$, and $S_{xy}=-24$. Test $H_0:\rho=0$ against $H_A:\rho\ne0$ at level $0.05$.

Method. First compute $r$ from the sums, then use the correlation statistic. As an independent check, compute the regression slope statistic directly.

The correlation is $$ r=\frac{-24}{\sqrt{20(45)}}=\frac{-24}{\sqrt{900}}=\frac{-24}{30}=-0.80. $$ There are $n-2=5$ degrees of freedom, and $1-r^2=1-0.64=0.36$. Hence $$ t=-0.80\sqrt{\frac{5}{0.36}} =-0.80\left(\frac{\sqrt{5}}{0.60}\right) =-2.9814. $$

For the regression check, the estimated slope is $b_1=S_{xy}/S_{xx}=-24/20=-1.20$. The residual sum of squares is $$ SSE=S_{yy}-\frac{S_{xy}^2}{S_{xx}} =45-\frac{576}{20} =45-28.8 =16.2. $$ The residual mean square is $16.2/5=3.24$, so $$ SE(b_1)=\sqrt{\frac{3.24}{20}}=\sqrt{0.162}=0.4025, \qquad \frac{b_1}{SE(b_1)}=\frac{-1.20}{0.4025}=-2.9814. $$

The two calculations agree. The two-sided 5% critical value for $t_5$ is approximately $2.571$. Since $|-2.9814|>2.571$, reject $H_0$; the data provide evidence of a nonzero population correlation.

Check. The regression statistic and correlation statistic match, including the negative sign. The common mistake is to calculate $SSE$ as $S_{yy}$ without subtracting $S_{xy}^2/S_{xx}$; that would ignore the variation explained by the fitted line and give the wrong standard error.

Common Exam Traps

Dropping the sign. The formula uses $r$, not $|r|$. Retain the sign for directional tests; only a two-sided comparison uses $|t|$.
Using $n-1$ degrees of freedom. The correct degrees of freedom are $n-2$, because the equivalent simple regression estimates an intercept and slope.
Using a one-sided cutoff for a two-sided alternative. For $H_A:\rho\ne0$, the significance level is split between two tails.
Assuming every calculated statistic has an exact $t$ distribution. The exact null result requires the stated normal sampling or regression assumptions.
Interpreting non-rejection as proof of zero correlation. A test result describes the evidence in the sample at a chosen significance level; it does not prove the null hypothesis.

Key takeaway: Under the bivariate normal model, the zero-correlation test uses $t=r\sqrt{(n-2)/(1-r^2)}$ with $n-2$ degrees of freedom. The statistic is the simple-regression slope statistic written in terms of $r$, so preserve its sign and match the rejection region to the alternative.

Check Your Understanding

For each problem, use the stated assumptions and significance level. Check both the calculation and the direction of the test.

A bivariate normal sample has $n=9$ and $r=0.50$. Test $H_0:\rho=0$ against a two-sided alternative at 5%. Give the statistic, degrees of freedom, and decision. (Answer: $t=1.5275$, $7$ degrees of freedom; do not reject.)
A bivariate normal sample has $n=15$ and $r=-0.60$. For $H_A:\rho<0$, test at 1%. Give the statistic and decision using the lower-tail critical value $t_{0.01,13}\approx-2.650$. (Answer: $t=-2.7042$, $13$ degrees of freedom; reject.)
For $n=8$, suppose $S_{xx}=14$, $S_{yy}=32$, and $S_{xy}=16$, with independent bivariate normal observations. Find $r$ and the test statistic for $H_0:\rho=0$. At the two-sided 5% level, use the critical value $t_{0.025,6}\approx2.447$ to decide. (Answer: $r=2/\sqrt7\approx0.7559$, $t=2\sqrt2\approx2.8284$, $6$ degrees of freedom; reject.)
A bivariate normal sample has $n=20$ and $r=-0.25$. Test $H_0:\rho=0$ against $H_A:\rho\ne0$ at 5%. Give the statistic and decision using the two-sided $t_{18}$ critical value $2.101$. (Answer: $t\approx-1.0954$, $18$ degrees of freedom; do not reject.)
A sample of $n=11$ paired observations has $r=0.70$, but the data contain a severe outlier and are not plausibly bivariate normal. Calculate the usual statistic and state whether its exact $t_9$ reference distribution is justified. (Answer: $t\approx2.9406$, $9$ nominal degrees of freedom; the calculation is defined, but the exact $t_9$ inference is not justified by the stated data conditions.)

What You'll Learn