Testing Nonzero Population Correlations With Fisher's Transform

Why Transform a Sample Correlation?

A sample correlation $r$ is bounded between $-1$ and $1$, and its sampling distribution can be skewed, especially when the population correlation is far from zero. This makes a direct normal approximation for $r$ less reliable. Fisher’s transformation converts correlations to an unbounded scale on which the sampling distribution is approximately normal and has a nearly constant standard error.

This procedure is useful when the hypothesis specifies a nonzero population correlation. The hypotheses are tested on the transformed scale: transform both the observed sample correlation and the hypothesized population value, then compare their difference with its standard error. The method is approximate and depends on the sampling model.

Definition: For any correlation $c$ with $-1<c<1$, its Fisher transformation is $$ z(c)=\frac{1}{2}\ln\!\left(\frac{1+c}{1-c}\right) =\operatorname{atanh}(c). $$ For a sample correlation $r$, the transformed value is $z(r)$. For a hypothesized population correlation $\rho_0$, the transformed null value is $z(\rho_0)$.

Model, Standard Error, and Test Statistic

Suppose the data consist of $n$ independent, identically distributed pairs $(X_i,Y_i)$ from a bivariate normal population with population correlation $\rho$. The marginal means and variances may be unknown; the test concerns the association parameter $\rho$. The pairs must be independent across subjects, and the observations should not contain influential outliers that make the sample correlation misleading.

For a specified null value $\rho_0$, with $-1<\rho_0<1$, test $H_0:\rho=\rho_0$. Under the bivariate normal model, Fisher’s transformed sample correlation is approximately normal with standard error $1/\sqrt{n-3}$. The resulting test statistic is:

W=\frac{z(r)-z(\rho_0)}{1/\sqrt{n-3}} =\sqrt{n-3}\bigl[z(r)-z(\rho_0)\bigr] \ \approx\ N(0,1)\quad\text{under }H_0.

The approximation is generally more useful for moderate or large samples than for very small samples. It is not an exact finite-sample test. In particular, the standard error formula requires $n>3$, and both $r$ and $\rho_0$ must lie strictly between $-1$ and $1$. If the observations are strongly nonnormal, dependent, or dominated by outliers, the stated approximation may not be dependable.

Why Fisher’s Transformation Helps

The transform is increasing and has derivative $z'(c)=1/(1-c^2)$. A delta-method argument explains its variance-stabilizing effect. Under the bivariate normal model, the large-sample variance of the sample correlation is approximately $(1-\rho^2)^2/n$. Applying the delta method gives:

\operatorname{Var}\bigl(z(r)\bigr) \ \approx\ [z'(\rho)]^2\operatorname{Var}(r) =\frac{1}{(1-\rho^2)^2}\cdot\frac{(1-\rho^2)^2}{n} =\frac{1}{n}.

Thus the transformation approximately removes the dependence of the variance on the unknown correlation. Fisher’s commonly used finite-sample standard error is $1/\sqrt{n-3}$, a refinement of the large-sample $1/\sqrt{n}$ result. Under $H_0$, the transformed sample value is centered approximately at $z(\rho_0)$, which yields the test statistic $W$.

Choosing the Rejection Rule

The alternative hypothesis determines which tail or tails of the standard normal distribution are relevant. For a two-sided test at significance level $\alpha$, reject when $|W|>z_{1-\alpha/2}$. For an upper-sided alternative, reject when $W>z_{1-\alpha}$; for a lower-sided alternative, reject when $W<-z_{1-\alpha}$. Here $z_q$ denotes the $q$-quantile of the standard normal distribution.

An approximate p-value is calculated from the standard normal distribution in the same way. For example, a two-sided p-value is $2P(Z\ge |W|)$, where $Z\sim N(0,1)$. A one-sided p-value uses the tail specified by the alternative. State the alternative before calculating the p-value: using the wrong tail changes the decision.

Set the hypotheses.
Write the specified null correlation and the direction, if any, of the alternative.

Transform both correlations.
Calculate $z(r)$ and $z(\rho_0)$; do not subtract the untransformed correlations.

Calculate the standard error and statistic.
Use $1/\sqrt{n-3}$ and $W=\sqrt{n-3}[z(r)-z(\rho_0)]$.

Use the appropriate normal tail.
Compare $W$ with the critical value or calculate the p-value for the stated alternative.

Conclude in context.
State whether the evidence supports a population correlation different from, greater than, or less than the specified value.

Worked Applications

Worked Example: Two-Sided Test of a Positive Null Correlation

A hypothetical study records paired measurements on $n=40$ independent subjects. Assume the pairs follow a bivariate normal distribution with unknown marginal means and variances and population correlation $\rho$. The sample correlation is $r=0.50$. At the 5% level, test $H_0:\rho=0.20$ against $H_1:\rho\ne0.20$.

Method. Use the two-sided Fisher-transformation test. The null value is nonzero, so both $r$ and $\rho_0$ must be transformed.

First calculate the transformed values:

z(r)=\frac12\ln\!\left(\frac{1+0.50}{1-0.50}\right) =\frac12\ln(3)=0.549306, $$ $$ z(\rho_0)=\frac12\ln\!\left(\frac{1+0.20}{1-0.20}\right) =\frac12\ln(1.5)=0.202733.

The standard error is $1/\sqrt{40-3}=1/\sqrt{37}=0.1644$, to four decimal places. Therefore:

W=\frac{0.549306-0.202733}{0.1644} =\frac{0.346573}{0.1644} \approx 2.108.

At the 5% two-sided level, the critical values are approximately $-1.96$ and $1.96$. Since $|2.108|>1.96$, reject $H_0$. The approximate two-sided p-value is $0.0350$, to four decimal places. The data provide evidence that the population correlation differs from $0.20$. Check: the observed correlation exceeds the null value, and $W$ is positive, consistent with that direction. A common mistake is to use $0.50-0.20=0.30$ as the numerator; the test requires the difference of the transformed values, $0.346573$.

Worked Example: Upper-Sided Test of a Negative Null Correlation

In a hypothetical investigation of paired measurements, $n=30$ independent pairs are sampled from a bivariate normal population with population correlation $\rho$. The observed sample correlation is $r=0.10$. Test $H_0:\rho=-0.30$ against $H_1:\rho>-0.30$ at the 5% level.

Method. Use the upper tail because the alternative claims that the population correlation is greater than $-0.30$. Transform the observed and null correlations:

z(r)=\frac12\ln\!\left(\frac{1.10}{0.90}\right)=0.100335, \qquad z(\rho_0)=\frac12\ln\!\left(\frac{0.70}{1.30}\right)=-0.309520.

The standard error is $1/\sqrt{30-3}=1/\sqrt{27}=0.1925$, to four decimal places. Hence:

W=\frac{0.100335-(-0.309520)}{0.1925} =\frac{0.409855}{0.1925} \approx 2.130.

The 5% upper-tail critical value is $1.645$. Since $2.130>1.645$, reject $H_0$. The approximate upper-tail p-value is $0.0166$, to four decimal places. The sample supports a population correlation greater than $-0.30$. Check: because $z$ is increasing, $r=0.10>-0.30$ implies $z(r)>z(-0.30)$, so the numerator and statistic are positive. A common mistake is to use a two-sided p-value automatically; this specified alternative requires only the upper tail.

Worked Example: A Large Sample Correlation That Is Not Significant

A hypothetical study collects $n=25$ independent pairs from a bivariate normal population with correlation $\rho$. Its sample correlation is $r=0.72$. Test $H_0:\rho=0.60$ against $H_1:\rho\ne0.60$ at the 5% level.

Method. Apply the two-sided Fisher test. The difference between the observed and null correlations looks noticeable on the original scale, but significance depends on that difference relative to its sampling standard error.

z(r)=\frac12\ln\!\left(\frac{1.72}{0.28}\right) =\frac12\ln(6.142857)=0.907645, $$ $$ z(\rho_0)=\frac12\ln\!\left(\frac{1.60}{0.40}\right) =\frac12\ln(4)=0.693147.

Here $n-3=22$, so the standard error is $1/\sqrt{22}=0.2132$, to four decimal places. The test statistic is:

W=\frac{0.907645-0.693147}{0.2132} =\frac{0.214498}{0.2132} \approx 1.006.

Since $|1.006|<1.96$, do not reject $H_0$ at the 5% level. The approximate two-sided p-value is $0.3143$, to four decimal places. The observed sample correlation is $0.72$, but the data do not provide sufficient evidence that the population correlation differs from $0.60$. Check: the transformed difference is positive but small relative to the standard error, so the statistic is positive and less than the two-sided critical value. A common mistake is to treat a high sample correlation by itself as proof that the population correlation differs from the specified value.

How This Test Differs From the Zero-Correlation t Test

The exact t test derived from the sample correlation applies to testing zero population correlation under its specified assumptions. Its test statistic uses $r\sqrt{n-2}/\sqrt{1-r^2}$. For a specified nonzero value, that formula is not the general test statistic: the Fisher test instead compares $z(r)$ with $z(\rho_0)$ and uses the approximate standard error $1/\sqrt{n-3}$.

This distinction is especially important when a question gives a nonzero benchmark. Do not substitute $\rho_0$ into the zero-correlation t formula or act as if the null is zero. The Fisher procedure is approximate, whereas the zero-correlation t result has a different distributional justification. The conclusion from either procedure concerns population association under its assumptions; it does not establish causation.

Key takeaway: For an approximate test of $H_0:\rho=\rho_0$, transform both correlations, calculate $W=\sqrt{n-3}[z(r)-z(\rho_0)]$, and use the normal tail specified by the alternative. State the bivariate-normal sampling assumption and interpret the result as evidence about $\rho$, not as proof of a causal relationship.

Check Your Understanding

For each problem, assume independent paired observations from a bivariate normal population unless the question asks about applicability. Use approximate standard normal probabilities and report p-values to four decimal places.

A sample of $n=35$ pairs has $r=0.40$. Test $H_0:\rho=0.10$ against $H_1:\rho\ne0.10$ at the 5% level. (Answer: $W\approx1.829$, p-value $\approx0.0674$; do not reject.)
For $n=50$ pairs, $r=0.65$. Test $H_0:\rho=0.40$ against $H_1:\rho>0.40$ at the 5% level. (Answer: $W\approx2.410$, upper-tail p-value $\approx0.0080$; reject.)
For $n=20$ pairs, $r=-0.55$. Test $H_0:\rho=-0.20$ against $H_1:\rho<-0.20$ at the 5% level. (Answer: $W\approx-1.714$, lower-tail p-value $\approx0.0433$; reject.)
A study has only $n=3$ paired observations and reports a sample correlation strictly between $-1$ and $1$. Can the standard Fisher test statistic given here be calculated? (Answer: No; $n-3=0$, so its stated standard error is undefined and the approximation is not applicable.)
For $n=60$ pairs, $r=0.25$. Test $H_0:\rho=-0.10$ against $H_1:\rho\ne-0.10$ at the 5% level. (Answer: $W\approx2.685$, two-sided p-value $\approx0.0072$; reject.)

What You'll Learn