Why Put a Correlation on a Different Scale?
A sample Pearson correlation \(r\) estimates the population Pearson correlation \(\rho\), but its sampling distribution is generally not well represented by a normal distribution, particularly when \(\rho\) is far from zero. Also, a correlation must lie between \(-1\) and \(1\), while a normal distribution is unbounded. A confidence interval calculated directly as \(r\) plus or minus a normal margin can therefore have poor coverage or even extend beyond the possible range.
Fisher’s \(z\) transformation addresses these issues by mapping correlations from \((-1,1)\) to the whole real line. A confidence interval is constructed on that transformed scale and then mapped back to the correlation scale. The resulting interval is usually not symmetric around \(r\).
For the usual two-sided interval, the standard approximation requires independent, identically distributed paired observations from a bivariate normal population. The pairs may have unknown means and positive variances; the parameter of interest is their population correlation \(\rho\). The approximation is not exact for every finite sample. It is commonly used when the sample size is adequate and the model assumptions are plausible.
Why the Interval Has This Form
The key result is that the transformed sample correlation is approximately normal, with its center near \(\theta=\operatorname{atanh}(\rho)\) and standard error \(1/\sqrt{n-3}\). The transformed scale is useful because it makes the sampling variability much less dependent on the unknown value of \(\rho\). The \(n-3\) expression is a standard finite-sample refinement to the large-sample approximation; it is not the exact variance for every sample.
Under the stated model, \(w=\operatorname{atanh}(r)\) is approximately normal with mean \(\theta\) and standard error \(1/\sqrt{n-3}\).
Approximately, \(P\!\left(-z_{1-\alpha/2}\leq (w-\theta)\sqrt{n-3}\leq z_{1-\alpha/2}\right)=1-\alpha\).
Solving the inequalities for \(\theta\) gives \(w-z_{1-\alpha/2}/\sqrt{n-3}\leq\theta\leq w+z_{1-\alpha/2}/\sqrt{n-3}\).
The hyperbolic tangent is increasing, so applying it to both endpoints preserves their order and gives the interval for \(\rho\).
This derivation also explains the interval’s interpretation. In repeated samples from the same model, the procedure is designed to contain the fixed population correlation approximately \(100(1-\alpha)\%\) of the time. After a particular interval has been calculated, it is not correct to say that the fixed \(\rho\) has a \(95\%\) probability of lying in that particular interval.
Constructing and Interpreting the Interval
The calculation has two scales. First calculate the center and margin on the Fisher scale. Add and subtract the margin there. Then back-transform each endpoint separately. Keep unrounded values through the back-transformation and round only the final endpoints. The number \(n-3\), not \(n-1\), appears under the square root in this standard approximation.
Worked Example: A 90% Interval for Paired Risk Estimates
A hypothetical analysis has \(n=60\) independent paired observations, each pairing two risk estimates for the same unit. The sample Pearson correlation is \(r=0.40\). Assume the pairs are identically distributed and bivariate normal, with unknown marginal means and positive variances and population correlation \(\rho\). There is no discounting or mortality model in this statistical calculation. Construct a 90% confidence interval for \(\rho\), using \(z_{0.95}=1.645\).
Method and transformed center. Apply Fisher’s transformation: \(\operatorname{atanh}(0.40)=\frac12\ln(1.40/0.60)=\frac12\ln(2.333333\ldots)\approx0.4236489\).
Standard error and margin. Here \(n-3=60-3=57\), so \(\sqrt{57}\approx7.549834\) and the standard error is \(1/\sqrt{57}\approx0.1324532\). The margin is \(1.645/\sqrt{57}\approx0.2178856\).
Transformed endpoints. The lower endpoint is \(0.4236489-0.2178856=0.2057633\), and the upper endpoint is \(0.4236489+0.2178856=0.6415345\).
Back-transform. Applying \(\tanh\) gives \((\tanh(0.2057633),\tanh(0.6415345))\approx(0.2029,0.5661)\), rounding the final endpoints to four decimal places.
Interpretation and check. Under the model assumptions, the procedure gives an approximate 90% confidence interval from 0.2029 to 0.5661 for the population correlation. Both endpoints are between \(-1\) and \(1\), and the interval is not centered at the sample value \(r=0.40\). The method avoids constructing a symmetric interval directly around \(r\).
Worked Example: Does a 95% Interval Include Zero?
A hypothetical study records \(n=45\) independent pairs of monthly risk measurements and obtains \(r=0.32\). Assume identically distributed bivariate-normal pairs, with unknown marginal means and positive variances and population correlation \(\rho\). Construct the approximate 95% interval using \(z_{0.975}=1.96\), then assess whether it includes zero.
Method and transformed center. Calculate \(w=\operatorname{atanh}(0.32)=\frac12\ln(1.32/0.68) =\frac12\ln(33/17)\approx0.3316471\).
Standard error and margin. Since \(n-3=45-3=42\), \(\sqrt{42}\approx6.480741\), and \(1/\sqrt{42}\approx0.1543033\). Thus the margin is \(1.96/\sqrt{42}\approx0.3024346\).
Transformed interval. The lower transformed endpoint is \(0.3316471-0.3024346=0.0292125\). The upper is \(0.3316471+0.3024346=0.6340817\). Back-transforming gives \((\tanh(0.0292125),\tanh(0.6340817))\approx(0.0292,0.5609)\), to four decimal places.
Interpretation and check. Zero is not in this interval: its lower endpoint is positive. This is evidence against a zero population correlation at the corresponding two-sided 5% level under the model assumptions. The check can be made before back-transformation because \(\tanh\) is increasing and \(\tanh(0)=0\). The calculation avoids mistaking a rounded intermediate value for the unrounded margin: \(1.96/\sqrt{42}\approx0.3024346\), not \(0.30246\).
Worked Example: A 95% Interval With a Small Sample
A hypothetical portfolio study has \(n=19\) independent paired observations of two risk measures and reports \(r=0.50\). Assume the pairs are identically distributed and bivariate normal, with unknown marginal means and positive variances and population correlation \(\rho\). Find the approximate 95% interval using \(z_{0.975}=1.96\).
Method and transformed center. The Fisher-transformed sample correlation is \(w=\operatorname{atanh}(0.50)=\frac12\ln(3)\approx0.5493061\).
Standard error and margin. Identify \(n-3=19-3=16\) before calculating. Therefore \(\sqrt{n-3}=4\), the standard error is \(1/4=0.25\), and the margin is \(1.96(0.25)=0.49\).
Transformed endpoints. The limits are \(0.5493061-0.49=0.0593061\) and \(0.5493061+0.49=1.0393061\).
Back-transform and interpret. The interval is \((\tanh(0.0593061),\tanh(1.0393061))\approx(0.0592,0.7776)\), to four decimal places. It is entirely positive, although the lower endpoint is close to zero. The method avoids using \(n-1\) in the standard error; the specified Fisher approximation uses \(n-3\).
Worked Example: A 99% Interval and Asymmetry
A hypothetical analysis of \(n=67\) independent paired loss indicators gives \(r=0.45\). Assume identically distributed bivariate-normal pairs, with unknown marginal means and positive variances and population correlation \(\rho\). Construct a 99% interval using \(z_{0.995}=2.576\), and determine whether it is symmetric around \(r\).
Method and transformed center. \(w=\operatorname{atanh}(0.45)=\frac12\ln(1.45/0.55)\approx0.4847003\).
Standard error and margin. Here \(n-3=64\), so \(\sqrt{64}=8\), the standard error is \(1/8=0.125\), and the margin is \(2.576/8=0.322\).
Transformed endpoints and back-transformation. The limits are \(0.4847003-0.322=0.1627003\) and \(0.4847003+0.322=0.8067003\). Thus the correlation interval is \((\tanh(0.1627003),\tanh(0.8067003))\approx(0.1613,0.6678)\), to four decimal places.
Interpretation and check. The distances from \(r\) to the endpoints are approximately \(0.45-0.1613=0.2887\) and \(0.6678-0.45=0.2178\). They differ, so the interval is not symmetric around \(r\). This asymmetry is expected from back-transforming a symmetric interval on the Fisher scale.
Assumptions, Scope, and Exam Traps
The confidence level describes the performance of a procedure under repeated sampling; it does not remove uncertainty about whether the model describes the paired data. The Fisher interval targets population Pearson correlation, a measure of linear association. It does not by itself establish causation or describe all forms of dependence between the variables.
The principal assumptions are that each observation is a pair, that the pairs are independent and identically distributed, and that the bivariate-normal model is a reasonable basis for the approximation. Dependence across pairs, such as repeated observations on the same unit treated as separate independent pairs, can invalidate the stated standard error. Strong departures from the assumed model can also undermine the approximation. A computed interval should therefore be reported together with the assumptions supporting it.
- Using the wrong sample-size adjustment. The standard error in this method is \(1/\sqrt{n-3}\), not \(1/\sqrt{n-1}\) or \(1/\sqrt{n}\).
- Stopping on the transformed scale. The transformed limits are limits for \(\operatorname{atanh}(\rho)\); apply \(\tanh\) to both to obtain limits for \(\rho\).
- Forgetting the transformation changes the midpoint. The Fisher-scale interval is symmetric around \(\operatorname{atanh}(r)\). Its back-transformed interval is generally not symmetric around \(r\).
- Rounding too early. Calculate the standard error, margin, and transformed endpoints at full available precision. Round only the final correlation endpoints.
- Making a probability claim about a fixed parameter. The confidence level is a repeated-sampling property of the method, not a posterior probability for the fixed \(\rho\) after observing the data.
Check Your Understanding
For each problem, assume independent, identically distributed bivariate-normal pairs with unknown marginal means and positive variances, unless the question asks you to assess an assumption. Keep unrounded values until the final step.
- A hypothetical sample of \(n=28\) paired observations has \(r=0.20\). Find a 90% interval using \(z_{0.95}=1.645\). (Answer: \(n-3=25\), transformed center \(\approx0.2027\), margin \(0.3290\), interval \(\approx(-0.1256,0.4867)\).)
- A hypothetical sample has \(n=35\) and \(r=-0.25\). Find a 95% interval using \(z_{0.975}=1.96\), and state whether it includes zero. (Answer: \(n-3=32\), transformed center \(\approx-0.2554\), margin \(\approx0.3465\), interval \(\approx(-0.5385,0.0908)\); it includes zero.)
- A risk analyst has \(n=52\) paired observations with \(r=0.55\). Find a 99% interval using \(z_{0.995}=2.576\), and interpret whether it contains zero. (Answer: \(n-3=49\), margin \(2.576/7=0.368\), transformed limits \(\approx0.2504\) and \(0.9864\), interval \(\approx(0.2453,0.7558)\); it excludes zero.)
- Using the Fisher approximation, how many independent pairs are needed for the transformed-scale standard error to be at most \(0.10\)? (Answer: \(1/\sqrt{n-3}\leq0.10\) requires \(n-3\geq100\), so at least \(103\) pairs.)
- A hypothetical sample has \(n=24\) and \(r=-0.60\). Construct the 95% interval using \(z_{0.975}=1.96\), and decide whether the interval includes zero. (Answer: \(n-3=21\); the interval is \(\left(\tanh[-0.6931472-1.96/\sqrt{21}],\tanh[-0.6931472+1.96/\sqrt{21}]\right)\), approximately \((-0.808, -0.259)\), and it does not include zero.)