Distance Correlation and Its Independence Characterization

From Distance Covariance to a Scale-Free Measure

Distance covariance measures dependence by comparing the joint characteristic function of $X$ and $Y$ with the product of their marginal characteristic functions. Its value can change when the measurement units change: recording height in centimetres rather than metres changes the numerical size of the statistic. Distance correlation removes this scale effect by normalizing distance covariance with the corresponding marginal distance covariances.

Let $X$ take values in $\mathbb{R}^p$ and $Y$ in $\mathbb{R}^q$, with $p,q\geq 1$. Write their characteristic functions as $\varphi_X(t)=E[e^{i t^\top X}]$ and $\varphi_Y(s)=E[e^{i s^\top Y}]$, and the joint characteristic function as $\varphi_{XY}(t,s)=E[e^{i(t^\top X+s^\top Y)}]$. The population definitions below use the Euclidean norm.

Definition: For an integer $d\geq 1$, let $c_d=\pi^{(d+1)/2}/\Gamma((d+1)/2)$. If $E\|X\|+E\|Y\|<\infty$, the squared distance covariance is $$ \mathcal V^2(X,Y)=\frac{1}{c_p c_q}\int_{\mathbb R^p}\int_{\mathbb R^q} \frac{|\varphi_{XY}(t,s)-\varphi_X(t)\varphi_Y(s)|^2} {\|t\|^{p+1}\|s\|^{q+1}}\,ds\,dt. $$ Define $\mathcal V^2(X)=\mathcal V^2(X,X)$, regarding $(X,X)$ as a pair of $p$-dimensional variables. If both marginal distance variances are positive, the squared distance correlation and distance correlation are $$ \mathcal R^2(X,Y)=\frac{\mathcal V^2(X,Y)} {\sqrt{\mathcal V^2(X)\mathcal V^2(Y)}}, \qquad \mathcal R(X,Y)=\sqrt{\mathcal R^2(X,Y)}. $$

The superscript $2$ in $\mathcal V^2$ and $\mathcal R^2$ is part of the standard notation for the squared quantities; it does not mean that the displayed integral itself should be squared again. The finite-first-moment condition makes the distance-covariance integral finite. Distance variance is positive exactly when the corresponding random vector is not almost surely constant. Consequently, the ratio defining distance correlation requires both $X$ and $Y$ to be nonconstant.

Why the Normalization Is Well Behaved

Distance correlation is dimensionless and lies between zero and one when it is defined. A useful way to see the bound is through a Hilbert-space representation of Euclidean distance. The identity

\int_{\mathbb R^d}\frac{1-\cos(t^\top x)}{\|t\|^{d+1}}\,dt=c_d\|x\|

gives a feature map $\Phi_d(x)$ into a complex $L^2$ space such that $\|\Phi_d(x)-\Phi_d(x')\|^2=\|x-x'\|$. For example, its coordinate at $t$ can be taken as $(e^{it^\top x}-1)/\sqrt{2c_d\|t\|^{d+1}}$. The identity also shows $E\|\Phi_d(X)\|^2=E\|X\|<\infty$, so centering this feature map is legitimate under the stated moment condition.

Let $U=\Phi_p(X)-E\Phi_p(X)$ and $W=\Phi_q(Y)-E\Phi_q(Y)$. The cross-covariance tensor $C=E[U\otimes W]$ is Hilbert–Schmidt. Substitution of the feature maps in the characteristic-function definition gives $\mathcal V^2(X,Y)=4\|C\|_{\mathrm{HS}}^2$; applying the same identity to each marginal gives $\mathcal V^2(X)=4\|E[U\otimes U]\|_{\mathrm{HS}}^2$, and likewise for $Y$.

Theorem: If $E\|X\|+E\|Y\|<\infty$ and both variables are nonconstant, then $0\leq\mathcal R(X,Y)\leq1$.

Proof. Take an independent copy $(U',W')$ of $(U,W)$. The Hilbert–Schmidt norm of the cross-covariance tensor satisfies

\|C\|_{\mathrm{HS}}^2 = E\!\left[\langle U,U'\rangle\langle W,W'\rangle\right] \leq \sqrt{E|\langle U,U'\rangle|^2\ E|\langle W,W'\rangle|^2}.

The inequality is Cauchy–Schwarz. The first expectation under the square root is $\|E[U\otimes U]\|_{\mathrm{HS}}^2$, and the second is $\|E[W\otimes W]\|_{\mathrm{HS}}^2$. Multiplying the resulting inequality by $4$, and using the three distance-covariance identities above, yields

\mathcal V^2(X,Y)\leq \sqrt{\mathcal V^2(X)\mathcal V^2(Y)}.

Distance covariance is nonnegative by its integral definition. Dividing by the positive denominator proves $0\leq\mathcal R^2(X,Y)\leq1$, hence $0\leq\mathcal R(X,Y)\leq1$. $\square$

Scale and Coordinate Transformations

The integral definition makes the effect of rescaling explicit. For nonzero scalars $a$ and $b$, and fixed vectors $u$ and $v$, replacing $X$ by $aX+u$ and $Y$ by $bY+v$ changes the joint characteristic-function difference only by a phase factor and a change of integration variables. The phase has modulus one; substituting $r=at$ and $w=bs$ in the integral gives

\mathcal V^2(aX+u,bY+v)=|a||b|\mathcal V^2(X,Y).

Applying the same calculation to each marginal gives $\mathcal V^2(aX+u)=a^2\mathcal V^2(X)$ and $\mathcal V^2(bY+v)=b^2\mathcal V^2(Y)$. Thus the factors cancel in the ratio: $\mathcal R(aX+u,bY+v)=\mathcal R(X,Y)$. More generally, translations and orthogonal transformations preserve Euclidean distances, so they also leave distance covariance unchanged; scalar multiplication adds the factors just derived. Invariance under arbitrary nonsingular linear transformations does not follow: such transformations can change distances by different amounts in different directions.

Worked Example: Distance Correlation for Two Associated Binary Variables

Suppose $X,Y\in\{0,1\}$ have joint probabilities $P(X=0,Y=0)=0.4$, $P(X=0,Y=1)=0.1$, $P(X=1,Y=0)=0.1$, and $P(X=1,Y=1)=0.4$. Both marginal probabilities of 1 are $1/2$.

For a binary variable with success probability $r$, Euclidean distance is $|x-x'|$. Double-centering this distance gives $-2(x-r)(x'-r)$. To check the identity, the distance is $x+x'-2xx'$; subtracting its two conditional means and adding its overall mean leaves $-2(x-r)(x'-r)$. For bounded variables, the centered-distance expression for squared distance covariance is the expectation of the product of the two double-centered distances, using independent copies of the paired observations. Therefore, for these binary variables,

\mathcal V^2(X,Y)=4\{\operatorname{Cov}(X,Y)\}^2.

Here $E[XY]=P(X=1,Y=1)=0.4$, so $\operatorname{Cov}(X,Y)=0.4-(0.5)(0.5)=0.15$. Hence $\mathcal V^2(X,Y)=4(0.15)^2=0.09$. For either marginal, $\mathcal V^2(X)=4\{\operatorname{Var}(X)\}^2$; since $\operatorname{Var}(X)=\operatorname{Var}(Y)=0.25$, both marginal distance variances equal $4(0.25)^2=0.25$. It follows that

\mathcal R^2(X,Y)=\frac{0.09}{\sqrt{0.25(0.25)}}=0.36, \qquad \mathcal R(X,Y)=\sqrt{0.36}=0.6.

For this binary pair the distance correlation equals the absolute Pearson correlation. That agreement in this special case should not be taken as a general identity for arbitrary distributions.

Worked Example: Checking the Scale Invariance Numerically

Use the preceding binary pair, and set $U=3X+7$ and $W=-2Y+4$. The transformation multiplies the distances in the $X$ margin by 3 and those in the $Y$ margin by 2. The scale rules therefore give

\mathcal V^2(U,W)=6(0.09)=0.54,\qquad \mathcal V^2(U)=9(0.25)=2.25,\qquad \mathcal V^2(W)=4(0.25)=1.

The normalized quantities are $\mathcal R^2(U,W)=0.54/\sqrt{2.25(1)}=0.36$ and $\mathcal R(U,W)=0.6$, exactly as before. The negative multiplier reverses the numerical ordering of $Y$, but distance correlation depends on distances, so the reversal does not change it.

Zero Distance Covariance Characterizes Independence

The central property of distance covariance is stronger than detecting linear association. Its weighted characteristic-function integral vanishes precisely when the joint characteristic function factors into its marginal characteristic functions. Characteristic functions determine probability distributions, so this factorization is equivalent to independence.

Theorem: If $X\in\mathbb R^p$ and $Y\in\mathbb R^q$ satisfy $E\|X\|+E\|Y\|<\infty$, then $$ \mathcal V^2(X,Y)=0 \quad\Longleftrightarrow\quad X\ \text{and}\ Y\ \text{are independent}. $$ If both variables are nonconstant, the same equivalence holds with $\mathcal R(X,Y)=0$ in place of $\mathcal V^2(X,Y)=0$.

Proof. Set $g(t,s)=\varphi_{XY}(t,s)-\varphi_X(t)\varphi_Y(s)$. This is a continuous function because characteristic functions are continuous. The integrand defining $\mathcal V^2$ is nonnegative, and its weight is strictly positive whenever $t\ne0$ and $s\ne0$.

If $X$ and $Y$ are independent, their joint characteristic function factors everywhere, so $g(t,s)=0$ for all $t,s$. The integral is then zero. Conversely, suppose the integral is zero. A nonnegative function with zero integral is zero almost everywhere, so $g(t,s)=0$ for almost every $(t,s)$ off the coordinate axes. If $g$ were nonzero at any point off those axes, continuity would make its modulus positive on a neighborhood of that point. The weight is positive there, giving a strictly positive integral, a contradiction. Thus $g=0$ off the axes. Continuity extends the equality to the axes as well. Therefore $\varphi_{XY}(t,s)=\varphi_X(t)\varphi_Y(s)$ for every $t,s$. Uniqueness of characteristic functions implies that the joint distribution is the product of the two marginal distributions, which is independence. Finally, when both marginal distance variances are positive, the defining normalization shows that $\mathcal R=0$ exactly when $\mathcal V^2=0$. $\square$

Worked Example: Independent Bernoulli Variables

Let $X$ and $Y$ be independent Bernoulli variables, each with success probability $1/2$. The joint probabilities are all $1/4$. With $r=1/2$, the centered-distance calculation for binary variables gives $\mathcal V^2(X,Y)=4\{\operatorname{Cov}(X,Y)\}^2$. Independence gives $E[XY]=1/4$, so $\operatorname{Cov}(X,Y)=1/4-(1/2)(1/2)=0$. Thus $\mathcal V^2(X,Y)=0$. Each marginal is nonconstant and has distance variance $4(1/4)^2=1/4$, so the denominator is positive and $\mathcal R(X,Y)=0$. Here the calculation agrees with the independence characterization; the theorem also covers nonbinary variables and dependence that a covariance calculation alone might not detect.

Interpretation and a Common Pitfall

Distance correlation is a measure of general dependence, not a signed measure of association. Its value is nonnegative, and it does not indicate whether larger values of one variable tend to accompany larger or smaller values of the other. Its zero value is informative under the stated moment conditions: for nonconstant variables it means independence, not merely zero covariance. The finite-first-moment assumption is important to the theorem and to the definition used here; it should not be silently replaced by a weaker condition.

Pitfall: treating the ratio as defined for every pair. If $X$ is constant, then $\mathcal V^2(X)=0$, making the denominator zero. One cannot conclude that the ratio is zero by substituting into the formula. Distance correlation is undefined in this case unless a separate convention is explicitly adopted.
Pitfall: interpreting the normalization as removing every transformation effect. Nonzero scalar changes of units, translations, and orthogonal coordinate changes preserve distance correlation. General linear transformations need not preserve Euclidean distances or the resulting value.
Pitfall: reading a positive value as a direction or causal effect. Distance correlation detects dependence, but does not by itself establish direction, causality, or the form of a relationship.

Key takeaway: Under finite first moments, distance covariance is zero exactly under independence. For nonconstant variables, normalizing by the marginal distance variances produces a scale-invariant distance correlation between zero and one.

Check Your Understanding

Answer each question using the definitions and conditions above.

What moment condition is used for the population independence characterization of distance covariance?
If $X$ and $Y$ are replaced by $aX+u$ and $bY+v$, with $a,b\ne0$, how does their squared distance covariance change, and what happens to distance correlation?
Why is distance correlation undefined when one variable is almost surely constant under the ratio definition?
What does zero distance covariance imply, and which property of characteristic functions is used to establish the implication?
Does a positive distance correlation identify the direction of association? Explain briefly.