Characteristic-Function Distance for Independence

From Independence to a Characteristic-Function Distance

Correlation describes a particular kind of association, but dependence need not appear as nonzero correlation. A more general way to express independence is to compare the joint distribution with the product of its marginal distributions. Characteristic functions make this comparison precise: independence holds exactly when the joint characteristic function factors into the two marginal characteristic functions at every pair of arguments.

This tutorial turns that factorization condition into a nonnegative numerical discrepancy by integrating its squared magnitude over all arguments. The integration weight matters: it must give a finite integral, and to detect every failure of independence it must assign positive weight throughout the argument space.

Characteristic functions are complex-valued, so the size of the discrepancy is measured by its complex modulus. The product \(\varphi_X(t)\varphi_Y(s)\) is the characteristic function of the product distribution \(F_X\otimes F_Y\), the distribution that makes \(X\) and \(Y\) independent while retaining their actual marginal distributions.

This is a family of distances, one for each admissible weight, rather than a single weight-independent number. The Gaussian choice is convenient because it is strictly positive everywhere and integrates to one. Since every characteristic function has modulus at most one, \(|\Delta(t,s)|\leq 2\), so \(D_w(X,Y)\leq 4\int w(t,s)\,ds\,dt<\infty\). No moment assumption on \(X\) or \(Y\) is needed for this definition or bound.

Why Factorization Characterizes Independence

The key fact is that a probability distribution is uniquely determined by its characteristic function. In particular, if two probability distributions on the same Euclidean space have equal characteristic functions at every argument, then they are the same distribution. We use this standard uniqueness theorem to connect factorization with independence.

Proof. If \(X\) and \(Y\) are independent, then the expectation of the product of the two bounded complex-valued functions factors: $$ \varphi_{X,Y}(t,s) =E[\exp(i t^\mathsf{T}X)\exp(i s^\mathsf{T}Y)] =E[\exp(i t^\mathsf{T}X)]E[\exp(i s^\mathsf{T}Y)] =\varphi_X(t)\varphi_Y(s). $$ Thus \(\Delta(t,s)=0\) everywhere.

Conversely, suppose \(\Delta(t,s)=0\) for every \((t,s)\). The joint distribution of \((X,Y)\) then has characteristic function \(\varphi_X(t)\varphi_Y(s)\). The product distribution \(F_X\otimes F_Y\) has that same characteristic function, by the definition of a product distribution. Uniqueness of characteristic functions implies that the joint distribution equals \(F_X\otimes F_Y\). This equality is precisely independence.

It remains to connect the integral to pointwise factorization. If \(\Delta\) vanishes everywhere, then its squared modulus is zero everywhere, so \(D_w(X,Y)=0\). In the other direction, suppose \(D_w(X,Y)=0\). The integrand is nonnegative, so \(|\Delta(t,s)|^2w(t,s)=0\) almost everywhere. Characteristic functions are continuous, hence \(\Delta\) is continuous. If \(\Delta\) were nonzero at some point, continuity would give an open ball around that point on which \(|\Delta|\) is bounded below by some positive number. Since \(w>0\) almost everywhere, its integral over this ball is positive; consequently the integral defining \(D_w\) would be positive, a contradiction. Therefore \(\Delta=0\) everywhere, and the first part of the proof gives independence. \(\square\)

Worked Calculations

Interpreting the Distance and Avoiding Pitfalls

The distance summarizes discrepancies across all frequencies \((t,s)\), rather than checking only a local feature such as covariance. The theorem shows that with an integrable weight positive almost everywhere, no nonzero factorization discrepancy can be hidden from the integral. A weight that is zero on a region of positive measure may lose this guarantee: it could ignore discrepancies confined to that region.

The numerical size of \(D_w\) is not an intrinsic scale of dependence. Changing the weight changes which arguments receive emphasis and can change the value, even though the zero-versus-positive characterization remains the same for every admissible weight. In particular, interpreting two distances computed with different weights as directly comparable requires care. The Gaussian weight provides a simple bounded construction; other weights can be chosen for particular analytical purposes, provided their integrability and positivity properties are checked.

A common mistake is to reason that zero covariance proves independence. The \(Y=X^2\) example disproves that inference: covariance is zero, but the joint characteristic function fails to factor. Another mistake is to conclude that a single nonzero value of \(\Delta(t,s)\) directly gives the value of the integrated distance. It proves that the distance is positive when the weight is positive almost everywhere, but evaluating its numerical value still requires integrating the squared discrepancy over the full argument space.