Empirical Characteristic-Function Independence Statistics

From a Population Discrepancy to a Sample Statistic

A characteristic-function independence discrepancy compares the joint characteristic function of $X$ and $Y$ with the product of their marginal characteristic functions. In data analysis, the distributions are unknown, so the natural construction is to replace each population characteristic function by its empirical counterpart. The result is a nonnegative sample statistic whose value depends not only on the observed pairs but also on which frequencies receive weight.

Let $X$ take values in $\mathbb{R}^p$ and $Y$ in $\mathbb{R}^q$. For a paired sample $(X_1,Y_1),\ldots,(X_n,Y_n)$, write $t\in\mathbb{R}^p$, $s\in\mathbb{R}^q$, and $u=(t,s)$. The population characteristic functions are

\varphi_{XY}(t,s)=E\!\left[e^{i(t^\top X+s^\top Y)}\right],\qquad \varphi_X(t)=E\!\left[e^{it^\top X}\right],\qquad \varphi_Y(s)=E\!\left[e^{is^\top Y}\right].

The population discrepancy at frequency $(t,s)$ is $\varphi_{XY}(t,s)-\varphi_X(t)\varphi_Y(s)$. It is zero at every frequency under independence. To make an overall discrepancy, the squared modulus is integrated against a nonnegative weight.

Definition: Given a measurable weight $w:\mathbb{R}^{p+q}\to[0,\infty)$, define the weighted population independence discrepancy by $$ \mathcal{D}_w=\int_{\mathbb{R}^{p+q}} \left|\varphi_{XY}(t,s)-\varphi_X(t)\varphi_Y(s)\right|^2 w(t,s)\,dt\,ds, $$ whenever the integral exists. The empirical characteristic functions are $$ \widehat\varphi_{XY}(t,s)=\frac1n\sum_{j=1}^n e^{i(t^\top X_j+s^\top Y_j)},\quad \widehat\varphi_X(t)=\frac1n\sum_{j=1}^n e^{it^\top X_j},\quad \widehat\varphi_Y(s)=\frac1n\sum_{j=1}^n e^{is^\top Y_j}. $$ For an integrable weight $w$, the empirical discrepancy and its commonly scaled statistic are $$ \widehat{\mathcal{D}}_w=\int \left|\widehat\varphi_{XY}(t,s)-\widehat\varphi_X(t)\widehat\varphi_Y(s)\right|^2w(t,s)\,dt\,ds, \qquad T_n=n\widehat{\mathcal{D}}_w. $$

The product of the two empirical marginal characteristic functions is important: it is not the empirical joint characteristic function, and it is not obtained by multiplying each observed pair's two contributions before averaging. It corresponds to combining an observed $X$-value and an observed $Y$-value across all possible sample indices.

Worked Examples: Substituting the Empirical Functions

Worked Example: A Two-Observation Empirical Discrepancy

Take the paired observations $(X_1,Y_1)=(0,0)$ and $(X_2,Y_2)=(1,1)$. In one dimension for each variable, $$ \widehat\varphi_{XY}(t,s)=\frac{1+e^{i(t+s)}}2,\quad \widehat\varphi_X(t)=\frac{1+e^{it}}2,\quad \widehat\varphi_Y(s)=\frac{1+e^{is}}2. $$

Multiplying the marginal functions and subtracting gives $$ \widehat\varphi_{XY}(t,s)-\widehat\varphi_X(t)\widehat\varphi_Y(s) =\frac14\left(1-e^{it}-e^{is}+e^{i(t+s)}\right) =\frac14(1-e^{it})(1-e^{is}). $$ Since $|1-e^{it}|^2=4\sin^2(t/2)$, the squared modulus is $$ \left|\widehat\varphi_{XY}(t,s)-\widehat\varphi_X(t)\widehat\varphi_Y(s)\right|^2 =\sin^2(t/2)\sin^2(s/2). $$ At $(t,s)=(\pi,\pi)$, this contribution is $1$. At either axis, where $t=0$ or $s=0$, it is zero. The statistic integrates these contributions over frequencies rather than relying on one point.

What the Weight Does

A weight controls how much each frequency contributes to the integral. A weight concentrated near the origin emphasizes low-frequency differences; one that places more mass farther from the origin gives greater influence to finer-scale oscillations. A nonnegative integrable weight ensures the empirical statistic is finite: the difference of two characteristic functions has modulus at most $2$, so its squared modulus is at most $4$, and therefore $\widehat{\mathcal{D}}_w\leq 4\int w$.

One useful choice is a normalized Gaussian frequency weight. For a scale parameter $a>0$, let $d=p+q$ and set

w_a(u)=\left(\frac{a}{\sqrt{2\pi}}\right)^d \exp\!\left(-\frac{a^2\|u\|^2}{2}\right),\qquad u\in\mathbb{R}^d.

This integrates to one. As a density over frequencies, it is a centered Gaussian with covariance matrix $a^{-2}I_d$. Thus smaller $a$ gives a broader frequency distribution, while larger $a$ concentrates the weight closer to zero. Normalization separates this change in frequency emphasis from a simple change in the total mass of the weight.

Worked Example: Integrating with a Gaussian Frequency Weight

Continue with the two observations $(0,0)$ and $(1,1)$, and use $w_a(t,s)=\frac{a^2}{2\pi}\exp[-a^2(t^2+s^2)/2]$. The empirical squared discrepancy factors into one function of $t$ and one of $s$, and the Gaussian weight does too. Hence $$ \widehat{\mathcal{D}}_{w_a} =\frac{a^2}{2\pi} \left(\int_{\mathbb{R}}\sin^2(t/2)e^{-a^2t^2/2}\,dt\right)^2. $$

Use $\sin^2(t/2)=(1-\cos t)/2$, together with the Gaussian Fourier integral $$ \int_{\mathbb{R}}e^{-a^2t^2/2}\,dt=\frac{\sqrt{2\pi}}a,\qquad \int_{\mathbb{R}}\cos(t)e^{-a^2t^2/2}\,dt =\frac{\sqrt{2\pi}}a e^{-1/(2a^2)}. $$ It follows that the one-dimensional integral is $\frac{\sqrt{2\pi}}{2a}(1-e^{-1/(2a^2)})$. Substitution and simplification give $$ \widehat{\mathcal{D}}_{w_a} =\frac14\left(1-e^{-1/(2a^2)}\right)^2,\qquad T_n=2\widehat{\mathcal{D}}_{w_a} =\frac12\left(1-e^{-1/(2a^2)}\right)^2. $$ For $a=1$, $T_n=\frac12(1-e^{-1/2})^2\approx0.0774$. For $a=0.5$, $T_n=\frac12(1-e^{-2})^2\approx0.3738$. The broader frequency weight in the second calculation assigns relatively more weight away from the origin, where this sample's discrepancy is larger.

Finite-Sample Results

The empirical statistic is a finite-sample quantity. Its nonnegativity follows directly from its definition, but the integral also has a useful finite-sum representation. This representation makes clear how a chosen frequency weight induces a function of differences between observations.

Theorem: Suppose $w$ is nonnegative and integrable on $\mathbb{R}^{p+q}$, and define the finite signed measure $$ \Delta_n=\frac1n\sum_{i=1}^n\delta_{(X_i,Y_i)} -\frac1{n^2}\sum_{j=1}^n\sum_{k=1}^n\delta_{(X_j,Y_k)}. $$ Then $$ \widehat\varphi_{XY}(t,s)-\widehat\varphi_X(t)\widehat\varphi_Y(s) =\int e^{i(t^\top x+s^\top y)}\,\Delta_n(d x,d y), $$ and $$ \widehat{\mathcal{D}}_w =\int\!\!\int K_w(z-z')\,\Delta_n(dz)\Delta_n(dz'),\qquad K_w(h)=\int_{\mathbb{R}^{p+q}}e^{iu^\top h}w(u)\,du. $$

Proof. Integrating the exponential against the first sum of point masses in $\Delta_n$ gives $n^{-1}\sum_i e^{i(t^\top X_i+s^\top Y_i)}=\widehat\varphi_{XY}(t,s)$. Integrating against the second gives $$ \frac1{n^2}\sum_{j,k}e^{i(t^\top X_j+s^\top Y_k)} =\left(\frac1n\sum_j e^{it^\top X_j}\right) \left(\frac1n\sum_k e^{is^\top Y_k}\right) =\widehat\varphi_X(t)\widehat\varphi_Y(s). $$ This proves the first identity. Write $u=(t,s)$ and $z=(x,y)$. Since $\Delta_n$ is real and finite, the squared modulus of its transform is $$ \left|\int e^{iu^\top z}\Delta_n(dz)\right|^2 =\int\!\!\int e^{iu^\top(z-z')}\Delta_n(dz)\Delta_n(dz'). $$ The total variation of $\Delta_n$ is finite, and $\int w(u)\,du<\infty$. Therefore the absolute value of the integrand integrated against $w(u)\,du\,|\Delta_n|(dz)|\Delta_n|(dz')$ has finite integral, bounded by $\|w\|_1|\Delta_n|(\mathbb{R}^{p+q})^2$. Fubini's theorem permits exchanging the integrations. Integrating the exponential against $w$ then yields $K_w(z-z')$, proving the second identity. $\square$

If $w(-u)=w(u)$, the kernel $K_w$ is real-valued: the sine part of its defining integral cancels under $u\mapsto -u$. For the normalized Gaussian weight above, the Gaussian Fourier integral gives

K_{w_a}(h)=\exp\!\left(-\frac{\|h\|^2}{2a^2}\right).

The theorem therefore gives a direct finite-sum route to the Gaussian-weighted statistic: expand $\Delta_n$ into its point masses and evaluate the resulting kernel at their pairwise differences. The signed measure contains both the observed pairs and all cross-combinations of observed marginal values.

Theorem: Suppose $w$ is nonnegative, integrable, and positive Lebesgue-almost everywhere. Then $\widehat{\mathcal{D}}_w=0$ if and only if the empirical joint distribution is the product of the two empirical marginal distributions.

Proof. Let $D_n(t,s)=\widehat\varphi_{XY}(t,s)-\widehat\varphi_X(t)\widehat\varphi_Y(s)$. If $\widehat{\mathcal{D}}_w=0$, the nonnegative function $|D_n|^2w$ has integral zero, so it is zero almost everywhere. Because $w>0$ almost everywhere, $D_n=0$ almost everywhere. Each empirical characteristic function is continuous, so $D_n$ is continuous. If $D_n$ were nonzero at any point, continuity would make it nonzero on an open neighborhood, which has positive Lebesgue measure; this contradicts its being zero almost everywhere. Thus $D_n=0$ everywhere. The characteristic-function uniqueness theorem now implies that the empirical joint probability measure equals the product of the empirical marginal probability measures. Conversely, if those two measures are equal, their characteristic functions agree everywhere, so $D_n=0$ everywhere and the integral is zero. $\square$

This is a statement about the empirical distributions, not a claim that the population variables are independent. It also relies on the weight being positive almost everywhere. If a weight excludes a region of frequencies, equality of the transforms only where the weight is positive need not force equality at every frequency.

Worked Example: A Frequency-Specific Contribution

Worked Example: Comparing Weight at a Higher Frequency

Consider the three observed pairs $(0,0),(1,1),(0,1)$. At $t=s=\pi$, the joint empirical characteristic function is $$ \widehat\varphi_{XY}(\pi,\pi)=\frac{1+1-1}{3}=\frac13. $$ The marginal functions are $\widehat\varphi_X(\pi)=(1-1+1)/3=1/3$ and $\widehat\varphi_Y(\pi)=(1-1-1)/3=-1/3$. Consequently, $$ D_n(\pi,\pi)=\frac13-\left(\frac13\right)\left(-\frac13\right)=\frac49,\qquad |D_n(\pi,\pi)|^2=\frac{16}{81}. $$

For the normalized Gaussian weight in two frequency dimensions, its density at this point is $$ w_a(\pi,\pi)=\frac{a^2}{2\pi}e^{-a^2\pi^2}. $$ The weighted integrand there is therefore $\frac{16}{81}\frac{a^2}{2\pi}e^{-a^2\pi^2}$. For $a=1$, its weight factor is $e^{-\pi^2}/(2\pi)$; for $a=0.5$, it is $e^{-\pi^2/4}/(8\pi)$. The ratio of the first factor to the second is $4e^{-3\pi^2/4}$, approximately $0.00244$. Thus the $a=1$ weight puts far less density at this higher-frequency point than the broader $a=0.5$ weight. A value at one point has zero effect on an integral by itself, but this comparison illustrates how the weight scales contributions in its surrounding frequency region.

Why Weight Choice Matters

The weight is part of the statistic's definition, not a cosmetic multiplier. Changing it changes which features of the empirical characteristic-function difference drive the result. A broad weight can retain contributions over a wider range of frequencies; a concentrated weight focuses attention near the origin. In applied work, the selected scale should be reported because statistics computed with different weights are not interchangeable merely because they use the same sample.

Check integrability. The finite-sample definition above uses an integrable weight, which guarantees a finite statistic without additional tail conditions.
Check where the weight is positive. Positivity almost everywhere is the condition used in the zero-statistic characterization.
Keep scale and normalization distinct. An unnormalized weight can change the overall magnitude simply by changing its total mass, as well as changing frequency emphasis.

A common mistake is to evaluate the discrepancy at the origin and conclude that it is zero, hence that the sample shows no dependence. In fact, for every sample, $\widehat\varphi_{XY}(0,0)=\widehat\varphi_X(0)\widehat\varphi_Y(0)=1$; the discrepancy is automatically zero there, whether or not the paired data show dependence. The correction is to integrate the squared discrepancy over the frequencies specified by the weight. Checking only a few selected frequencies is useful for illustration, but it is not a substitute for the defined integral.

Key takeaway: The empirical independence statistic is the weighted integral of the squared difference between the joint empirical characteristic function and the product of the marginal empirical characteristic functions. Its weight determines how frequency-specific discrepancies are combined.

Check Your Understanding

Use the definitions and derivations above to answer these questions.

Why does the product $\widehat\varphi_X(t)\widehat\varphi_Y(s)$ contain a sum over all cross-pairs of sample indices?
For what reason is $\widehat{\mathcal{D}}_w$ nonnegative, and which simple condition on $w$ ensures it is finite?
What frequency range receives more relative emphasis when $a$ decreases in the normalized Gaussian weight $w_a$?
State the additional condition on $w$ used to conclude that a zero empirical discrepancy means equality of the empirical joint and product distributions.
Why does a zero discrepancy at $(t,s)=(0,0)$ provide no evidence of independence?