Kernel Choice and Bandwidth Sensitivity

Why Bandwidth Changes the View of Dependence

The Gaussian kernel compares observations according to their distance: nearby points receive a similarity close to one, while sufficiently separated points receive a similarity close to zero. Its bandwidth controls what counts as “nearby.” Since HSIC combines similarities in the two variables, changing either bandwidth changes which patterns of paired observations contribute most to the statistic.

A narrow bandwidth emphasizes very local matches; a wide bandwidth smooths over local structure and can make HSIC respond mainly to broad, nearly linear features. The value of HSIC therefore cannot be interpreted without its kernel and bandwidths. This is a sensitivity issue, not a change to the basic independence target: for Euclidean data, Gaussian kernels with any fixed, positive bandwidth are characteristic, so the population HSIC is zero exactly under independence. But finite samples can reveal markedly different evidence at different bandwidths.

The three terms involve different pairings of independent copies; the third pairs \(X\) with \(X'\), but \(Y\) with \(Y''\). Keeping that pairing straight is important in calculations. Equivalently, HSIC is the expectation of the product of the two population-centered kernels evaluated on two independent paired observations:

where \(\widetilde{k}(x,x')=k(x,x')-E[k(x,X)]-E[k(X,x')]+E[k(X,X')]\), and \(\widetilde{l}\) is defined analogously. These expressions make clear that bandwidth affects the similarities before they are centered and combined.

Use Distances to Set a Scale

A useful first diagnostic is to examine the distributions of pairwise distances in \(X\) and \(Y\), separately. In a sample, calculate distances between distinct observations, and inspect their median and perhaps lower and upper quantiles. A bandwidth much smaller than nearly all distances makes most off-diagonal similarities tiny; a bandwidth much larger than nearly all distances makes similarities similar to one. Either extreme can be uninformative for finite-sample comparisons.

A practical starting point is to consider bandwidths around a robust distance summary, such as the median pairwise distance, and then inspect a range below and above it. This is a scale diagnostic, not a universal optimality rule. If \(X\) and \(Y\) have different units or distance scales, choose and vary \(\sigma_X\) and \(\sigma_Y\) separately. Rescaling a variable changes its distances, so the corresponding bandwidth must be rescaled as well to preserve the same kernel similarities.

What the Large-Bandwidth Limit Emphasizes

When both bandwidths are large, a Gaussian kernel is nearly constant. Its constant part disappears under centering, and its first nonconstant contribution is governed by ordinary cross-covariance. The following approximation makes precise why a very wide kernel may be insensitive to nonlinear dependence that has no cross-covariance.

Proof. Write \(X'\) for an independent copy of \(X\). The pointwise limit \(\sigma^2(k_\sigma(x,x')-1)\to-\|x-x'\|^2/2\) follows from \((e^{-u}-1)/u\to-1\) as \(u\to0\). Moreover,

because \(1-e^{-u}\leq u\) for \(u\geq0\). The right-hand side is square-integrable by the finite fourth-moment assumption. Dominated convergence therefore gives convergence in \(L^2\) to \(-\|X-X'\|^2/2\). Population double-centering is a contraction in \(L^2\), so \(\sigma^2\widetilde{k}_\sigma(X,X')\) converges in \(L^2\) to the double-centered limit. Expanding the squared distance shows that this centered limit is \((X-E[X])^T(X'-E[X])\): terms depending on only one of \(X,X'\), and constants, vanish under double-centering. The same argument applies to \(Y\).

By the centered-kernel expression for HSIC and the Cauchy–Schwarz inequality, the product of these two \(L^2\)-convergent quantities has convergent expectation. Its limit is

The equality follows by expanding the product and using independence of the primed and unprimed paired observations. The sum is the squared Frobenius norm, proving the result. \(\square\)

What the Small-Bandwidth Limit Emphasizes

For continuous distributions, a very narrow Gaussian kernel makes similarities between distinct observations vanish. At the population level this can drive HSIC to zero, even under dependence. Thus narrowing the bandwidth does not increase sensitivity without limit. The precise limiting statement below applies when both marginal distributions have no atoms, meaning \(P(X=x)=P(Y=y)=0\) for every point \(x,y\).

Proof. For independent copies, \(k_{\sigma_X}(X,X')\to \mathbf{1}\{X=X'\}\) pointwise as \(\sigma_X\to0\). Nonatomicity implies \(P(X=X')=0\), so the limit is zero almost surely. Since the kernel is bounded by one, dominated convergence gives \(E[k(X,X')]\to0\). The same argument gives \(E[l(Y,Y')]\to0\). Also, the product \(k(X,X')l(Y,Y')\) tends to zero almost surely because the event \(X=X'\) has probability zero, and it is bounded by one; hence its expectation tends to zero. Finally, \(k(X,X')l(Y,Y'')\leq k(X,X')\), so its expectation tends to zero as well. All three terms in the HSIC formula therefore tend to zero. \(\square\)

This result is about the population quantity. In a finite sample, very narrow kernels can instead yield a Gram matrix close to the identity, leaving a statistic strongly affected by the diagonal and by sampling variation. Discrete distributions also behave differently: exact repeated values can retain nonzero similarity as the bandwidth shrinks.

Diagnosing Bandwidth Sensitivity in Practice

A useful diagnostic is a small sensitivity analysis rather than a single supposedly universal choice. First inspect pairwise-distance summaries separately for the two variables. Then calculate HSIC across a reasonable grid of bandwidth pairs, including values below and above the marginal distance summaries. Interpret the pattern, not merely the largest statistic: a sharp peak at a narrow scale, a gradual decline at wide scales, or a marked difference between the two margins may indicate which geometric features the kernel is emphasizing.

A bandwidth comparison does not by itself compare calibrated evidence against independence. HSIC values at different bandwidths have different scales, and any inferential comparison must account for the statistic’s null distribution and the procedure used to select or report bandwidths. The material on characteristic kernels establishes the population independence characterization for each fixed positive Gaussian bandwidth; it does not imply that all bandwidths have equal finite-sample sensitivity.

• Distances are scale guides, not optimality guarantees. A median can conceal clusters, tails, or a small but meaningful local scale.
• Check each variable separately. Different units and spread can make equal numerical bandwidths inappropriate.
• Look for nonlinear structure. A wide-bandwidth approximation driven by cross-covariance can attenuate relationships with zero cross-covariance.
• Account for ties and discreteness. The limiting behavior for nonatomic margins does not describe distributions with point masses.
• Separate magnitude from evidence. A change in the numerical HSIC value across bandwidths is not, on its own, a change in dependence or a calibrated change in significance.