Why a Kernel Entry Is Not Just “Closeness”
Patient similarity matrices are useful because they summarize relationships among omics profiles in a form that can be compared or used in downstream analysis. But the word similarity can hide important differences. A raw distance measures separation in a specified data space; a graph affinity is a chosen edge weight; and a kernel entry, when it has a feature-space representation, is an inner product. These quantities need not have the same scale, sign, or interpretation.
This tutorial focuses on reading the entries of a kernel matrix geometrically. The central question is: what does a particular entry say about two patients, and what does it not say? The answer depends on the feature representation and on whether the matrix was actually constructed as inner products. A generic symmetric similarity matrix should not automatically be given that interpretation.
The feature space may be the original coordinate space or another space defined by the chosen representation. For this interpretation to apply, the matrix entries must have the stated inner-product representation. A matrix of pairwise scores labelled “similarity” is not thereby a kernel matrix in this geometric sense.
An inner product reflects both direction and length. If two feature vectors point in similar directions and have large lengths, their inner product can be large. If they are orthogonal, it is zero; if they point in opposing directions, it can be negative. Consequently, the diagonal is generally not zero, and off-diagonal entries need not be nonnegative or bounded by one. Those properties depend on the particular feature representation or on additional transformations.
Interpreting Entries and Relating Them to Distance
For two patients \(i\) and \(j\), \(K_{ij}\) alone does not give the full geometric relationship unless the feature-vector lengths are also considered. For example, a large inner product may result partly from one or both vectors having large norms. The diagonal entries provide those squared norms, and together with \(K_{ij}\) they determine the squared Euclidean distance between the feature representations.
This identity follows by expanding the inner product of the difference with itself: \(\langle a-b,a-b\rangle=\langle a,a\rangle+\langle b,b\rangle-2\langle a,b\rangle\). It gives a distance in the feature space, not necessarily the raw distance between the original omics profiles. In particular, it does not make \(K_{ij}\) itself a distance.
Cosine similarity is related but distinct. For two nonzero feature vectors, it divides their inner product by the product of their lengths. This removes the effect of overall vector magnitude, whereas the unnormalized kernel entry retains it. Cosine similarity is undefined if either vector has zero length.
Worked Example: Reading a Small Patient Kernel Matrix
Suppose, hypothetically, that three patients have feature representations \(\phi(x_A)=(2,0)\), \(\phi(x_B)=(1,2)\), and \(\phi(x_C)=(0,1)\). Assume the reported entries are ordinary Euclidean inner products of these vectors. Compute the kernel matrix and interpret selected entries.
The diagonal entries are squared lengths: \(K_{AA}=2^2+0^2=4\), \(K_{BB}=1^2+2^2=5\), and \(K_{CC}=0^2+1^2=1\). The off-diagonal entries are \(K_{AB}=2(1)+0(2)=2\), \(K_{AC}=2(0)+0(1)=0\), and \(K_{BC}=1(0)+2(1)=2\). Symmetry gives the remaining entries.
The value \(K_{AB}=2\) is the inner product of A’s and B’s feature vectors; it is not a distance of 2. Their feature-space distance is \(\sqrt{4+5-2(2)}=\sqrt{5}\approx2.236\). For A and C, the inner product is zero, so the vectors are orthogonal; their feature-space distance is \(\sqrt{4+1-0}=\sqrt{5}\approx2.236\). Orthogonality does not mean that the patients are identical or that they have no relationship in every possible sense.
Worked Comparisons: Magnitude, Direction, and Network Edges
Worked Example: Inner Product Versus Cosine Similarity
Consider two hypothetical feature vectors \(u=(3,0)\) and \(v=(-1,\sqrt{8})\). Both have length 3 because \(\|u\|=\sqrt{9}=3\) and \(\|v\|=\sqrt{1+8}=3\). Their inner product is \(u\cdot v=3(-1)+0(\sqrt{8})=-3\).
The cosine similarity is therefore \(-3/(3\cdot3)=-1/3\approx-0.3333\), not \(-1\). Their Euclidean distance is \(\sqrt{(3-(-1))^2+(0-\sqrt{8})^2}=\sqrt{16+8}=\sqrt{24}\approx4.899\). The kernel-based distance identity gives the same result: the two squared norms are 9 and the inner product is \(-3\), so the squared distance is \(9+9-2(-3)=24\).
The negative inner product indicates an obtuse angle between the vectors, not a negative distance. The cosine value reports the angle-related comparison after adjusting for length; the inner product retains the vector lengths as well.
Worked Example: From Kernel Values to a Graph Affinity
Return to the hypothetical matrix for A, B, and C. Suppose an analyst chooses the explicit graph rule “connect a pair when its off-diagonal kernel entry is at least 1.5, assign each retained edge weight 1, and set all diagonal entries to 0.” This is a possible graph-construction rule, not a property forced by the kernel.
The off-diagonal values are \(K_{AB}=2\), \(K_{AC}=0\), and \(K_{BC}=2\). The rule retains A–B and B–C, but not A–C. The resulting symmetric graph-affinity matrix is
Here \(K\) records feature-space inner products, including squared lengths on its diagonal; \(W\) records a particular thresholded network with binary edge weights and zero diagonal. The two matrices answer different questions. Another threshold, rescaling rule, or neighborhood rule could yield a different graph from the same kernel matrix.
The following comparisons help prevent a common category error. A raw distance \(d(x_i,x_j)\) is a measure of separation in the original profile space under a specified metric. A kernel entry is an inner product if a feature map exists as described above. A graph affinity \(W_{ij}\) is the edge weight assigned under a network-construction rule. The same pair of patients can therefore have three different numerical summaries without contradiction.
| Quantity | What it represents | Typical interpretation caution |
|---|---|---|
| Raw distance | Separation between original profiles under a chosen metric | Smaller values usually mean closer under that metric, but the scale depends on measurement and preprocessing. |
| Kernel entry \(K_{ij}\) | Inner product between feature representations, when the representation applies | Magnitude depends on vector lengths as well as directions; entries can be negative or greater than one. |
| Graph affinity \(W_{ij}\) | Weight assigned to the edge between two nodes by a graph rule | It may be thresholded, locally scaled, or otherwise transformed; it is not automatically an inner product. |
Assumptions, Diagnostics, and Failure Modes
Before interpreting a matrix geometrically, establish how its entries were produced. The key assumption is that \(K_{ij}\) equals an inner product in a shared feature space for all patient pairs. The representation must be the same for every pair; separately calculated pairwise scores with incompatible scales do not define the stated common geometry. If the construction is unknown, describe the entries as similarity scores and avoid claims about angles, norms, or feature-space distances.
Several checks can reveal an interpretation problem. First, verify symmetry: an inner product obeys \(K_{ij}=K_{ji}\). Second, check whether diagonal entries are consistent with squared feature-vector lengths and whether the construction intended a zero diagonal or a unit diagonal instead. Such conventions may signal a transformed similarity rather than an unnormalized inner-product matrix. Third, compare typical off-diagonal values with the diagonal: a large value may reflect patient-specific scale rather than unusually aligned profiles. Finally, inspect whether negative values are permitted by the chosen representation; they are valid inner products, but they may be incompatible with a downstream method that requires nonnegative edge weights.
A further diagnostic is to calculate a few feature-space distances using \(K_{ii}+K_{jj}-2K_{ij}\). Under the assumed inner-product representation, these quantities must be nonnegative because they equal squared lengths. A negative result indicates rounding or data-handling error, or that the matrix is not behaving as the assumed Gram matrix. This check does not, by itself, establish every property required by a later kernel method.
Why the Distinction Matters
Kernel matrices and patient networks are both pairwise summaries, but they preserve different information. A kernel entry retains a signed inner-product value and is interpreted relative to its feature space. A graph may discard diagonal entries, remove weak edges, rescale weights, or represent only selected neighbors. Those choices can be useful for network analysis, but they change the object being analyzed. Treating a thresholded graph weight as if it were the original kernel entry can lead to incorrect statements about feature-space geometry.
Likewise, replacing an inner product with cosine similarity changes the interpretation by removing dependence on vector length. That can be desirable when direction is the scientific target, but it is not merely a relabelling of the same quantity. In multi-omics data, vector magnitude may reflect meaningful signal, technical scale, or preprocessing choices; the scientific interpretation requires knowing which of these is plausible.
The practical sequence is to identify the matrix’s construction, state what one entry measures, then choose the comparison appropriate to the question. Use kernel entries to discuss feature-space inner products, the associated identity to discuss feature-space distances, and the specified graph rule to discuss network edges. Keep the units and scale in view: kernel entries have the product scale of the feature coordinates, whereas distances have the feature-coordinate scale and graph weights have the units or convention chosen by the graph construction.
Check Your Understanding
Answer each question using the distinctions above. Full solutions follow the questions.
- A kernel matrix has \(K_{ii}=16\), \(K_{jj}=9\), and \(K_{ij}=6\). What is the feature-space distance between patients \(i\) and \(j\)?
- Can a kernel entry be negative even though distances cannot be negative? Explain.
- If every feature vector is multiplied by 2, what happens to each inner-product kernel entry and to each feature-space distance?
- Using the matrix in the first worked example, apply the graph rule \(K_{ij}\geq1.5\) for off-diagonal pairs, with retained edges assigned weight 1. Which pairs are connected?
- If \(K_{ij}=0\) under an inner-product representation, what can be concluded, and what cannot be concluded?
Solutions. (1) The squared distance is \(16+9-2(6)=13\), so the distance is \(\sqrt{13}\approx3.606\) in feature-coordinate units. (2) Yes. An inner product can be negative when the vectors point in opposing directions; a distance is a nonnegative length. (3) Each kernel entry is multiplied by \(2^2=4\), since \(\langle2u,2v\rangle=4\langle u,v\rangle\). Each distance is multiplied by 2, since \(\|2u-2v\|=2\|u-v\|\). (4) A–B and B–C are connected; A–C is not. (5) The two feature vectors are orthogonal. This does not establish that the original omics profiles are identical, unrelated in every sense, or unconnected in a graph built using some other rule.