Cosine Normalization of MNN Correction Vectors

Why Normalize MNN Correction Vectors?

A mutual nearest neighbor (MNN) pair identifies cells from different batches that are close to one another in a chosen feature space. The difference between their coordinates supplies a candidate direction for alignment. But the raw length of that difference can reflect more than the batch displacement: it may also depend on local cell density, measurement noise, or the scale of the feature space. Cosine normalization separates the direction of a candidate correction from its raw magnitude and assigns the vector a length based on a local neighborhood scale.

This is a geometric rescaling, not a test that an MNN pair is biologically valid. It does not establish that the two cells represent the same state, and it does not remove the need to inspect match quality. The calculation is meaningful only after the embedding and the MNN pair have been specified.

Definition: Let $x_a$ be the coordinate of a cell $a$ in a reference batch and $x_b$ the coordinate of its MNN partner $b$ in a target batch, both in the same $q$-dimensional feature space. Define the raw correction vector for $b$, under the convention that it is moved toward $a$, as $v_b=x_a-x_b$. Let $d_{b,1},\ldots,d_{b,k}$ be the Euclidean distances from $b$ to its $k$ selected neighbors in its own batch, and let $r_b$ be their median. If $\lVert v_b\rVert>0$ and $r_b>0$, the cosine-normalized correction is $$ \widetilde v_b = \frac{v_b}{\lVert v_b\rVert}r_b. $$ Thus $\widetilde v_b$ has the direction of $v_b$ and the magnitude $r_b$. The corresponding corrected coordinate, if this vector is applied, is $x_b+\widetilde v_b$.

The local scale here is explicitly the median distance from the target cell to its same-batch neighbors. Implementations can differ in their exact neighborhood or scale convention, so those choices must be documented. For the definition above, all distances and vector coordinates must use the same embedding and distance metric. The median is less sensitive than a mean to one unusually distant neighbor, but it is not immune to an unrepresentative neighborhood.

What the Rescaling Does

For a nonzero vector, dividing by its Euclidean norm produces a unit vector. Multiplying that unit vector by $r_b$ then assigns the vector the local length. The direction is unchanged: cosine normalization cannot rotate a correction, reverse it, or repair an incorrect MNN match. It changes only the length.

\lVert \widetilde v_b\rVert = \left\lVert \frac{v_b}{\lVert v_b\rVert}r_b\right\rVert = \frac{\lVert v_b\rVert}{\lVert v_b\rVert}r_b = r_b.

For any nonzero direction $u$, the cosine similarity between $v_b$ and $\widetilde v_b$ is one:

\frac{v_b^\mathsf{T}\widetilde v_b} {\lVert v_b\rVert\lVert\widetilde v_b\rVert} = \frac{v_b^\mathsf{T}(v_b r_b/\lVert v_b\rVert)} {\lVert v_b\rVert r_b} =1.

The formula also makes the change in magnitude explicit. The raw length is $\lVert v_b\rVert$; the normalized length is $r_b$. Normalization increases the length if $r_b>\lVert v_b\rVert$, decreases it if $r_b<\lVert v_b\rVert$, and leaves it unchanged if the two lengths are equal. Therefore “normalization” does not always mean shrinking a vector.

Worked Example: Rescaling a Two-Dimensional Correction

Suppose a hypothetical target cell has coordinates $x_b=(2,1)$, and its MNN partner in the reference batch has coordinates $x_a=(5,5)$. Its distances to four same-batch neighbors are $1.2, 1.6, 2.0,$ and $2.4$ embedding units. Calculate the raw and normalized corrections.

The raw vector and its length are $$ v_b=x_a-x_b=(5-2,5-1)=(3,4), \qquad \lVert v_b\rVert=\sqrt{3^2+4^2}=5. $$

The median of the four neighbor distances is the average of the two middle values: $$ r_b=\frac{1.6+2.0}{2}=1.8. $$

The unit direction is $v_b/\lVert v_b\rVert=(3/5,4/5)=(0.6,0.8)$. Hence $$ \widetilde v_b=1.8(0.6,0.8)=(1.08,1.44). $$

Its length is $\sqrt{1.08^2+1.44^2}=1.8$, as required. The raw correction had length 5 embedding units, while the normalized correction has length 1.8 embedding units. If applied, the corrected coordinate is $(2,1)+(1.08,1.44)=(3.08,2.44)$. The correction points in the same direction as before, but is shorter.

Local Scale Determines the New Magnitude

The purpose of using $r_b$ is to make the assigned magnitude depend on the target cell’s local geometry rather than the length of its MNN difference alone. A cell in a locally dispersed region can have a larger neighborhood-distance scale than a cell in a dense region. Two raw MNN vectors of equal length can therefore receive different normalized lengths; likewise, raw vectors of different lengths can receive similar normalized lengths if their local scales are similar.

Worked Example: Different Raw Shifts and Different Local Scales

Consider two hypothetical target cells. For cell 1, the raw correction is $v_1=(6,8)$, and the median same-batch neighbor distance is $r_1=2.5$. For cell 2, the raw correction is $v_2=(0,3)$, and the corresponding median is $r_2=1.5$. Calculate both normalized vectors and compare their lengths.

For cell 1, $$ \lVert v_1\rVert=\sqrt{6^2+8^2}=10,\qquad \frac{v_1}{\lVert v_1\rVert}=(0.6,0.8), $$ so $$ \widetilde v_1=2.5(0.6,0.8)=(1.5,2.0), \qquad \lVert\widetilde v_1\rVert=2.5. $$

For cell 2, $$ \lVert v_2\rVert=3,\qquad \frac{v_2}{\lVert v_2\rVert}=(0,1), $$ so $$ \widetilde v_2=1.5(0,1)=(0,1.5), \qquad \lVert\widetilde v_2\rVert=1.5. $$

The raw lengths were 10 and 3 embedding units; the normalized lengths are 2.5 and 1.5 embedding units because those are the two local scales. Each direction is preserved. The second correction is shorter after normalization even though its raw length was already much smaller than the first.

This example illustrates the scale dependence rather than demonstrating that either magnitude is biologically correct. If the local distances are inflated by noise or distorted by inappropriate features, the assigned correction lengths inherit that problem.

Assumptions and a Geometric Justification

Cosine normalization is a deterministic transformation, but its interpretation relies on assumptions about the geometry. The two batches must have coordinates in a common feature space with comparable units; otherwise the vector difference and its norm have no consistent meaning. The chosen distance metric must be appropriate for that space. The MNN pair should plausibly link cells in a shared state, as required by the shared-state premise of MNN alignment. Finally, the target cell’s same-batch neighbors must provide a meaningful local scale.

Why use the local distance as the target length? The underlying geometric choice is to preserve the estimated direction while replacing an unstable raw norm with a local reference length. Algebraically, every nonzero vector can be written as its length times its unit direction:

v_b=\lVert v_b\rVert u_b,\qquad u_b=\frac{v_b}{\lVert v_b\rVert},\qquad \lVert u_b\rVert=1.

Replacing $\lVert v_b\rVert$ by $r_b$ gives $\widetilde v_b=r_bu_b$. This is the unique vector on the ray from the origin in direction $v_b$ whose length is $r_b$. The derivation establishes the geometric properties; it does not prove that $r_b$ is the correct biological correction magnitude. That choice is an algorithmic assumption that must be assessed in context.

Worked Example: Verifying Direction and Magnitude

A hypothetical correction vector is $v=(2,-1,2)$, and the local scale is $r=3$ embedding units. Verify the normalized vector’s magnitude and alignment with the original direction.

The raw norm is $$ \lVert v\rVert=\sqrt{2^2+(-1)^2+2^2}=3, $$ so the unit direction is $u=(2/3,-1/3,2/3)$. Therefore $$ \widetilde v=3u=(2,-1,2). $$

In this case normalization leaves the vector unchanged because its raw length already equals the local scale. The normalized norm is $\sqrt{2^2+(-1)^2+2^2}=3=r$, and the cosine similarity is $$ \frac{v^\mathsf{T}\widetilde v}{\lVert v\rVert\lVert\widetilde v\rVert} =\frac{9}{3\cdot3}=1. $$

Diagnostics, Failure Modes, and Alternatives

Before interpreting normalized vectors, inspect both the directions and the local scales. A length rescaling cannot correct a direction that points away from the intended alignment, and a plausible-looking length does not validate the MNN match. Useful checks include plotting vectors in the selected embedding, examining the distribution of raw norms and local scales, and repeating the calculation under reasonable choices of neighborhood size or features.

Zero raw vector. If $\lVert v_b\rVert=0$, the unit direction is undefined; division by zero is not allowed. Such a pair has identical coordinates in the selected space, so its direction cannot be recovered from this formula. Handle it explicitly, for example by recording a zero correction rather than pretending a direction exists.
Zero or near-zero local scale. If $r_b=0$, the normalized vector is zero when the raw vector is nonzero. Near-zero scales can produce negligible corrections and may signal duplicate coordinates, a very small neighborhood, or a degenerate feature space. Check the data and neighborhood construction rather than adding an arbitrary positive constant without disclosure.
Unstable neighborhoods. If small changes in $k$ or the feature representation produce large changes in $r_b$, the assigned magnitudes are not robust. Sparse or unevenly sampled regions are especially important to inspect.
Unreliable direction or match. If an MNN pair connects distinct biological states, cosine normalization preserves the resulting wrong direction. The shared-state assumption and match quality require separate evaluation.
Incomparable scales. Distances computed in differently scaled coordinates, or under inconsistent metrics, cannot be used as a coherent magnitude reference. State the feature space, metric, and neighbor rule.

Raw vectors retain the MNN pair’s observed displacement, but their lengths may reflect local scale and noise. Cosine-normalized vectors retain the observed direction while substituting a cell-specific length. A single global length would be simpler, but would assign the same magnitude across dense and sparse regions. None of these choices is automatically superior: local normalization is useful only when local neighbor distances are a defensible scale for the correction being applied.

A common mistake is to say, “Cosine normalization makes every correction a unit vector.” That is true only for the intermediate direction $v_b/\lVert v_b\rVert$. Under the definition here, the final normalized vector has length $r_b$, not necessarily one. Another mistake is to interpret a larger normalized vector as stronger evidence of a batch effect. Its length is set by local geometry, not by a confidence measure or a test statistic.

Key takeaway: Cosine normalization replaces the raw MNN correction length with a chosen local scale while preserving the correction direction. It is well-defined only for a nonzero raw vector, and its usefulness depends on whether the selected neighborhood scale and MNN direction are credible in the chosen embedding.

Check Your Understanding

Work through each item, then compare with the solutions below.

A raw vector is $(5,12)$, and the local scale is 2 embedding units. Find the normalized vector and its length.
A raw vector has length 4, while its target cell’s local scale is 6. Does normalization increase or decrease its magnitude? What happens to its direction?
Why is cosine normalization undefined for a zero raw vector?
Two cells have the same raw correction length but different local scales. Can their normalized correction lengths differ? Explain.
Give one reason a normalized correction could still be misleading even when its arithmetic is correct.

Solutions. 1. The raw norm is $\sqrt{5^2+12^2}=13$, so the unit vector is $(5/13,12/13)$. The normalized vector is $2(5/13,12/13)=(10/13,24/13)$, with length 2 embedding units. 2. Its magnitude increases from 4 to 6 embedding units; its direction is unchanged. 3. The formula divides by the raw norm, and $0/\lVert 0\rVert$ requires division by zero. The zero vector has no defined direction to preserve. 4. Yes. Each normalized length equals its own local scale, so different local scales give different normalized lengths even if the raw lengths match. 5. For example, the MNN pair may link different biological states, giving an unreliable direction; normalization changes magnitude only and cannot repair that match.