Why Feature Count Changes the Weighting Question
When two omics blocks measure the same subjects, concatenating their features makes a combined distance possible. But the blocks may contain very different numbers of features. Even if their features have comparable scales, a block with more features can contribute more to a squared Euclidean distance simply because it has more terms. Conversely, a block with larger feature variances can dominate because its terms are larger. A weighting analysis makes these contributions explicit: it does not discover a uniquely correct weight, but shows what different weighting goals imply for the combined distances.
This tutorial compares three choices for two matched blocks: leave both unweighted, scale each to the same total Frobenius norm, or scale each to the same average feature variance. The calculations use a small hypothetical data set so that each pairwise distance can be checked directly.
The weight multiplies the feature values, so it is the squared weight that multiplies a squared-distance contribution. Confusing these two quantities leads to incorrect calculations. The unweighted choice is \(a_A=a_B=1\). Equal-total-norm weighting chooses weights to make the Frobenius norms of the weighted matrices equal. Equal-average-variance weighting chooses weights to make the average sample variance per feature equal across blocks.
Hypothetical Matched Data and Assumptions
Consider three hypothetical subjects measured on two features in block A and four features in block B. Each feature column has been centered across the three subjects, and there are no missing values. The values are illustrative; no real study is represented.
| Subject | Block A, feature 1 | Block A, feature 2 | Block B, feature 1 | Block B, feature 2 | Block B, feature 3 | Block B, feature 4 |
|---|---|---|---|---|---|---|
| 1 | -1 | 0 | -2 | 0 | 1 | -1 |
| 2 | 0 | 1 | 0 | 1 | -2 | -1 |
| 3 | 1 | -1 | 2 | -1 | 1 | 2 |
The main assumptions for these calculations are modest but important. Both matrices must refer to the same subjects in the same order, and each feature must be numeric on a scale for which Euclidean differences are meaningful. The columns are centered, so the Frobenius norm can be related directly to sample variances. The equal-average-variance calculation also requires nonzero average variance in each block. These weights describe the supplied data after its preprocessing; they do not correct for measurement error, batch effects, or differences in biological relevance.
Worked Example: Calculating Block Norms and Average Variances
For each block, add the squared entries to obtain its squared Frobenius norm, \(\lVert X^{(k)}\rVert_F^2\). Because the columns are centered and \(n=3\), each sample variance is the column sum of squares divided by \(n-1=2\).
Block A. Feature 1 has squared entries \(1,0,1\), summing to \(2\); feature 2 has squared entries \(0,1,1\), also summing to \(2\). Thus \( \lVert X^{(A)}\rVert_F^2=2+2=4 \), and \( \lVert X^{(A)}\rVert_F=\sqrt{4}=2 \). The two sample variances are \(2/2=1\) and \(2/2=1\), so the average feature variance is \((1+1)/2=1\).
Block B. Its four feature sums of squares are \(8,2,6,\) and \(6\), respectively. Therefore \( \lVert X^{(B)}\rVert_F^2=8+2+6+6=22 \), and \( \lVert X^{(B)}\rVert_F=\sqrt{22}\approx4.690 \). The sample variances are \(8/2=4\), \(2/2=1\), \(6/2=3\), and \(6/2=3\). Their average is \((4+1+3+3)/4=11/4=2.75\).
The blocks differ in both width and average variance: A has 2 features and average variance 1, while B has 4 features and average variance 2.75. Consequently, neither equal-total-norm nor equal-average-variance weighting will leave both blocks unchanged.
Three Weighting Choices
For equal-total-norm weighting, set \(a_k=1/\lVert X^{(k)}\rVert_F\). This makes each weighted block have Frobenius norm one. Any shared positive target norm could be used instead; it would multiply all blocks by the same constant and hence multiply every combined squared distance by that constant squared. Choosing one is a convenient convention.
For equal-average-variance weighting, let \(\bar{s}_k^2\) be the mean of the \(p_k\) sample variances in block \(k\), and set \(a_k=1/\sqrt{\bar{s}_k^2}\). Multiplication by this weight divides every feature variance in that block by \(\bar{s}_k^2\), so its average feature variance becomes one. Again, another common target is possible, but it would rescale all blocks together.
Worked Example: Computing the Three Sets of Weights
The unweighted multipliers are \(a_A=1\) and \(a_B=1\). For equal-total-norm weighting, the values from the previous calculation give \(a_A=1/2=0.5000\) and \(a_B=1/\sqrt{22}\approx0.2132\). The squared multipliers used on distance contributions are therefore \(0.25\) and \(1/22\approx0.04545\).
For equal-average-variance weighting, \(a_A=1/\sqrt{1}=1\) and \(a_B=1/\sqrt{2.75}\approx0.6030\). The corresponding squared multipliers are \(1\) and \(1/2.75\approx0.3636\). Thus equal-average-variance weighting leaves A at its original scale and reduces B, whereas equal-total-norm weighting reduces both blocks, especially B.
These weights are not probabilities and need not add to one. Their purpose is to set the relative scale of block contributions. A common rescaling of both weights changes the numerical distance scale but not which block contributes more relative to the other.
Deriving the Relationship Between the Weighting Targets
The distinction between the two normalization goals follows from the relationship between centered feature sums of squares and sample variance. For feature \(j\) in block \(k\), centering gives \(\sum_{i=1}^n (x_{ij}^{(k)})^2=(n-1)s_{kj}^2\). Summing this identity over the \(p_k\) features yields
Thus the squared Frobenius norm depends on both the number of features and their average variance. Equal-total-norm weighting compensates for both factors together. Equal-average-variance weighting compensates for average variance but leaves the feature-count factor in place: after weighting, total squared norm is proportional to \(p_k\). This is why equal average variance does not mean equal total block contribution when block widths differ.
A useful check comes from the identity for centered rows: \(\sum_{i<j}\lVert x_i-x_j\rVert^2=n\sum_i\lVert x_i-\bar{x}\rVert^2=n\lVert X\rVert_F^2\). For this three-subject example, it says that the sum of all unordered pairwise squared distances in a block should be three times its squared Frobenius norm. It provides a way to verify the distance arithmetic, not a separate weighting rule.
Worked Pairwise Distances and Their Interpretation
Worked Example: Comparing Combined Distances for All Subject Pairs
First calculate the block-specific squared distances directly from the table. In block A, the subject pairs \(1\)–\(2\), \(1\)–\(3\), and \(2\)–\(3\) have distances \(1^2+(-1)^2=2\), \((-2)^2+1^2=5\), and \((-1)^2+2^2=5\), respectively. Their total is \(2+5+5=12\), which agrees with \(3\lVert X^{(A)}\rVert_F^2=3(4)=12\).
In block B, the same pairs have squared distances \(2^2+(-1)^2+3^2+0^2=14\), \(4^2+1^2+0^2+3^2=26\), and \(2^2+2^2+(-3)^2+3^2=26\). Their total is \(14+26+26=66\), agreeing with \(3\lVert X^{(B)}\rVert_F^2=3(22)=66\).
Unweighted combination. Add the two block distances for each pair: \(1\)–\(2\) gives \(2+14=16\); \(1\)–\(3\) gives \(5+26=31\); and \(2\)–\(3\) also gives \(5+26=31\). The combined total across pairs is \(78\), of which A contributes 12 and B contributes 66.
Equal-total-norm combination. Multiply A’s distances by \(1/4\) and B’s by \(1/22\). The pair values are \(2/4+14/22=0.5+0.6364=1.1364\), \(5/4+26/22=1.25+1.1818=2.4318\), and \(5/4+26/22=2.4318\). The total contribution from each block across all three pairs is \(12/4=3\) and \(66/22=3\); the combined total is 6.
Equal-average-variance combination. Leave A’s distances unchanged and multiply B’s by \(1/2.75=4/11\). The pair values are \(2+14(4/11)=2+5.0909=7.0909\), \(5+26(4/11)=5+9.4545=14.4545\), and \(5+26(4/11)=14.4545\). Across all pairs, A contributes 12 and B contributes \(66(4/11)=24\), for a combined total of 36. B contributes twice as much total distance as A, matching the ratio of their feature counts, \(4:2\), after equalizing average variance.
The three distance scales are different because the weights are different; values from separate weighting schemes should not be compared as though they shared a common unit scale. Within each scheme, however, the relative block contributions are clear. The unweighted distance is dominated by B here. Equal-total-norm weighting gives both blocks the same aggregate contribution. Equal-average-variance weighting removes their average-variance difference but retains a larger aggregate contribution for the wider block.
Diagnostics, Alternatives, and Limitations
Several checks help detect a weighting calculation that has gone wrong. Confirm that rows represent the same subjects in the same order; otherwise, a combined distance compares mismatched observations. Recalculate the feature counts and the sum of squares from the retained, processed matrices, not from an earlier version of the data. If columns are not centered, the norm-to-variance identity above does not apply as stated. If a block has zero average variance, equal-average-variance weighting is undefined; a constant block contains no variation to use in this distance calculation. Extremely small average variance can produce a very large multiplier and should prompt an investigation of measurement quality and preprocessing.
The main alternatives answer different questions. Unweighted concatenation retains the processed units and scales as provided, but a block can dominate because of its width or feature variability. Equal-total-norm weighting sets each block’s overall observed squared magnitude to the same value, but it does not ensure equal influence for each feature or equal relevance to the scientific question. Equal-average-variance weighting equalizes a block-level average, but the wider block can still contribute more total squared distance. Feature-by-feature standardization is another possible choice, but it is not equivalent to either block-level method: it gives each feature a comparable variance scale and may amplify noisy or weakly measured features. Choice of method should follow the intended meaning of distance, and a sensitivity analysis across defensible weights can reveal whether conclusions depend on that choice.
These calculations are descriptive and cohort-dependent. Adding or removing subjects can change the norms and average variances, and filtering features can change both block width and weights. The weights do not account automatically for reliability, correlation among features, biological importance, or technical artifacts. Nor do the resulting distances establish a causal or inferential conclusion. They define a geometry for subsequent analysis, so conclusions based on that geometry should be reported with the weighting rule and preprocessing choices.
Check Your Understanding
Use the definitions and calculations above to answer the questions. Short solutions follow the questions.
- For a centered block with \(n=5\), \(p=3\), and average sample variance \(4\), what is its squared Frobenius norm?
- If two centered blocks have equal average feature variance but one has six features and the other has two, what is the ratio of their weighted squared Frobenius norms under equal-average-variance weighting?
- In the worked data, what is the equal-total-norm combined squared distance between subjects 1 and 3?
- Why does equal-total-norm weighting give equal aggregate pairwise contributions for these two blocks?
- Name one condition that makes equal-average-variance weighting undefined or potentially unstable.
Solutions. (1) The identity gives \((n-1)p\bar{s}^2=4(3)(4)=48\). (2) With equal average variance, squared norm is proportional to feature count, so the ratio is \(6:2=3:1\). (3) The value is \(5/4+26/22=1.25+1.1818=2.4318\). (4) Each block is scaled to Frobenius norm one, so its squared norm is one; for \(n\) shared centered rows, its sum of unordered pairwise squared distances is \(n\) times that norm squared. (5) A block with zero average variance has no finite inverse-square-root weight; a very small average variance can also produce an unstable, very large multiplier.