How Feature Filtering Changes Block Weights

Why filtering can change a block’s influence

A multi-omics block may contain thousands of measured features, but preprocessing often removes features that are uninformative, unreliable, or unsuitable for analysis. This changes the data matrix being integrated. If a block’s weight was chosen using its pre-filtering variance or norm, that weight may no longer achieve its original purpose after features are removed. The size of the change depends on which features were removed and on whether the retained features were re-scaled.

The central question is not simply whether a weight should change. It is whether the reason for choosing that weight still applies to the filtered matrix. For a weight intended to equalize block norms, the norm must be recalculated after filtering. A weight chosen for reliability, interpretability, or another objective does not automatically need an inverse-norm update; its objective should be checked instead.

Variance and norm before and after filtering

Let \(X_b\) be the matrix for block \(b\), with \(n\) subjects in rows and \(p_b\) features in columns. For the formulas below, assume the block uses the same subjects before and after filtering, has no missing entries, and each feature column has been centered across those subjects. Let \(s_{bj}^2\) denote the sample variance of feature \(j\) in block \(b\). Define the block’s summed feature variance as the sum of its marginal sample variances.

Definition: The summed feature variance of block \(b\) is \(V_b=\sum_{j=1}^{p_b}s_{bj}^2\). Its Frobenius norm is \(\|X_b\|_F=\sqrt{\sum_{i=1}^{n}\sum_{j=1}^{p_b}x_{bij}^2}\). For centered columns, \(\|X_b\|_F^2=(n-1)V_b\).

Suppose filtering retains a set \(R_b\) of features. If the retained columns are otherwise unchanged, the filtered summed variance and squared norm are

V_{b,\mathrm{ret}}=\sum_{j\in R_b}s_{bj}^2 =V_b-\sum_{j\notin R_b}s_{bj}^2, \qquad \|X_{b,\mathrm{ret}}\|_F^2=(n-1)V_{b,\mathrm{ret}}.

Thus, removing a feature with positive variance decreases the block’s summed variance and norm. How much they decrease depends on the removed features’ variances, not only on how many features were removed. Removing a small number of high-variance features can reduce the norm more than removing many low-variance features.

The norm relation follows directly from the definition of sample variance. For a centered feature \(j\), \(s_{bj}^2=\sum_i x_{bij}^2/(n-1)\). Rearranging and summing over the columns gives

\|X_b\|_F^2 =\sum_{j=1}^{p_b}\sum_{i=1}^{n}x_{bij}^2 =(n-1)\sum_{j=1}^{p_b}s_{bj}^2 =(n-1)V_b.

This identity concerns the sum of the marginal feature variances. It does not require the features to be uncorrelated: the Frobenius norm sums squared entries, so cross-feature covariances do not appear in this formula. If columns are not centered, the squared norm also includes contributions from their nonzero means, and the identity with summed sample variances no longer holds as written. Therefore, when this variance interpretation is intended, calculate it on the centered block.

Worked Example: Removing a High-Variance Feature

Suppose a hypothetical centered block has \(n=5\) subjects and three features with sample variances \(4\), \(1\), and \(9\) squared measurement units. Filtering removes the third feature, whose variance is \(9\). Find the summed variance and Frobenius norm before and after filtering.

Before filtering, the summed variance is \(V_b=4+1+9=14\) squared units. The squared Frobenius norm is \((n-1)V_b=4(14)=56\) squared units, so \(\|X_b\|_F=\sqrt{56}\approx7.483\) measurement units.

After filtering, \(V_{b,\mathrm{ret}}=4+1=5\) squared units. The squared norm is \(4(5)=20\), so \(\|X_{b,\mathrm{ret}}\|_F=\sqrt{20}\approx4.472\) measurement units. The retained block has \(5/14\approx0.3571\) of its original summed variance. The large reduction occurs because filtering removed a feature that contributed \(9/14\) of the original summed variance.

If the original weight was intended to preserve the block’s weighted norm, its compensating multiplier would be \(\sqrt{56/20}\approx1.673\). Whether to use that multiplier depends on the weighting objective, not on filtering alone.

When does the weight need to be recomputed?

A block weight \(w_b\) multiplies the block matrix, giving the weighted matrix \(w_bX_b\). Since \(\|w_bX_b\|_F=|w_b|\|X_b\|_F\), a weight chosen to make a block’s norm equal a target \(c\) must satisfy \(|w_b|\|X_b\|_F=c\). For nonnegative weights, this gives \(w_b=c/\|X_b\|_F\). The earlier material on Frobenius-norm normalization defines this type of weighting objective; after filtering, its input norm is the norm of the retained matrix.

For a block whose features are retained without further transformation, the update needed to preserve its previous weighted norm is particularly simple. If its original weight was \(w_b\), then

w_{b,\mathrm{new}} =w_b\frac{\|X_b\|_F}{\|X_{b,\mathrm{ret}}\|_F} =w_b\sqrt{\frac{V_b}{V_{b,\mathrm{ret}}}}.

This update preserves that block’s old weighted norm; it does not by itself ensure that all blocks have equal weighted norms. To equalize multiple retained blocks, calculate a norm for each retained matrix and choose weights according to the same common target or normalization rule. If weights are constrained to sum to one, for example, normalize the inverse retained norms to sum to one.

Worked Example: Updating an Equal-Norm Weight

A hypothetical block has \(n=6\) subjects and eight centered features, each with sample variance \(1\) squared unit. Its initial norm is \(\sqrt{(6-1)(8)}=\sqrt{40}\approx6.325\). Suppose its weight was set to \(1/\sqrt{40}\approx0.1581\), giving weighted norm \(1\). Filtering retains three features, all still with sample variance \(1\).

The retained norm is \(\sqrt{(6-1)(3)}=\sqrt{15}\approx3.873\). Keeping the old weight would give weighted norm \(0.1581(3.873)\approx0.6124\), which is below the target of \(1\). To restore that target, set \(w_{b,\mathrm{new}}=1/\sqrt{15}\approx0.2582\). Equivalently, the update factor is \(\sqrt{40/15}=\sqrt{8/3}\approx1.633\), and \(0.1581(1.633)\approx0.2582\).

This calculation assumes that the retained columns are unchanged and have unit sample variance after filtering. If their variances differ from \(1\), use their actual retained norm rather than the number of features alone.

Feature scaling changes the calculation

The preceding formulas describe filtering columns while leaving retained values unchanged. If filtering is followed by a transformation such as re-standardizing each retained feature, calculate the norm after that transformation. Re-standardization can restore a removed feature’s contribution per retained column even though the number of columns has fallen. The relevant quantity for choosing a norm-based weight is always the norm of the matrix that will actually enter the integrated analysis—not a norm from an earlier preprocessing stage.

Worked Example: Equal Feature Variances After Filtering

Consider two hypothetical centered blocks measured on \(n=5\) subjects. Block A has four features, each with sample variance \(1\), so \(V_A=4\) and \(\|X_A\|_F=\sqrt{4(4)}=4\). Block B has two features, each also with sample variance \(1\), so \(V_B=2\) and \(\|X_B\|_F=\sqrt{4(2)}=\sqrt{8}\approx2.828\).

Suppose a filter retains two features from A and one from B, and the retained features are then re-standardized to sample variance \(1\). The new summed variances are \(V_{A,\mathrm{ret}}=2\) and \(V_{B,\mathrm{ret}}=1\). Their norms are \(\sqrt{4(2)}=\sqrt{8}\approx2.828\) and \(\sqrt{4(1)}=2\), respectively. Inverse-norm weights normalized to sum to one are

w_A=\frac{1/\sqrt{8}}{1/\sqrt{8}+1/2}\approx0.4142, \qquad w_B=\frac{1/2}{1/\sqrt{8}+1/2}\approx0.5858.

The weighted norms are both approximately \(1.172\): \(0.4142(2.828)\approx1.172\) and \(0.5858(2)\approx1.172\). These weights achieve equal weighted norms under the stated sum-to-one constraint. They are not simply obtained by applying a feature-count ratio to the old weights; the post-filtering norms determine the answer.

A cross-block update and practical checks

The effect of filtering on relative block weights is easiest to see when different blocks lose different amounts of summed variance. Consider two blocks with the same subjects and columns centered on those subjects. If the objective is equal weighted norms, use the retained norms for both blocks, even if one block’s norm changed little.

Worked Example: Recomputing Relative Weights After Filtering

Suppose hypothetical block A has four features with variances \(1,1,1,1\), while block B has three with variances \(4,1,1\). Both blocks have \(n=5\) subjects. Initially, their norms are \(\sqrt{4(4)}=4\) and \(\sqrt{4(6)}=\sqrt{24}\approx4.899\). With nonnegative inverse-norm weights constrained to sum to one, the initial weights are

w_A=\frac{1/4}{1/4+1/\sqrt{24}}\approx0.5505, \qquad w_B=\frac{1/\sqrt{24}}{1/4+1/\sqrt{24}}\approx0.4495.

Now suppose filtering removes one variance-\(1\) feature from A and the variance-\(4\) feature from B. The retained summed variances are \(3\) and \(2\), so the retained norms are \(\sqrt{12}\approx3.464\) and \(\sqrt{8}\approx2.828\). Recomputing the weights gives

w_{A,\mathrm{ret}} =\frac{1/\sqrt{12}}{1/\sqrt{12}+1/\sqrt{8}}\approx0.4495, \qquad w_{B,\mathrm{ret}} =\frac{1/\sqrt{8}}{1/\sqrt{12}+1/\sqrt{8}}\approx0.5505.

The weights reverse their relative order because B lost more summed variance. The weighted norms match: \(0.4495(3.464)\approx1.557\) and \(0.5505(2.828)\approx1.557\). This conclusion follows from the equal-norm objective; it does not establish that equal norms are the scientifically preferred objective.

Useful diagnostics after filtering include recording the number of retained features, the retained summed variances, the retained norms, and the weights actually applied. Check that weights produce the intended target on the post-filtering matrices. Also check that comparisons use the same subject set and the same preprocessing state. If feature variances are not comparable because transformations differ by block, the numerical norm comparison may not have the interpretation intended by the weighting scheme.

Alternatives, limitations, and common mistakes

There is no universal rule that every block weight must be recomputed. A norm-based weight should be updated if the goal is to retain a specified weighted norm or equalize weighted norms after filtering. A weight chosen for a different stated goal—such as reliability-based influence—should be reassessed against that goal, rather than automatically replaced by an inverse-norm weight. Another option is to retain fixed weights deliberately, accepting that filtering will change the blocks’ relative contributions. The choice should be explicit and applied consistently.

A frequent error is to assume that removing half a block’s features halves its norm and therefore requires doubling its weight. The norm is the square root of a sum of squared entries. If the removed features have the same variance as those retained, removing half the features halves the squared norm, so the norm is multiplied by \(\sqrt{1/2}\), not \(1/2\). The compensating weight multiplier is then \(\sqrt{2}\), not \(2\). With unequal feature variances, even that square-root-of-count calculation is inappropriate; use the actual retained variances or compute the retained matrix norm directly.

Norm matching also has limits. It controls a particular measure of total block magnitude, not the quality of the features, their relevance to a scientific outcome, or their dependence structure. Filtering chosen using outcomes can introduce selection bias into later analyses, and weights calculated on one sample may not generalize to another. These considerations do not alter the norm arithmetic, but they affect how the resulting weights should be interpreted and validated.

Key takeaway: Filtering changes a block’s norm according to the variance removed. Recompute a weight when its defined objective depends on the retained norm, and calculate that norm after all transformations that precede integration.

Check Your Understanding

For each question, state your reasoning before checking the solution.

A centered block has \(n=4\) subjects and feature variances \(2,2,8\). If the feature with variance \(8\) is removed without changing the other columns, what are the retained summed variance and norm?
A block’s norm falls from \(10\) to \(5\) after filtering. By what factor must its weight change to preserve its original weighted norm?
Two centered blocks have the same number of subjects, and filtering removes the same number of features from each. Is it necessarily true that their norms decrease by the same proportion?
Why must a norm-based weight be calculated from the post-filtering, post-transformation matrix rather than from the original matrix?

Solutions. (1) The retained summed variance is \(2+2=4\); the retained norm is \(\sqrt{(4-1)(4)}=\sqrt{12}\approx3.464\). (2) The compensating multiplier is old norm divided by new norm, \(10/5=2\). (3) No. The proportions depend on the variances of the removed and retained features, not only on the number removed. (4) The weight’s effect is determined by the norm of the matrix it multiplies. Filtering and subsequent transformations can change that norm, so using an earlier norm may fail to achieve the intended weighted magnitude.