Weighted Kernel Combinations and Constraints

Why Constrain Kernel Weights?

When several omics views describe the same patients, each view can supply a kernel matrix: a matrix of pairwise similarities. A weighted combination offers a direct way to form one similarity matrix from several such views. The weights control how much each input matrix contributes numerically. Constraining them to be nonnegative and to sum to one makes the combination a convex average, which gives the result useful mathematical guarantees and a clear scale interpretation relative to its inputs.

Those guarantees are not a substitute for checking the input matrices. In particular, the weights do not make kernels with very different scales comparable, and a weight is not automatically a measure of biological importance. The construction is most interpretable when the kernels refer to the same patients in the same order and their scales have been considered in advance.

Definition: Let $K_1,\ldots,K_M$ be $M$ kernel matrices, each of size $n\times n$, for the same $n$ patients in the same order. A convex kernel combination is $$ K_w=\sum_{m=1}^{M} w_m K_m, \qquad w_m\geq 0, \qquad \sum_{m=1}^{M}w_m=1. $$ The vector of weights $w=(w_1,\ldots,w_M)$ lies in the probability simplex. Each $w_m$ is the coefficient assigned to kernel $K_m$.

A kernel matrix is positive semidefinite (PSD) if it is symmetric and $a^\mathsf{T}K_ma\geq 0$ for every real vector $a$ of length $n$. This property matters because it permits the matrix to represent inner products in a feature space, including cases where that space is not written explicitly. The convex-combination guarantee below relies on each input being PSD; it does not follow from the weight constraints alone.

What the Simplex Constraint Guarantees

The nonnegative weights prevent one input kernel from being subtracted from another. The sum-to-one condition fixes the total coefficient at one, so the combination is a weighted average rather than an arbitrary rescaling of a sum. For two kernels, the constraint can be written $K_w=\lambda K_1+(1-\lambda)K_2$, where $0\leq\lambda\leq1$. At $\lambda=0$ or $1$, the combination is exactly one input; values strictly between them mix both.

Theorem: If every $K_m$ is symmetric and positive semidefinite and the weights satisfy $w_m\geq0$, then $K_w=\sum_m w_mK_m$ is symmetric and positive semidefinite. The sum-to-one condition is not needed for this PSD result, but it makes the combination a convex average and controls its overall coefficient scale relative to the inputs.

The PSD claim follows directly from the definition. For any vector $a$, the quadratic form of the combination is a weighted sum of the input quadratic forms. Each term is nonnegative, so the total is nonnegative. Symmetry is preserved by the same weighted sum.

a^\mathsf{T}K_w a =a^\mathsf{T}\left(\sum_{m=1}^{M}w_mK_m\right)a =\sum_{m=1}^{M}w_m(a^\mathsf{T}K_m a) \geq 0.

This short argument also clarifies why negative coefficients are different: a negative $w_m$ can make a nonnegative input quadratic form contribute a negative amount. A sum of PSD matrices with nonnegative coefficients is PSD, but a signed sum is not guaranteed to be. The simplex is a stronger constraint than the one required for the PSD guarantee: it requires nonnegative coefficients and also fixes their sum to one.

Worked Example: Combining Two Valid Kernels

Suppose two hypothetical omics kernels for two patients are $K_1=\begin{pmatrix}1&0.2\\0.2&1\end{pmatrix}$ and $K_2=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix}$. Choose $w_1=0.25$ and $w_2=0.75$. Both matrices are symmetric. Their eigenvalues are $1.2,0.8$ and $1.8,0.2$, respectively, so both are PSD.

The weights are nonnegative and sum to one. The combined matrix is $$ K_w=0.25K_1+0.75K_2 =0.25\begin{pmatrix}1&0.2\\0.2&1\end{pmatrix} +0.75\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix} =\begin{pmatrix}1&0.65\\0.65&1\end{pmatrix}. $$

For example, the off-diagonal similarity is $0.25(0.2)+0.75(0.8)=0.05+0.60=0.65$. The combined matrix has eigenvalues $1.65$ and $0.35$, so it is PSD, as the theorem guarantees. Its diagonal remains one because both input diagonals are one and the weights sum to one.

Scale and Interpretation

For any pair of patients $i,j$, the combined entry is $K_w(i,j)=\sum_m w_mK_m(i,j)$. Thus, if all input entries for that pair lie between $L$ and $U$, the combined entry also lies between $L$ and $U$: it is an average with nonnegative weights summing to one. In particular, if every input entry lies in $[0,1]$, then so does every combined entry. If the inputs have different ranges or magnitudes, however, the combination inherits a weighted average of those values; the simplex constraint does not force the result into a chosen common range.

The same logic applies to diagonal entries and to the trace (the sum of diagonal entries): each is the weighted average of the corresponding input quantities. If every input has diagonal entries equal to one, the fused diagonal is also one. If one input has diagonal entries near 100 and another near 1, the fused diagonal reflects both scales. A sum-to-one constraint does not erase that difference.

A weight describes a coefficient in the formula, not a proportion of total biological signal, explained variance, or causal contribution. For instance, assigning weight $0.6$ to one kernel means that its entries are multiplied by $0.6$ in the specified sum. It does not mean that the corresponding omics assay explains 60% of patient similarity. That stronger interpretation would require additional modeling assumptions and evidence.

Worked Example: The Weights Do Not Equal Relative Influence

Consider two hypothetical kernels with off-diagonal entries $0.4$ and $40$, and choose $w_1=w_2=0.5$. Assume both are valid PSD kernels on the same patients. The fused off-diagonal entry is $$ K_w(i,j)=0.5(0.4)+0.5(40)=0.2+20=20.2. $$

Although the coefficients are equal, the second kernel contributes $20$ to this entry while the first contributes $0.2$. The second contribution is 100 times the first, because its input value is 100 times as large. The weights are equal, but the numerical contributions are not.

If the second kernel were rescaled by multiplying all its entries by $0.1$, its off-diagonal value would become $4$. With the same weights the fused entry would then be $0.5(0.4)+0.5(4)=2.2$. The combination remains a convex average of the supplied matrices, but its numerical scale and the relative contributions have changed. Therefore, scale comparability must be considered before the weights are interpreted.

Multiplying every input kernel by the same positive constant $c$ multiplies the fused kernel by $c$: $\sum_m w_m(cK_m)=cK_w$. The weights do not identify an absolute scale. Rescaling only one input is different: it changes that input’s numerical contribution while leaving its coefficient unchanged. These facts are why weights should be interpreted together with the definitions and scales of the kernels being combined.

Worked Example: Reading a Three-Kernel Average

Suppose three hypothetical kernels, all built for the same patients and with entries between zero and one, have a particular pairwise similarity of $0.2$, $0.6$, and $0.9$. Assign weights $w_1=0.2$, $w_2=0.3$, and $w_3=0.5$. They are nonnegative and sum to $0.2+0.3+0.5=1$.

For this pair of patients, the fused similarity is $$ K_w(i,j)=0.2(0.2)+0.3(0.6)+0.5(0.9) =0.04+0.18+0.45=0.67. $$

The result lies between the smallest and largest input values, $0.2$ and $0.9$, as an average must. The third kernel has the largest coefficient and also contributes the most to this entry: $0.45$, compared with $0.18$ and $0.04$. That numerical statement is specific to this entry; for another patient pair, the input values could differ and so could the contributions. The calculation does not establish that the third omics view is generally the most biologically important.

Assumptions, Checks, and Alternatives

Before forming or interpreting a weighted combination, verify that each input matrix describes the same set of patients in the same ordering. Otherwise, adding entries at position $(i,j)$ may combine similarities for different pairs. Check that the matrices are symmetric and PSD, as intended for valid kernels. Numerical calculations can produce tiny negative eigenvalues through rounding; a small discrepancy near zero should be assessed in light of numerical tolerance, while substantial negative eigenvalues indicate that the PSD guarantee cannot be assumed for that input.

Also verify the weights themselves: each must be nonnegative, and their sum must equal one up to the chosen numerical tolerance. A negative weight violates the convex-mixture interpretation. A sum different from one may still produce a PSD matrix if the coefficients remain nonnegative, but it is no longer a convex combination as defined here and changes the total coefficient scale.

At the boundary of the simplex, some weights can be zero. A zero weight excludes that kernel from the sum; it does not imply that the corresponding omics measurements are absent from the study. If all but one weight are zero, the fused kernel is just the remaining input. Such boundary solutions are valid, though they indicate that the formula is not mixing every view.

A direct additive kernel fusion can use fixed coefficients without requiring that they sum to one. If its coefficients are nonnegative, the result is still PSD, but its total coefficient scale is not fixed by the sum-to-one rule. The earlier material on product kernels gives a different construction: multiplying kernels combines similarities jointly rather than averaging their entries. Neither construction is universally preferable; the choice depends on whether an average of view-specific similarities or a joint similarity is the intended representation.

The convex combination also has limits. It does not decide which weights are appropriate, repair incompatible patient sets, or establish that the views are informative. It cannot make a poorly scaled view harmless simply by constraining weights, and it does not ensure that each view contributes meaningfully. Inspect the input definitions, ranges, diagonals, and resulting fused entries. If the fused matrix behaves unexpectedly, check the ordering and scales before attributing the pattern to the weights.

Common Mistake

A common error is to say, “The weights sum to one, so the fused kernel has a standardized scale and a weight of $0.7$ means that view explains 70% of the similarity.” The first claim is false when input kernels have different scales; the second confuses a formula coefficient with an explained proportion. The correction is to say that the result is a weighted average of the supplied entries, and that each weight controls a multiplier in that average. Interpreting relative influence requires considering the actual kernel values and their scales.

Key takeaway: Nonnegative weights summing to one define a convex kernel combination. If the input matrices are PSD, the result is PSD; its entries are weighted averages of the input entries. The constraint controls the combination’s coefficient scale, not the scale compatibility or biological meaning of the inputs.

Check Your Understanding

Answer each question, then compare with the solutions below.

Two PSD kernels receive weights $0.3$ and $0.7$. What property of the weights makes their combination a convex combination?
Why is a nonnegative weighted sum of PSD kernels PSD? Is the sum-to-one condition needed for that guarantee?
For one patient pair, two kernels have similarities $0.1$ and $0.9$, with weights $0.25$ and $0.75$. Calculate the fused entry.
Two kernels receive equal weights, but one has entries roughly 100 times larger than the other. Does that imply equal numerical contributions? Explain.

Solutions. (1) Both weights are nonnegative and $0.3+0.7=1$. (2) For every vector $a$, the fused quadratic form is $\sum_m w_m(a^\mathsf{T}K_ma)$, a sum of nonnegative terms. Nonnegative coefficients suffice for this PSD guarantee; summing to one is needed for the convex-average interpretation, not for PSD itself. (3) $0.25(0.1)+0.75(0.9)=0.025+0.675=0.700$. (4) No. Equal coefficients multiply unequal values, so the larger-scale kernel can make a much larger numerical contribution. The weights alone do not show equal influence.