Choosing Outcome Signal Forms for Benchmarks

Why Outcome Form Is Part of the Benchmark

A multi-omics benchmark can vary which modalities contain predictive information, how that information is distributed across blocks, and how an outcome responds to it. These are distinct design choices. If the outcome is generated by a linear function of the features, a method that fits linear effects may have an advantage for reasons that do not arise from integration itself. Conversely, a benchmark using a curved or interaction-based signal may be difficult because the outcome relationship is misspecified, even when the relevant modalities have been combined appropriately.

The goal here is to define the outcome-generating rule explicitly. Keep the predictor signal, the outcome scale, and the random noise separate in the specification. Then a benchmark can vary outcome structure deliberately, rather than allowing it to change unnoticed along with the integration scenario.

The same signal function can be used with different outcome forms. For a continuous outcome, a common model adds a continuous error to the signal. For a binary outcome, the signal is commonly mapped to a probability, from which a Bernoulli outcome is drawn. The predictor features may be correlated within or across blocks; what matters for a controlled comparison is that their distribution and the signal-generating rule are stated clearly.

Continuous Outcomes: Linear and Nonlinear Signals

A linear continuous-outcome model has the form \(Y_i=f_i+\epsilon_i\), where the signal \(f_i\) is a linear combination of selected features and \(\epsilon_i\) is random error. A standard simulation assumption is that the errors are independent across individuals, have mean zero, have constant variance \(\sigma^2\), and are independent of the features. Gaussian errors, \(\epsilon_i\sim N(0,\sigma^2)\), are a convenient specific choice, not a requirement for every linear model.

Here, \(\beta_0\) is the intercept and \(\boldsymbol{\beta}^{(m)}\) determines which features in block \(m\) contribute to the signal and by how much. Coefficients describe changes in the expected outcome per one-unit change in a feature, holding the other included features fixed. Thus the units and scaling of features should be stated: a coefficient for a standardized feature corresponds to a one-standard-deviation increase, not a one-unit increase on its original assay scale.

A nonlinear continuous signal changes the function \(f_i\), not necessarily the error distribution. For example, a squared term can create a U-shaped relationship, a threshold can create a piecewise relationship, or a cross-block product can represent an interaction. A model with Gaussian errors can still be used, but its conditional mean is now nonlinear in the original features. The chosen nonlinear form should reflect the intended benchmark challenge, rather than being added without a clear interpretation.

For an intentionally nonlinear signal, the simulation rule must state the transformation and its scale. Centering a squared feature is often useful because it makes the nonlinear contribution have mean zero under a stated feature distribution, leaving the intercept easier to interpret.

Binary Outcomes: Mapping Signal to Probability

For a binary outcome \(Y_i\in\{0,1\}\), a linear signal cannot itself be used as a probability because it may fall outside the interval from zero to one. A logistic model maps a linear predictor \(\eta_i\) to a probability using the logistic function. Conditional on the features, \(Y_i\) is Bernoulli with that probability. In this model, observations are independent conditional on their feature values and the specified parameters.

The linear predictor is on the log-odds scale: \(\eta_i\) is the logarithm of the conditional odds of \(Y_i=1\). A one-unit feature increase changes the log-odds by its coefficient, holding other features fixed. It does not generally change the probability by a constant amount; the probability change depends on the starting value of \(\eta_i\).

A short derivation shows why this link gives valid probabilities. Suppose \(Y_i=1\) when \(\eta_i+E_i>0\), where \(E_i\) has the standard logistic cumulative distribution function \(F(e)=1/(1+\exp(-e))\). Then

This construction is one interpretation of logistic regression; a benchmark can instead specify the Bernoulli probabilities directly. A nonlinear binary signal can be created by defining \(\eta_i\) as a nonlinear function, then applying the same probability mapping. The probability link and the shape of the feature-to-signal relationship are separate design choices.

Designing a Benchmark That Isolates the Intended Challenge

To study integration, specify the outcome signal and its feature allocation before comparing strategies. For instance, the signal may use features from one block, several blocks additively, or a cross-block interaction. Earlier material on shared, complementary, redundant, and interaction signals provides ways to specify which blocks contain predictive information; the outcome rule then determines how that information maps to \(Y_i\).

A useful design principle is to preserve the same feature distribution and outcome-generating function when the benchmark question concerns how strategies combine the blocks. If the outcome form changes at the same time as the integration scenario, a performance difference cannot be attributed cleanly to either change. Conversely, if the purpose is to test robustness to outcome structure, vary the outcome form deliberately and label each scenario accordingly.

When comparing linear and nonlinear outcome scenarios, the numerical coefficients alone do not guarantee equal difficulty. A nonlinear function can have a different variance, and binary scenarios can have different event prevalence and probability distributions. Consider reporting the signal definition, error variance for continuous outcomes, and prevalence for binary outcomes. If equal difficulty is an objective, calibrate the settings using an explicitly chosen predictive target; do not claim that two scenarios are equally difficult merely because they use the same coefficient values.

Diagnostics, Alternatives, and Common Pitfalls

A simulation should be checked against its mathematical specification, not only against a successful software run. For continuous outcomes, verify that the implemented conditional mean follows the intended curve, that residuals have the specified mean and variance, and that no unintended clipping or transformation has changed the outcome scale. For binary outcomes, verify that calculated probabilities lie in \([0,1]\), that empirical event prevalence is plausible under the feature distribution, and that the Bernoulli sampling rule was applied to each individual. A nonlinear effect can be missed by checking only a linear correlation; inspect the generating function or compare signal values over a feature grid.

The alternatives are not interchangeable. A linear continuous model is direct when a continuous scale and additive mean signal are intended. A nonlinear continuous model is appropriate when curvature, thresholds, or interactions are part of the desired scenario. Logistic generation is a standard option for a binary target whose conditional event probability changes with features. Thresholding a continuous simulated outcome to make it binary is another possible construction, but it defines a different data-generating process and can induce prevalence and boundary behavior determined by the threshold. It should not be substituted silently for a Bernoulli probability model.

A particularly common mistake is to conclude that a strategy is poor at integration when it performs poorly in a benchmark with an outcome relationship its prediction model cannot represent. The correction is to state and inspect the outcome signal, then distinguish a benchmark of integration under a fixed outcome form from a benchmark that intentionally varies outcome form. A second mistake is to compare a continuous outcome and a binary outcome as if identical coefficients implied identical signal strength. They are on different scales, and the binary model's probability response is nonlinear even when its log-odds predictor is linear.