Varying Sample Size And Feature Dimensions

Why Vary These Dimensions Systematically?

An integration method can behave differently when it has fewer subjects to estimate relationships, more features to search or combine, or larger groups of correlated features. A simulation that varies only one convenient sample size and one feature count cannot show whether a method is robust across those conditions. A systematic design varies these dimensions deliberately, so that performance differences can be interpreted in relation to the conditions that produced them.

The aim here is to specify scenarios over three factors: the number of subjects, the number of features in relation to the number of subjects, and the number of features grouped into a block. A block may represent a set of features assigned to the same dependence group in the generator. Block size is therefore distinct from the total number of features: changing block size should not silently change the total dimension unless that is explicitly part of the design.

Definition: Let \(n\) be the number of subjects, \(p_k\) the number of features in modality \(k\), and \(b_k\) the number of features per block in that modality. The modality-specific dimensionality ratio is \(\gamma_k=p_k/n\). The pooled feature count is \(P_{\mathrm{tot}}=\sum_{k=1}^{K}p_k\), and the pooled dimensionality ratio is \(\Gamma=P_{\mathrm{tot}}/n\). A scenario specifies values of these factors, along with any remaining generator settings held fixed or deliberately varied.

The distinction between \(\gamma_k\) and \(\Gamma\) matters. If each of two modalities has \(p_k<n\), then neither modality is individually high-dimensional, even though their pooled feature count may exceed \(n\). Conversely, one modality may have \(p_k>n\) even when the pooled ratio is not the specific quantity used to assess a method. State which ratio is relevant to the method and report both modality-specific and pooled dimensions when useful.

Constructing the Factorial Scenarios

A factorial design crosses the selected levels of its factors. For example, two values of \(n\), two values of \(b\), and two values of \(\gamma\) give \(2\times2\times2=8\) combinations, provided every combination is feasible. Crossing factors is valuable because it allows the design to reveal whether the effect of one factor depends on another. It is not enough to list a few scenarios that happen to vary all three: if every increase in \(n\) is paired with a decrease in \(\gamma\), their effects cannot be separated by the design.

For each modality, \(p_k\) must be an integer. If the design uses equal-sized blocks with no remainder, \(p_k\) must also be divisible by \(b_k\). Thus a ratio level does not always yield a feasible feature count for every \(n\). Choose compatible levels, define a rule for any remainder, or report the achieved ratio rather than presenting a rounded target ratio as exact. When modalities differ, specify whether block sizes and ratios are shared across modalities or set separately.

A practical starting point is to select levels that cover distinct operating conditions rather than many nearly identical values. Include a sample-size range that represents the intended application, and ratio levels that span lower-dimensional and high-dimensional regimes. The latter includes \(\gamma_k>1\), where that modality has more features than subjects. The design need not use a ratio above one for every method or application; it should include that regime when it is relevant to the benchmark question.

\text{high-dimensional within modality } k \iff \gamma_k=\frac{p_k}{n}>1, \qquad \text{high-dimensional pooled data} \iff \Gamma=\frac{\sum_{k=1}^{K}p_k}{n}>1.

These thresholds describe dimensions, not method performance. A ratio above one does not by itself imply that estimation is impossible, nor does a ratio below one guarantee reliable integration. The data-generating structure and method determine what can be recovered. The ratio is a concise design descriptor that makes the subject-to-feature balance explicit.

Worked Example: Crossing Sample Size, Ratio, and Block Size

Suppose a hypothetical simulation has two modalities with the same feature count, ratio, and block size. Choose \(n\in\{80,240\}\), \(\gamma\in\{0.5,2\}\), and \(b\in\{10,20\}\), with exact equal-sized blocks. How many scenarios result, and what dimensions do they imply?

There are \(2\times2\times2=8\) combinations. For each combination, set \(p_k=\gamma n\) in each modality. At \(n=80\), the two feature counts per modality are \(0.5(80)=40\) and \(2(80)=160\). At \(n=240\), they are \(0.5(240)=120\) and \(2(240)=480\).

All four feature counts are divisible by both proposed block sizes: \(40,160,120,\) and \(480\) are each divisible by 10 and 20. Thus every crossed combination is feasible. For example, at \(n=80,\gamma=2,b=20\), each modality has \(p_k=160\) features in \(160/20=8\) blocks. This is high-dimensional within each modality because \(160/80=2>1\); pooled across two modalities, \(P_{\mathrm{tot}}=320\) and \(\Gamma=320/80=4\).

At \(n=240,\gamma=0.5,b=10\), each modality has \(120\) features in 12 blocks. The within-modality ratio is \(120/240=0.5\), while the pooled ratio is \(240/240=1\). Reporting only the pooled ratio would conceal that each modality separately has fewer features than subjects.

What the Factorial Contrasts Mean

A balanced factorial design gives a direct way to distinguish average factor effects from interactions. For a performance summary \(Y\), such as a recovery score calculated for each simulated data set, let \(\bar{Y}_{n,\gamma}\) denote its mean over the other factor levels and replications at a specified pair of levels. In a two-level design, the sample-size contrast can be defined as the average outcome at the larger \(n\) minus the average outcome at the smaller \(n\). An interaction asks whether that contrast changes with \(\gamma\).

The justification is arithmetic. In a balanced design with two levels of \(n\) and two levels of \(\gamma\), write the four cell means as \(m_{11},m_{12},m_{21},m_{22}\), where the first index denotes the \(n\) level and the second the \(\gamma\) level. The difference-in-differences is:

(m_{22}-m_{12})-(m_{21}-m_{11}).

The first parenthesis is the larger-minus-smaller sample-size contrast at the second ratio; the second is the same contrast at the first ratio. Subtracting them gives zero when the sample-size contrast is the same at both ratio levels. A nonzero value indicates that the observed sample-size contrast differs across ratios. This is a design interpretation, not proof of a general causal effect: it depends on the generator holding other mechanisms fixed and on the simulated summaries being computed consistently.

In a balanced factorial layout, each level of one factor occurs equally often with every level of the others. Consequently, the simple average contrast for a factor does not give extra weight to one particular level of another factor. If scenarios are omitted or replication counts differ across cells, averages can become unequally weighted; report the weighting rule and examine cell-level results rather than relying on a single overall contrast.

Worked Example: Interpreting a Sample-Size-by-Ratio Interaction

Consider hypothetical cell-average recovery scores at two sample sizes and two ratios. Scores are on a zero-to-one scale, with higher values indicating better recovery: \(m_{11}=0.60\) at small \(n\), low \(\gamma\); \(m_{12}=0.50\) at small \(n\), high \(\gamma\); \(m_{21}=0.75\) at large \(n\), low \(\gamma\); and \(m_{22}=0.55\) at large \(n\), high \(\gamma\).

At low \(\gamma\), the larger-minus-smaller sample-size contrast is \(0.75-0.60=0.15\). At high \(\gamma\), it is \(0.55-0.50=0.05\). Therefore the difference-in-differences is \(0.05-0.15=-0.10\). The sample-size contrast is 0.10 score units smaller at high dimensionality than at low dimensionality in these hypothetical results.

The negative interaction does not mean that increasing \(n\) reduced recovery: both sample-size contrasts are positive. It means only that the improvement associated with the larger sample size is smaller in the high-ratio condition in this example. No broader claim should be made without considering the particular generator, other factor levels, and variation across simulated data sets.

Factorial Choices and Feasibility Checks

The levels should make the design informative without creating accidental changes to the data-generating mechanism. In particular, changing \(n\) while holding \(p_k\) fixed changes \(\gamma_k\); changing \(p_k\) while holding \(n\) fixed changes both dimension and ratio. If the scientific question concerns the effect of subject count separately from the feature-to-subject balance, define the design in terms of \(n\) and \(\gamma_k\), then calculate \(p_k\). This makes clear that feature counts vary as needed to preserve the chosen ratio.

Block size needs equally careful treatment. If block size is intended to alter dependence-group size, keep the total feature count fixed across its levels wherever feasible. Then a larger \(b_k\) means fewer, larger blocks, not simply more features. If \(p_k\) is not divisible by \(b_k\), either select compatible factor levels or specify a consistent remainder rule, such as using one smaller final block. State that rule because it changes the block-size distribution.

Before simulation, check that each scenario has a unique, explicit tuple of dimensions and that the requested factor levels are feasible. Also confirm that the generator can implement all combinations: for instance, a block-correlation mechanism should be defined for every proposed \(b_k\), and increasing \(n\) should not accidentally change the signal proportion or noise scale unless that change is intended. These are design checks, not performance diagnostics.

Choose the factor definitions.
Specify whether dimensionality is represented by each \(\gamma_k\), the pooled \(\Gamma\), or both; define the meaning of each block size.

Set and cross the levels.
Choose interpretable levels for \(n\), \(b_k\), and the ratio, then form the planned combinations without confounding one factor with another.

Calculate integer dimensions.
For each combination, compute \(p_k=\gamma_k n\), verify that \(p_k\) is an integer, and check divisibility or the stated remainder rule for blocks.

Audit what changes.
Record \(n\), each \(p_k\), each \(b_k\), block counts, \(\gamma_k\), and \(\Gamma\), and confirm that unintended generator settings remain fixed.

Comparisons, Diagnostics, and Limitations

A full factorial design is useful when interactions are plausible and the number of combinations is manageable. A one-factor-at-a-time design is simpler, but if it changes only one factor around a single reference condition, it can miss interactions. A fractional factorial design uses fewer combinations, but some effects can be aliased with others; it is appropriate only when the omitted combinations and resulting ambiguity are acceptable for the benchmark question. A small set of hand-picked scenarios can still be useful for targeted stress tests, but it does not generally support the same clean separation of factor effects.

Common warning signs are structural. If each increase in \(n\) occurs only with a lower \(\gamma\), the two factors are confounded. If \(b_k\) changes together with \(p_k\), the design cannot attribute a difference specifically to block size. If a nominal ratio is rounded to produce an integer \(p_k\), but the achieved ratios are not reported, the stated factor levels are inaccurate. Inspect a scenario table before generating data; it should make these patterns visible.

Factorial structure does not make every scenario realistic, nor does it cover every possible combination of sample size, dimension, and dependence. The conclusions apply to the chosen levels and generator. A design can also become large quickly: the number of combinations is the product of the numbers of levels across factors. When computation is limited, prioritize scientifically meaningful contrasts and be explicit about which interactions or regions are not represented. The earlier material on matching omics marginal distributions concerns the feature distributions; in this design, those distributional settings should be kept consistent across dimensionality scenarios unless their variation is itself an intended factor.

Worked Example: Resolving an Infeasible Block Combination

A hypothetical single-modality design sets \(n=100\), targets \(\gamma=1.5\), and proposes \(b=20\) with equal-sized blocks and no remainder. Compute the requested feature count and assess feasibility.

The target count is \(p=\gamma n=1.5(100)=150\) features. The block count would be \(150/20=7.5\), which is not an integer. Thus the requested combination cannot be implemented with equal-sized blocks of 20 and no remainder.

One solution is to change the block size to 25: \(150/25=6\) exact blocks, while retaining the target ratio \(150/100=1.5\). A second solution is to keep \(b=20\) and allow a smaller final block: seven blocks of 20 use 140 features, leaving 10 features for the final block. If using this rule, report the block sizes as seven blocks of 20 and one block of 10; do not describe every block as size 20. The choice depends on the intended dependence structure, so it must be made before interpreting performance differences.

Common Pitfalls

Confusing pooled and modality-specific dimensionality. A pooled \(\Gamma>1\) does not imply that every modality has \(p_k>n\). Report the ratio that matches the method’s input and the question being studied.
Treating \(b_k\) as another name for \(p_k\). Block size describes group size; total features describe dimension. Varying both together prevents a clean interpretation of the block-size factor.
Calling a rounded ratio exact. Calculate the achieved \(p_k/n\) from the integer feature count and report it.
Assuming more subjects always offsets more features. The ratio summarizes their numerical balance, but performance also depends on signal, noise, dependence, and the method. Do not infer performance from the ratio alone.

Key takeaway: Define the dimensionality ratio explicitly, cross it with sample size and block size where feasible, and verify the resulting feature counts and blocks before running the simulation. The scenario table is part of the design: it should reveal both the intended comparisons and any constraints that limit them.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A study simulates \(n=120\) subjects and \(p_1=180\) features in one modality. What is \(\gamma_1\), and is this modality high-dimensional?
Two modalities each have 80 features and the simulation has 100 subjects. Find each modality-specific ratio and the pooled ratio. Which, if either, is above one?
For \(n=150\), a design targets \(\gamma=0.8\) and uses block size \(b=20\). Is exact equal blocking feasible? If not, give one compatible alternative that preserves the target ratio.
In a crossed design, the larger sample size appears only at the lower dimensionality ratio. What is the design problem, and what should be changed?

Solutions. (1) \(\gamma_1=180/120=1.5\), so the modality has more features than subjects and is high-dimensional. (2) Each modality has ratio \(80/100=0.8\); the pooled ratio is \((80+80)/100=1.6\). Neither modality-specific ratio is above one, but the pooled ratio is. (3) The target feature count is \(p=0.8(150)=120\); \(120/20=6\), so exact equal blocking is feasible with six blocks of 20. (4) Sample size and ratio are confounded because their levels do not occur in all combinations. Cross the levels so each sample-size level occurs with each ratio level, subject to feasibility, or state that the design cannot estimate their effects separately.