Why Correlation Changes the Meaning of Stability
In genomic data, predictors often occur in correlated blocks. Nearby variants may be correlated through linkage disequilibrium, and expression measurements can move together because genes share regulation or biological pathways. A penalized fit may predict well while selecting different members of such a block under different subsamples. Consequently, a low selection frequency for one feature does not necessarily mean that its correlated group is uninformative; conversely, frequent selection of some group member does not establish that every member, or any particular member, is important.
The key is to distinguish feature-level stability from group-level stability. Feature-level stability asks whether a particular predictor is repeatedly selected. Group-level stability asks whether at least one member of a specified correlated group is selected, or whether a group-level procedure repeatedly selects the group. These are different summaries and answer different questions.
The introductory material on stability selection establishes the use of repeated subsamples and selection frequencies. Here, the focus is how to interpret those frequencies when predictors are correlated. The resampling scheme, fitting method, and tuning rule must be held fixed when comparing selection patterns; otherwise, observed differences may reflect changes in the analysis protocol rather than feature correlation.
These frequencies describe how often the specified procedure selected a feature or group across the chosen resamples. They are not posterior probabilities that a feature is truly associated, and they do not establish causation. They depend on the observed data, the resampling design, the model-fitting and tuning rules, and the definition of selection. A group should generally be defined before inspecting its stability results, using scientific annotations or a stated correlation rule. Defining groups after seeing which features alternate risks overstating the evidence for a stable group.
Substitution Within a Correlated Group
When two predictors carry similar information, a sparse fitting procedure may select one in some resamples and the other in different resamples. The individual frequencies can then be moderate even though the group is selected often. Co-selection adds useful context: it shows whether group members tend to appear together or whether they tend to substitute for one another.
Worked Example: Alternating Selection Within a Correlated Pair
In a hypothetical genomic analysis, two standardized variants have sample correlation \(r=0.96\). A fixed penalized fitting and tuning procedure is applied to \(B=100\) subsamples. Variant A is selected alone in 38 subsamples, variant B alone in 32, both in 12, and neither in 18. These counts sum to \(38+32+12+18=100\).
Variant A is selected in its 38 A-only resamples and the 12 resamples where both are selected. Thus, \(\hat{\pi}_A=(38+12)/100=0.50\). Similarly, \(\hat{\pi}_B=(32+12)/100=0.44\). Neither feature reaches a hypothetical individual-frequency threshold of 0.70.
The pair is selected whenever A alone, B alone, or both are selected. Therefore, \(\hat{\pi}_{\{A,B\}}=(38+32+12)/100=0.82\). The group is selected in 82% of these resamples even though neither individual feature is selected in 70% of them.
The co-selection frequency is \(12/100=0.12\). Among resamples selecting A, the fraction also selecting B is \(12/50=0.24\); among resamples selecting B, the fraction also selecting A is \(12/44\approx0.273\). The high correlation and relatively infrequent co-selection are consistent with substitution: either feature can represent much of the same information. This pattern supports a stable group-level signal under this procedure, not a claim that both variants are independently predictive or that either is causally relevant.
For two features, inclusion-exclusion gives the group frequency exactly:
The joint-selection term must be subtracted because resamples selecting both would otherwise be counted twice. With more features, the any-member frequency is the fraction of resamples with at least one selection; it should be calculated directly from resample-level indicators rather than by adding marginal frequencies.
Group Frequencies, Group Size, and Dependence
An “any member selected” frequency is easy to interpret, but it depends on how many features are in the group and how their selection events depend on one another. To see the group-size effect, suppose, only for illustration, that the selection indicators for all members are independent and each member has the same selection frequency \(q\). The probability that none of the \(m\) features is selected is \((1-q)^m\). Hence:
This formula is not a general model for correlated genomic predictors: independence is often implausible, and empirical frequencies should be computed directly. It does show why a larger group can have a high any-member frequency even when every feature is individually selected infrequently.
Worked Example: Why Group Size Can Inflate an Any-Member Frequency
Consider two hypothetical groups and suppose, for this calculation only, that feature selections within each group are independent. Group A contains 10 features, each with selection probability \(q=0.20\). Its any-member probability is
Group B contains 2 features, each with selection probability \(q=0.40\). Its any-member probability is
Thus, the larger group has an any-member frequency of about 0.893 even though each of its features is selected only 20% of the time. It would be misleading to conclude from these numbers alone that group A has stronger evidence than group B. Group sizes differ, and the independence calculation is illustrative rather than a model for real correlated features.
Correlation changes the chance of joint selection as well as the chance that at least one group member is selected. Strongly substitutable members may have low co-selection, while features that contribute complementary information may be selected together more often. Therefore, report the group definition and size, individual frequencies, and at least one measure of joint selection or substitution when those details affect the scientific interpretation.
A Worked Comparison of Individual and Group Stability
Worked Example: A Stable Feature Versus a Frequently Selected Group
Suppose 100 hypothetical resamples produce the following counts. In a two-feature group \(G=\{g_1,g_2\}\), \(g_1\) is selected in 45 resamples, \(g_2\) in 43, and both in 10. A separate feature \(h\) is selected in 82 resamples. For illustration, use a frequency threshold of 0.75 and call a feature or group “stable at this threshold” when its empirical frequency is at least 0.75.
The group frequency follows from inclusion-exclusion:
The individual frequencies are \(\hat{\pi}_{g_1}=0.45\), \(\hat{\pi}_{g_2}=0.43\), and \(\hat{\pi}_h=0.82\). At the stated threshold, \(h\) is individually stable, and group \(G\) is stable as an any-member group, but neither \(g_1\) nor \(g_2\) is individually stable. The overlap of 10 resamples means that \(g_1\) and \(g_2\) are not always substitutes; the remaining 68 group-selected resamples select exactly one of them.
The defensible conclusion is that the fitting protocol repeatedly selects at least one member of \(G\), while it does not repeatedly identify either member alone at this threshold. The conclusion is not that \(g_1\) and \(g_2\) each have a 78% selection frequency. Nor does the comparison show that \(G\) predicts better than \(h\): selection stability and predictive performance are separate properties.
How to Diagnose Correlation-Driven Instability
A useful diagnostic begins with the full selection-indicator matrix, not just a ranked list of individual frequencies. For each prespecified group, calculate the number of members, each member's frequency, the any-member frequency, and pairwise or group-level co-selection summaries. Then compare the selection pattern with the predictor correlation structure. A combination of strong within-group correlation, moderate individual frequencies, high any-member frequency, and limited co-selection is a recognizable substitution pattern.
Check that the same feature definitions and preprocessing are used in every resample. For example, inconsistent variant filtering or feature mapping can make a frequency table compare different predictors under the same label. Any data-adaptive standardization or filtering should be handled in a way consistent with the intended resampling procedure. Also check the selected model size and tuning behavior: if some resamples select many more features than others, a high group frequency may partly reflect broad model size rather than a uniquely reproducible group.
Useful summaries answer distinct questions:
- Individual frequency: how often is this exact feature selected?
- Any-member group frequency: how often is at least one feature in the group selected?
- Co-selection: how often are specified members selected together?
- Group size and within-group correlation: how many possible substitutes are being counted, and how redundant are they?
A group-level result is most interpretable when groups are biologically meaningful or created by a transparent, prespecified correlation rule. Overlapping groups need special care: the same selected feature can make several groups appear selected, so their frequencies are not independent pieces of evidence. A threshold chosen for individual features should not automatically be treated as a calibrated threshold for groups. Any false-selection guarantee derived for feature-level selections does not automatically transfer to an any-member group summary; the group-level error target and procedure must be justified separately.
Alternatives, Limitations, and Common Mistakes
The simplest alternative is to report only feature-level frequencies. This is appropriate when the scientific question concerns exact feature identification, but it can understate reproducibility when correlated features interchange. Group-level summaries address reproducibility at a broader resolution, but can conceal which members are selected and can favor larger groups. Reporting both levels is often more informative than choosing one.
Another option is a method that selects groups directly, such as a grouped penalty when the groups are meaningful and specified in advance. This changes the fitting procedure and the definition of selection: the output concerns group inclusion rather than merely whether any member happened to enter an ordinary feature-level fit. It may be useful when group-level discovery is the target, but it does not by itself establish that all members are relevant, eliminate dependence on group construction, or solve issues caused by overlapping groups. A feature-level penalized fit followed by aggregation answers a different question from a group-penalized fit.
The any-member summary also has a resolution limitation. A group can be stable while its selected representative varies, and a broad group can appear stable because it contains many candidate features. Conversely, a group with low any-member frequency may still contain a scientifically important feature that is difficult to recover under the particular sample and fitting protocol. Stability is evidence about reproducibility under a specified procedure, not a universal property of the data-generating process.
Common mistake: “No gene in this correlated block has a frequency above 0.70, so the block is unstable.” This reasoning confuses individual-feature stability with group stability. If different correlated members substitute across resamples, the group may be selected frequently even though no member crosses the individual threshold. Calculate the group frequency directly, inspect co-selection, and report that the block—not a particular gene—was reproducibly selected. The reverse mistake is equally important: a high group frequency does not make every member stable.
Check Your Understanding
Use the stated counts and definitions to distinguish feature-level from group-level stability. Solutions follow each question.
- Across 50 resamples, feature A is selected 20 times, feature B 18 times, and both are selected 6 times. Find each individual frequency and the any-member group frequency for \(\{A,B\}\).
- Why can a group with many members have a high any-member frequency even if each member is selected infrequently?
- A correlated pair has group frequency 0.90, while each member has frequency below 0.60. State a defensible interpretation and one conclusion that is not justified.
- Name two diagnostics, in addition to the any-member group frequency, that help interpret substitution within a correlated group.
Solutions. (1) A is selected \(20/50=0.40\) of the time, and B is selected \(18/50=0.36\) of the time. The group count is \(20+18-6=32\), so its frequency is \(32/50=0.64\). (2) More members create more opportunities for at least one selection. Under independence with equal member frequency \(q\), the any-member probability is \(1-(1-q)^m\), which increases with group size \(m\) when \(0<q<1\). (3) The fitting protocol selected at least one member of the pair in 90% of resamples, but neither exact member was selected as often as 60%; this is consistent with group-level stability and member substitution. It does not justify saying both members are individually stable, or that either is causal. (4) Examples include individual selection frequencies, pairwise co-selection frequency, within-group correlation, and group size.