Why Factor Counts Require More Than a Fit Score
In a multi-omics factor model, the number of latent dimensions determines how much structure the model can represent and how that structure is divided among views. Too few factors can leave meaningful shared or view-specific patterns in the residuals. Too many can turn noise, technical variation, or small sample irregularities into apparently meaningful factors. The task is therefore not simply to maximize fit: more flexible models usually fit the observed data at least as well.
For \(M\) matched omics views, let \(X^{(m)}\) be the transformed data matrix for view \(m\), with \(n\) samples and \(p_m\) features. A useful schematic model is
Here \(Z\) contains the sample scores for \(K\) shared factors, \(W^{(m)}\) contains the corresponding loadings in view \(m\), and \(Z^{(m)}\) and \(W^{(m)}_{\mathrm{spec}}\) describe \(L_m\) factors specific to that view. \(E^{(m)}\) is residual variation. The rank choices are the shared factor count \(K\) and the view-specific counts \(L_1,\ldots,L_M\). They need not be equal across views.
The comparison is meaningful only when candidates use the same samples, features, preprocessing, and observation model. For likelihood comparisons, they must also use comparable likelihoods and account for the same data. Rank selection cannot repair a misspecified observation model, poorly aligned samples, or confounding between latent directions. In particular, a factor-count decision is less reliable if shared and view-specific directions are not adequately distinguishable in the fitted model.
Fit, Complexity, and Information Criteria
For a fixed likelihood, adding factors often improves or preserves in-sample fit. When candidate models are nested and optimized successfully, their maximized log-likelihood cannot decrease as parameters are added: the larger model can reproduce the smaller one by setting added contributions to zero. This does not mean the larger model is preferable, because its improved fit may be due to extra flexibility.
A common fit-based summary is the maximized log-likelihood \(\ell\), or a residual sum of squares when a Gaussian model with a suitable residual structure is used. Residual fit should be examined by view as well as in aggregate: a large view can dominate a combined residual score and conceal poor fit in a smaller view. When views use different likelihoods, compare candidates using the full joint log-likelihood rather than adding incomparable residual summaries.
Akaike’s information criterion and the Bayesian information criterion balance fit against model complexity. If \(q\) is the number of estimable model parameters and \(n\) is the number of independent sample units, their usual forms are
Smaller values are preferred among the candidates being compared. The parameter count must reflect the fitted model, including its constraints; it is not simply the number of factors. Factor models have scale, sign, and rotation conventions, so naïvely counting every entry in score and loading matrices can overcount free parameters. Information criteria also rely on regularity approximations that may be imperfect for factor models, particularly near rank boundaries or with strong regularization. Treat a small difference as weak evidence, not as a definitive rank discovery.
The penalty has a direct interpretation. If adding factors improves \(-2\ell\) by less than the added penalty, the information criterion increases and favors the simpler model. For AIC, adding \(\Delta q\) parameters requires a reduction in \(-2\ell\) greater than \(2\Delta q\). For BIC, the corresponding threshold is \(\Delta q\log(n)\). These are selection rules, not tests that establish the truth of one rank.
Worked Example: Comparing Candidate Ranks With BIC
Suppose a hypothetical analysis of \(n=120\) matched samples fits three candidates using the same data and likelihood. The candidates have shared counts \(K=1,2,3\), no view-specific factors, and correctly counted parameter totals \(q=20,30,40\). Their fitted values of \(-2\ell\) are 820, 790, and 770, respectively. For this comparison, assume the sample units are independent and use \(\mathrm{BIC}=-2\ell+q\log(n)\).
For \(K=1\), \(\log(120)\approx4.787\), so
For \(K=2\), \(\mathrm{BIC}_2=790+30(4.787)=933.6\). For \(K=3\), \(\mathrm{BIC}_3=770+40(4.787)=961.5\). Although the raw fit improves at each step, BIC is smallest for \(K=1\). On this criterion, the fit gains do not compensate for the added complexity. This is evidence favoring the smaller candidate under the stated likelihood and parameter-count assumptions; it does not prove that the data have exactly one shared biological factor.
Choosing Shared and View-Specific Counts
The shared count and the specific counts answer different questions. Increasing \(K\) adds directions intended to explain coordinated variation across views. Increasing \(L_m\) adds directions intended to explain variation in view \(m\) that is not represented by the shared component. A useful comparison therefore considers a grid or a deliberate set of candidate tuples, rather than choosing one total rank and assuming how it should be divided.
Start with candidates that are scientifically plausible and computationally estimable. For each, record fit by view, remaining residual patterns, and the behavior of the added factors across repeated fits or resamples. Interpretability is supporting evidence, not a license to select whichever model tells the most compelling story. A plausible factor should have coherent loadings, a meaningful score pattern, and a clear reason for being shared or specific; it should not be defined only by a convenient label.
Worked Example: Separating a Shared Factor From a View-Specific Factor
Consider two hypothetical standardized views, transcript and protein, measured in the same 90 samples. The following summaries are from three fitted candidates. Residual sums of squares are in standardized squared units; the residual cross-view covariance is a diagnostic summary, not a likelihood.
| Candidate | Counts \((K,L_{\mathrm{transcript}},L_{\mathrm{protein}})\) | Transcript residual sum of squares | Protein residual sum of squares | Residual cross-view covariance |
|---|---|---|---|---|
| A | (1, 0, 0) | 18 | 16 | 0.42 |
| B | (1, 1, 0) | 12 | 16 | 0.08 |
| C | (2, 1, 0) | 10 | 13 | 0.05 |
Candidate B adds a transcript-specific factor. The transcript residual sum of squares drops by \(18-12=6\), while the protein residual sum of squares is unchanged. The residual cross-view covariance also falls from \(0.42\) to \(0.08\). This pattern is consistent with the added factor capturing transcript variation without claiming that it is shared. Whether to retain it depends on its complexity cost and whether its loadings and scores are sufficiently coherent and stable to interpret.
Candidate C adds a second shared factor as well. Relative to B, its residual sums of squares decrease by \(12-10=2\) in transcript and \(16-13=3\) in protein. The cross-view residual covariance decreases by a further \(0.03\). These gains may or may not justify the additional shared dimension. If the second shared factor has weak, unstable loadings, the small improvement is not compelling evidence for it. The summaries alone do not determine a winner; a likelihood criterion, residual checks, stability, and the scientific purpose must be considered together.
A Practical Selection and Diagnostic Sequence
The following process keeps fit comparison connected to the structure that the model is intended to represent. It assumes that samples have been matched appropriately across views, that transformations and observation models are defensible, and that the fitting procedure has produced usable candidate solutions.
Choose plausible values of \(K\) and \(L_m\), allowing specific counts to differ by view. Avoid a search range so large that many candidates are not estimable with the available samples.
Use the same data and likelihood specification across candidates. Report a joint fit summary and view-level residual summaries so one view does not silently determine the decision.
Calculate an information criterion using an appropriate count of estimable parameters, or use another explicit complexity penalty. Note whether the criterion's assumptions are approximate for the fitted model.
Look for structured residual patterns, including residual cross-view association. A remaining pattern suggests possible misspecification or insufficient rank, but does not by itself identify which factor type should be added.
Assess loading coherence, score behavior, and sensitivity to resampling or reasonable modeling choices. Ask whether an added direction is reproducible and appropriately shared or view-specific.
When several candidates perform similarly, present the plausible range and explain which conclusions are robust across it rather than implying that one selected rank is known exactly.
Residual diagnostics are particularly useful when a low-complexity model leaves systematic structure. For example, residual correlation across two views can indicate that the shared component is too restricted, or that the observation model or preprocessing is inadequate. A residual feature cluster confined to one view may motivate considering a view-specific direction. Neither pattern is conclusive: the apparent structure could also arise from technical effects, outliers, or an incorrect residual distribution.
Underfitting, Overfitting, and Alternatives
Underestimating rank can leave correlated residuals, blur distinct sample profiles into one factor, and force view-specific variation into shared factors or vice versa. This can distort the interpretation of cross-omics coordination. Overestimating rank can split one coherent direction into several weak components, fit noise, and produce factors whose loadings or scores change substantially under resampling. Because factor labels can change sign, order, or orientation, stability should be judged using appropriately aligned patterns rather than by comparing raw component labels alone.
| Approach | What it contributes | Main caution |
|---|---|---|
| In-sample likelihood or residual fit | Shows how well each candidate represents the observed data. | Usually favors added flexibility and does not alone select a useful rank. |
| AIC or BIC | Penalizes fit improvement according to model complexity. | Depends on likelihood assumptions and parameter counting; small differences are not decisive. |
| Residual and interpretability checks | Reveal leftover structure and whether added factors have coherent meaning. | Can be subjective and sensitive to preprocessing or scientific expectations. |
| Predictive criteria | Compare how candidate ranks perform on data not used for fitting. | Require a validation design that respects the data structure; they answer a different question from in-sample fit. |
A common mistake is to choose the rank with the lowest residual sum of squares and call every resulting component a discovered biological factor. The correction is to recognize that residual fit typically improves as flexibility increases, then ask whether the gain survives a complexity penalty, whether residual structure is meaningfully reduced, and whether the added factor is stable and interpretable. Conversely, a stringent penalty should not be used mechanically to dismiss a factor when residual diagnostics show important structure the model fails to capture.
No criterion guarantees a uniquely correct rank. Rank selection is limited by sample size, correlated features, residual-model assumptions, and the difficulty of separating shared from view-specific structure. Candidate models may also differ in optimization quality; an apparent fit deficit can reflect a failed fit rather than an unsuitable rank. Check that comparisons are not driven by poor convergence, preprocessing artifacts, or one unusually influential sample. When conclusions change sharply across nearby ranks, report that sensitivity rather than hiding it.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Why does the lowest in-sample residual sum of squares generally not determine the preferred factor count?
- For \(n=100\), a candidate adds 8 estimable parameters and improves \(-2\ell\) by 30. Does BIC favor the larger candidate, all else equal?
- A new factor reduces residual variation in only one omics view. What count does it most directly motivate considering, and what else should be checked?
- Name two warning signs of overestimating rank and one sign of underestimating rank.
- Why should fit comparisons use the same samples, features, preprocessing, and likelihood specification?
Solutions. (1) More flexible models generally fit the observed data at least as well, even when added dimensions capture noise; complexity and interpretation must also be considered. (2) The BIC penalty is \(8\log(100)\approx8(4.605)=36.84\). The improvement of 30 is smaller than 36.84, so BIC increases by about \(6.84\) and favors the smaller candidate. (3) It most directly motivates considering a view-specific factor for that view. Check that the residual pattern is structured, that the factor is stable and coherent, and that the fit gain justifies the added complexity. (4) Weak or unstable added factors and factors that split one coherent pattern are overfitting warnings; systematic residual structure is an underfitting warning. (5) Otherwise a fit difference could reflect changes in the data or likelihood rather than the factor count, making the comparison uninterpretable.