Diagnosing ComBat Fit and Residual Batch Structure

Why Diagnose a ComBat Adjustment?

ComBat estimates and adjusts known batch effects while allowing specified biological covariates to be retained. A completed adjustment is not, by itself, evidence that the fit is appropriate. An omitted biological covariate can be mistaken for batch, a poor design can make effects unidentifiable, and a model that fits imperfectly can leave batch structure in adjusted expression. Conversely, an adjustment can reduce a batch pattern by removing biological variation that should have been preserved.

The diagnostic task is therefore two-sided: look for residual batch structure after adjustment, and check whether the biological variation represented in the model remains visible. The introductory material on ComBat Empirical Bayes Batch Adjustment establishes the adjustment model and its empirical Bayes estimation; here the emphasis is on checking its specification and fit. Diagnostics are evidence about the fit, not a guarantee that all unwanted variation has been removed.

What the Diagnostics Assume

ComBat is applied to expression values on a suitable continuous, approximately Gaussian scale, commonly after appropriate normalization and transformation. It is not a procedure to apply directly to untransformed RNA-sequencing counts. For feature \(i\) in sample \(j\), a useful representation of the model is

Here \(Y_{ij}\) is the transformed expression value, \(X_j\) contains the biological covariates to preserve, \(b(j)\) is the batch containing sample \(j\), \(\gamma_{i,b(j)}\) is the feature-specific batch location effect, and \(\delta_{i,b(j)}\) is a positive batch scale factor. The error \(\epsilon_{ij}\) is modeled as having mean zero and variance one. The covariate and batch terms must be estimable from the observed design. In particular, the biological groups of interest cannot be completely confounded with batch if their effects are to be distinguished from batch effects.

In the standard parametric empirical Bayes version, standardized feature-level batch location effects within a batch are modeled with a normal prior, and the squared batch scale factors, or batch variance factors, are modeled with an inverse-gamma prior. These are assumptions across features that support shrinkage; they do not assert that every feature has identical batch effects. Diagnostics should assess whether the data appear broadly compatible with the working model and whether the adjustment behaves plausibly.

Before interpreting plots, check that samples have correctly assigned batch labels, that the covariates are recorded accurately, and that the expression matrix has been normalized and transformed appropriately. Confirm that the model matrix is full rank and that each biological group has representation across batches sufficient to distinguish group from batch. Independence of biological samples is also important: repeated or paired measurements need a design that reflects their dependence, rather than being treated as unrelated samples.

Check the Design Before the Plots

A covariate should be included because it represents biological variation that the analysis intends to retain, not merely because it is associated with the observed expression. Omitting a relevant group can cause ComBat to treat some of its expression differences as batch effects. Including a covariate that is redundant with batch does not solve confounding: the data still cannot determine which factor explains the difference.

For a quick design check, tabulate sample counts by batch and biological group, then inspect the model matrix for linear dependence. A crossed design, in which groups occur in multiple batches, generally provides more information to separate their effects than a design in which each batch contains only one group. Severe imbalance can still make estimates unstable even when the design is technically identifiable.

Compare Expression Distributions Before and After Adjustment

Batch-stratified plots provide a direct check of feature-level behavior. For selected features, plot expression by batch before and after adjustment, ideally with biological groups indicated within each batch. Boxplots or dot plots can reveal persistent differences in location, spread, skewness, or outliers. Looking across a representative set of features is more informative than selecting only features that show the desired result.

Also inspect distributions across samples, such as sample-wise expression summaries and density plots, before and after adjustment. A substantial change in location or spread is not automatically a failure: removing estimated batch effects is intended to change these features. Instead, ask whether the adjusted values are plausible, whether batch differences have been reduced without erasing group contrasts, and whether a few samples or features are driving the apparent improvement.

Use Principal Components as a Complementary Check

Principal components summarize major directions of variation across samples. Plotting samples using the first two or three components of the expression matrix before and after adjustment can show whether batch-associated clustering has weakened. Color points by batch and use a separate symbol or panel for the biological group. If adjusted samples still separate strongly by batch, that is a reason to investigate residual batch structure; if biological groups that were separated before adjustment collapse together, investigate whether meaningful biology was removed.

Principal components are summaries, not batch-effect tests. A batch pattern may remain in later components, and an apparent pattern in a low-dimensional plot may be driven by a small number of features or outliers. Conversely, overlapping batch colors on the first two components do not establish that all batch effects are absent. Interpret the plot together with feature-level checks and the study design.

Why the Adjustment Works Under Its Model

The standardization and adjustment clarify what a successful fit would do. Let \(\hat{\alpha}_i\), \(\hat{\beta}_i\), and \(\hat{\sigma}_i\) denote the fitted biological mean components and pooled feature scale used to form standardized values. For sample \(j\), define

If the fitted biological mean and pooled scale, as well as the batch parameters, equal their true values, then substituting the model gives \(z_{ij}=\gamma_{i,b(j)}+\delta_{i,b(j)}\epsilon_{ij}\). With the true batch location and scale, removing the location and dividing by the scale yields the standardized error:

The qualification about the biological mean and pooled scale matters. Correct batch parameters alone do not imply this equality. For example, if the fitted biological mean is too large by \(c\), while the pooled scale is correct, then the standardized value is shifted by \(-c/\sigma_i\). That discrepancy can remain after batch location and scale correction. In practice the parameters are estimated, so residual patterns can arise from model misspecification, estimation error, or both.

Common Failure Modes and Alternatives

• Reading batch overlap as proof of success. Overlap in a two-component PCA plot can hide residual structure elsewhere. Check multiple summaries and features.
• Ignoring biological composition. Batch means can differ because group proportions differ. Compare groups within batches where possible and retain appropriate biological covariates.
• Using an incomplete covariate model. An omitted biological factor may be absorbed into the estimated batch effect. Review the study question and design before interpreting an adjusted contrast.
• Assuming shrinkage fixes confounding. Empirical Bayes estimation stabilizes batch-effect estimates across features under its prior assumptions; it cannot recover effects that the design does not identify.
• Treating transformed values as counts. ComBat’s continuous-expression model is not a count model. Use the data scale and method appropriate to the analysis.

Alternative approaches answer different questions. Leaving data unadjusted avoids model-based changes but leaves known batch structure in the analysis. A feature-wise linear adjustment for known batch effects can be useful, but it does not provide ComBat’s across-feature empirical Bayes shrinkage. ComBat itself is appropriate only when its design and working assumptions are suitable. Principal components are useful diagnostics, not a replacement correction procedure. The choice should be guided by the design and the intended downstream analysis, not by which method produces the most visually attractive plot.