Comparing Feature Distributions Before and After Correction

Why Compare Feature Distributions?

A batch adjustment is intended to reduce technical differences between batches while preserving variation relevant to the scientific question. A practical diagnostic is to compare the distribution of each feature across batches both before and after adjustment. Such comparisons can reveal a residual location offset, a difference in spread, or a more complex shape difference that a single average would miss.

The comparison is feature-specific: a feature measured in two batches is examined across the subjects in each batch. It is not enough to compare one overall distribution per assay or to inspect only a multivariate summary. A correction can make one feature’s batch distributions more similar while leaving another feature’s distributions different.

For a feature, useful summaries include the batch-specific sample size, mean, median, standard deviation, and interquartile range (IQR). Boxplots display medians, quartiles, and potential extreme values compactly; density plots display estimated distribution shapes. The different summaries answer different questions. Means and medians describe location, standard deviations and IQRs describe spread, and plots can expose skewness, multiple modes, or tails.

Comparisons require a meaningful common measurement scale within a feature. When comparing before with after, use the same subjects and clearly identify the values as unadjusted or adjusted. Across different features, units and scales may differ; a difference of one unit in one feature need not be comparable to a difference of one unit in another.

What Similarity Does—and Does Not—Mean

If batches differ mainly by a technical additive offset, their distributions may have similar shapes and spreads but different centers. A residual shift in the medians or means after adjustment is evidence of a remaining location difference. If centers are similar but the IQRs or standard deviations differ, the remaining discrepancy is primarily in scale. Differences in skewness, tails, or modality indicate shape differences that neither location nor scale summaries fully capture.

The expected degree of similarity depends on the data and the scientific design. If batches contain comparable subject populations and the adjustment is intended to remove batch-associated distributional differences, substantial residual differences deserve investigation. But if biological composition differs across batches, marginal distributions can differ for biological as well as technical reasons. For example, a feature may be higher in a batch because that batch contains more subjects from a biological subgroup. Making the marginal distributions identical could then remove meaningful variation. Distributional similarity is therefore not a universal target independent of study design.

Location and Scale Summaries

Let \(n_b\) be the number of observed subjects in batch \(b\) for a feature. Its batch-specific sample mean is the sum of the measured values divided by \(n_b\); its sample median is the middle value (or the average of the two middle values for an even sample size). The IQR is the third quartile minus the first quartile. The sample standard deviation \(s_b\) measures spread around the sample mean using the squared deviations and denominator \(n_b-1\).

For before-and-after assessment, calculate these summaries separately for each batch at each stage. Report changes in context and in the feature’s units. For example, a post-adjustment difference of 0.8 units in the batch medians is a residual location difference of 0.8 feature units; it is not automatically a practically important difference without reference to the feature and its scale.

The boxplot is a compact view of location and spread. A boxplot whose median lines differ but whose box widths are similar suggests a location shift with roughly similar central spread. Similar medians with substantially different box widths suggest a scale difference. Density plots add shape information, although their appearance depends on the smoothing bandwidth and can be unstable when sample sizes are small.

Why Pooled Summaries Can Mislead

A pooled standard deviation can be large because values vary within batches, because batch means differ, or both. The following finite-sample identity makes this distinction explicit. Let \(N=\sum_b n_b\), let \(\bar{x}\) be the overall sample mean, and let \(\bar{x}_b\) be the mean in batch \(b\). The sum of squared deviations can be decomposed by writing each deviation as \((x_{ib}-\bar{x}_b)+(\bar{x}_b-\bar{x})\). The cross-product sums to zero within each batch because \(\sum_i(x_{ib}-\bar{x}_b)=0\). Thus:

The first term is within-batch variation; the second is between-batch variation due to different batch means. Dividing by \(N-1\) gives the pooled sample variance. This identity does not assume normality or equal batch sizes. It explains why a feature can have a wide pooled distribution even when each batch has modest within-batch spread: separated batch centers contribute to the total.

A Practical Diagnostic Sequence

A consistent sequence makes comparisons interpretable and reduces the risk of relying on an attractive but incomplete plot.

Diagnostics, Alternatives, and Limitations

A plot or summary can mislead when there are few subjects per batch, highly unequal batch sizes, or extreme observations. Boxplots may conceal multimodality and differences in tails; density estimates can create or erase apparent modes depending on bandwidth. Always inspect sample sizes, consider plotting individual observations when feasible, and compare multiple summaries rather than treating one statistic as decisive.

Feature-wise comparisons are also not a replacement for multivariate diagnostics. They reveal marginal distributions, one feature at a time, but do not show whether the joint relationships among features changed. PCA provides a multivariate view of patterns across samples; it answers a different question and can obscure feature-specific discrepancies. Formal tests of distributional differences can quantify evidence under their assumptions, but they may detect small differences in large samples, miss relevant differences in small samples, and create a multiple-testing problem when many features are tested. A test result should not replace inspection of effect size, units, plots, and study design.

A further limitation is confounding between batch and biology. If a biological group occurs only in one batch, a feature’s batch distributions cannot by themselves separate a batch effect from a group effect. In that setting, apparent residual differences may reflect composition, while apparent alignment may reflect removal of real group variation. These diagnostics should be interpreted alongside the experimental design and the covariates the adjustment was intended to preserve.

Common Mistakes

• Wrong reasoning: “The post-adjustment batch means match, so the distributions match.” Correction: Equal means do not imply equal spread or shape; compare medians, IQRs, standard deviations, and plots.
• Wrong reasoning: “Every feature should have identical batch distributions after adjustment.” Correction: That is not necessarily appropriate when batch composition differs biologically or when the procedure is intended to preserve biological covariates.
• Wrong reasoning: “A remaining difference proves the correction failed.” Correction: The difference is descriptive; it may be technical, biological, or both. Interpret it in the context of design and adjustment goals.
• Wrong reasoning: “Pooled variability summarizes the batches adequately.” Correction: Pooled variation combines within-batch spread and between-batch location differences, so inspect batches separately.