Comparing Location and Spread Without Overrelying on Means

Why Compare More Than Means?

A mean and variance are useful summaries, but they do not describe every feature of a distribution. Two sets of values can have the same mean while having different typical values or different central spreads. Conversely, a small number of unusually distant values can strongly affect a mean or variance while leaving the median and central quantiles nearly unchanged. Comparing several summaries helps reveal these differences when assessing continuous values observed in the data and values generated by MICE.

This comparison is descriptive, not a test that imputed values must match observed values exactly. The observed cases and cases with missing values may differ in their predictors or in other ways related to missingness. A marginal discrepancy can therefore reflect differences between the groups as well as a problem with the imputation model. The comparison is most informative when the summaries refer to the same variable, units, and intended target population, and when relevant differences in case composition are considered.

Definition: For a set of quantitative values, the median is the middle value after sorting, or the average of the two middle values when the number of values is even. The first and third quartiles, \(Q_1\) and \(Q_3\), describe the lower and upper parts of the distribution. The interquartile range is \(IQR=Q_3-Q_1\). The median absolute deviation (MAD) is the median of the absolute distances from the median: \(MAD=\operatorname{median}(|x_i-\operatorname{median}(x)|)\). Unless stated otherwise, MAD here is unscaled.

Quartile values can depend on the convention used, especially in small samples. In the calculations below, when the sample size is even, split the sorted values into lower and upper halves and take the median of each half as \(Q_1\) and \(Q_3\). When the sample size is odd, omit the overall median before taking the medians of the lower and upper halves. Use the same convention for the observed and imputed sets being compared, and state it when reporting results.

What the Summaries Reveal

The mean and sample variance summarize location and spread using squared deviations. The median describes the center by rank, while the IQR describes the width of the middle half of the values. The MAD measures a typical absolute distance from the median. These measures are on the original measurement scale: the IQR and MAD have the same units as the variable, whereas the sample variance has squared units.

The median, IQR, and MAD are less sensitive than the mean and variance to the magnitude of a few extreme values. They are not immune to changes in the distribution: moving many observations can change them, and small samples can make their values coarse or unstable. They also do not reveal every feature. For example, two distributions can share a median and IQR yet differ in shape. Use these summaries as companions to the distributional displays and chain diagnostics already introduced, not as replacements for them.

Takeaway: Compare location with at least one center measure and spread with at least one spread measure. A useful descriptive set is the mean and sample variance alongside the median, IQR, and MAD, all reported in context and with units.

Worked Comparisons

Worked Example: Equal Centers, Different Spread

In a hypothetical dataset, suppose the observed values of a continuous measure are \(8,9,10,10,11,12\) units, while six imputed values are \(4,7,10,10,13,16\) units. Compare their centers and spreads. For this example, use the median-of-halves quartile convention defined above.

Centers. Both sorted sets have middle values 10 and 10, so each median is \((10+10)/2=10\) units. Each mean is also 10 units: the observed sum is 60 and the imputed sum is 60, and \(60/6=10\).

Observed quartiles and IQR. The lower half is \(8,9,10\), whose median is \(Q_1=9\). The upper half is \(10,11,12\), whose median is \(Q_3=11\). Thus \(IQR=11-9=2\) units.

Imputed quartiles and IQR. The lower half is \(4,7,10\), so \(Q_1=7\); the upper half is \(10,13,16\), so \(Q_3=13\). Thus \(IQR=13-7=6\) units. The imputed IQR is three times the observed IQR, despite the equal means and medians.

Check against sample variance. For the observed values, the squared deviations from 10 sum to \(4+1+0+0+1+4=10\), so \(s^2=10/(6-1)=2.0\) square units. For the imputed values, they sum to \(36+9+0+0+9+36=90\), so \(s^2=90/(6-1)=18.0\) square units. The MADs are 1 unit for the observed values and 3 units for the imputed values: the absolute deviations are respectively \(2,1,0,0,1,2\) and \(6,3,0,0,3,6\), whose medians are 1 and 3. All three spread measures indicate greater spread among these imputed values, while the mean and median alone would miss it.

Worked Example: Equal Means, Different Typical Values

Consider two hypothetical sets of eight values, in minutes. The observed values are \(0,1,2,3,4,5,6,11\), and the imputed values are \(1,2,3,4,4,4,6,8\). Calculate the means, medians, and IQRs.

Both sums are 32 minutes, so both means are \(32/8=4\) minutes. The observed median is the average of the fourth and fifth values, \((3+4)/2=3.5\) minutes. The imputed median is \((4+4)/2=4\) minutes.

For the observed set, the lower half is \(0,1,2,3\), giving \(Q_1=(1+2)/2=1.5\) minutes. The upper half is \(4,5,6,11\), giving \(Q_3=(5+6)/2=5.5\) minutes. Therefore \(IQR=5.5-1.5=4\) minutes.

For the imputed set, the lower half is \(1,2,3,4\), so \(Q_1=(2+3)/2=2.5\) minutes. The upper half is \(4,4,6,8\), so \(Q_3=(4+6)/2=5\) minutes. Therefore \(IQR=5-2.5=2.5\) minutes. The matching means conceal a 0.5-minute difference in median and a 1.5-minute difference in IQR. The MADs also differ: the observed absolute deviations from 3.5 have middle pair 1.5 and 2.5, giving MAD \(=2\) minutes; the imputed absolute deviations from 4 have middle pair 1 and 2, giving MAD \(=1.5\) minutes.

Why Robust Summaries Respond Differently

The difference in sensitivity can be justified directly. If one value \(x_i\) is replaced by a very large value \(M\), the mean changes by \((M-x_i)/n\), which grows without bound as \(M\) increases. Its squared deviation from the mean also grows on the order of \(M^2\), so the sample variance can increase sharply. By contrast, the median depends on the ordering of the values: making an already largest value even larger does not change the middle rank. The IQR and MAD are also based on ranks and distances among central values, so an extreme value can have little or no effect if it does not change the relevant central order statistics.

A related justification is that a median minimizes the sum of absolute distances from a candidate center \(a\), \(\sum_i |x_i-a|\). As \(a\) moves from left to right across the sorted data, each value below \(a\) contributes slope \(+1\) and each value above \(a\) contributes slope \(-1\). The total slope changes sign at the middle of the ordered values, so a median is a minimizer. This rank-based balance explains why a distant observation does not pull the median as strongly as it pulls the mean.

\bar{x}_{\mathrm{new}}-\bar{x}_{\mathrm{old}}=\frac{M-x_i}{n}, \qquad \text{while the median is determined by the middle order statistic(s).}

This is a sensitivity distinction, not a claim that medians or quantiles are always preferable. If the scientific estimand is a mean, a discrepancy in means matters even when the medians agree. If predictions at the center or across the central range are important, medians and quantiles add information that moments do not provide.

Worked Example: Comparing Imputation Sets Without Pooling Them

Suppose a hypothetical analysis has eight observed values \(10,11,12,12,13,14,15,16\) units, and two completed datasets produce these values for the corresponding missing entries: dataset A has \(9,11,12,12,13,14,15,16\), and dataset B has \(7,10,12,12,13,14,16,19\). Compare the robust summaries separately for each imputation set.

For the observed values, the median is \((12+13)/2=12.5\) units. The lower and upper halves are \(10,11,12,12\) and \(13,14,15,16\), so \(Q_1=11.5\), \(Q_3=14.5\), and \(IQR=3\) units.

For dataset A, the median is also \((12+13)/2=12.5\) units. Its halves are \(9,11,12,12\) and \(13,14,15,16\), so \(Q_1=11.5\), \(Q_3=14.5\), and \(IQR=3\) units. For dataset B, the median is likewise 12.5 units. Its halves are \(7,10,12,12\) and \(13,14,16,19\), so \(Q_1=(10+12)/2=11\), \(Q_3=(14+16)/2=15\), and \(IQR=4\) units.

The medians coincide, but the IQR in dataset B is wider than the observed IQR and dataset A's IQR. The MADs give a similar comparison. The observed and dataset A absolute deviations from 12.5 both have middle pair 1.5 and 1.5, so each MAD is 1.5 units. Dataset B's sorted absolute deviations are \(0.5,0.5,0.5,1.5,2.5,3.5,5.5,6.5\), giving MAD \(=(1.5+2.5)/2=2\) units.

These calculations describe each set; they do not establish that the model has failed. With only eight values, the summaries are imprecise descriptions, and the two imputation sets may differ partly because of imputation variability. In practice, examine summaries across completed datasets or chains rather than combining all imputed values as if they were independent observed cases. Also check that the chains have adequate mixing and stability before interpreting their summaries.

Assumptions, Diagnostics, and Limitations

These summaries require quantitative values on a common meaningful scale, a consistent unit, and a clear definition of which observed and imputed records are being compared. Their interpretation also depends on the comparison being relevant: observed values are not automatically a perfect reference distribution for cases whose values are missing. For example, if missingness is associated with predictors, the missing cases can have a different expected marginal distribution from observed cases even when the imputation model is appropriate. When possible, compare within relevant predictor groups or otherwise account for case composition before treating a marginal gap as evidence of model misfit.

A practical diagnostic is to look for a coherent pattern across several summaries. A changed median with similar IQR suggests a location difference in the central part of the distribution. A similar median with a larger IQR or MAD suggests greater central spread. A mean difference without a median difference can indicate that values away from the center are influencing the mean, although these summaries alone do not identify why. Use quantile, density, empirical CDF, boxplot, or violin displays alongside the numerical comparison. Review chain behavior as well: unstable or poorly mixing chains make a single set of summary values a weak basis for judgment.

There is no universal numerical threshold at which an observed–imputed difference becomes unacceptable. Sample size, missingness proportion, uncertainty, substantive context, and intended use all matter. Small samples can produce tied quantiles, and robust summaries can be insensitive to distribution differences outside the central range. Nor should a favorable match of means, medians, and spreads be treated as proof that the imputation model is correctly specified: distinct distributions can share those summaries.

Define the comparison.
Name the variable, its units, the records included, and whether each summary is calculated within an imputation dataset or chain.

Calculate complementary summaries.
Report mean and variance alongside median, IQR, and, when useful, MAD. Apply the same quantile convention to both groups.

Interpret with context.
Describe the size and direction of differences in units, then consider missingness-related group differences, chain diagnostics, and distribution plots before diagnosing a model problem.

Common Pitfalls

Relying on the mean alone. Equal means do not imply equal medians or spreads. Compare center and spread using complementary summaries.
Mixing quartile conventions. Different conventions can give different quartiles in small samples. State the convention and use it consistently. For six values, the halves each contain three values; their medians are \(Q_1\) and \(Q_3\).
Interpreting a marginal gap as automatic model failure. Observed cases and cases with missing values may differ. Consider relevant predictors and the target of the comparison before drawing a conclusion.
Pooling imputed values as independent data. Imputations represent alternative completed datasets, not extra independent people. Summarize each completed dataset or chain separately when diagnosing their behavior.
Treating robust summaries as a complete distribution check. Matching medians and IQRs does not guarantee matching shapes. Retain graphical checks and other diagnostics.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For the sorted values \(2,4,5,7,8,10\), calculate the median, \(Q_1\), \(Q_3\), and IQR using the median-of-halves convention.
Two sets each have mean 20 units. What additional summaries would help determine whether their typical centers and central spreads differ?
Why can an observed–imputed difference in marginal median occur even if the imputation is reasonable?
What is one reason to calculate summaries separately for different completed datasets rather than treating all imputed values as independent observations?

Solutions. (1) The median is \((5+7)/2=6\). The halves are \(2,4,5\) and \(7,8,10\), so \(Q_1=4\), \(Q_3=8\), and \(IQR=8-4=4\) units. (2) Compare medians for typical center and IQR or MAD for central spread, using the same units and quantile convention; the mean and variance can be included as complementary summaries. (3) The observed cases and cases with missing values may differ in predictors related to the variable, so their marginal distributions need not match. Consider case composition and relevant predictor groups before interpreting the gap. (4) Different completed datasets reflect imputation variability; combining their values as independent observations misrepresents the data structure and can hide between-imputation differences.