Within-Between Decomposition

Why Split a Predictor by Cluster?

In nested data, a level-1 predictor varies across observations and may also differ systematically from one cluster to another. For example, patients within hospitals may have different ages, while the average age may also vary across hospitals. A single raw predictor combines these two sources of variation. Within-between decomposition separates them arithmetically: each observation is represented by its cluster mean and its deviation from that mean.

This split is useful in multilevel analyses because it makes clear which part of a predictor represents differences among observations in the same cluster and which part represents differences among cluster averages. The decomposition itself is an identity, not a model and not a causal claim. It does not, by itself, determine how either component should enter an analysis.

The grand mean across all observations is \(\bar{x}=\frac{1}{N}\sum_j\sum_i x_{ij}\), where \(N=\sum_j n_j\). When cluster sizes differ, this is a size-weighted average of the cluster means: \(\bar{x}=\frac{1}{N}\sum_j n_j\bar{x}_j\). It is generally not the unweighted average of those means.

The Sum-of-Squares Decomposition

Suppose the clusters form a disjoint and exhaustive partition of the observations, and the predictor is available for every observation included in the calculation. The total sum of squared deviations from the grand mean can then be partitioned into a within-cluster sum and a between-cluster sum. The between-cluster component weights each squared difference between a cluster mean and the grand mean by that cluster’s size.

The first term on the right is the within-cluster sum of squares: it measures variation among observations around their own cluster means. The second is the between-cluster sum of squares: it measures variation in cluster means around the observation-weighted grand mean. These are sums of squares in the original predictor’s units squared.

Why the Identity Holds

For each observation, add and subtract its cluster mean. Then square the resulting two-part expression and sum it over observations:

Within each cluster, \(\bar{x}_j-\bar{x}\) is constant. Also, the deviations from the cluster mean sum to zero: \(\sum_i(x_{ij}-\bar{x}_j)=0\). Therefore, the cross-product term is zero in every cluster. The remaining middle term repeats \((\bar{x}_j-\bar{x})^2\) exactly \(n_j\) times, giving \(n_j(\bar{x}_j-\bar{x})^2\). This proves the stated identity.

No normality or independence assumption is needed for this arithmetic identity. The requirements are about the data being partitioned and measured consistently: each observation belongs to one cluster, every included observation has a predictor value, and each cluster mean is calculated from the observations designated as belonging to that cluster. Statistical models built using these components may require additional assumptions.

Calculating the Components Observation by Observation

The decomposition can also be used to construct two columns for each observation. The within-cluster column contains \(x^W_{ij}=x_{ij}-\bar{x}_j\). A between-cluster column can contain \(\bar{x}_j\), repeated for each observation in cluster \(j\), or the centered cluster mean \(\bar{x}_j-\bar{x}\). The former keeps the cluster mean on the predictor’s original scale; the latter expresses it as a difference from the grand mean.

For the previous example, observation A’s value 1 has a within-cluster deviation of \(1-4=-3\), and its cluster mean is 4. Its decomposition is \(1=4+(-3)\). Observation B’s value 12 has a deviation of \(12-10=2\), so \(12=10+2\). The within deviations sum to zero within each cluster; that is a useful check on the calculations.

Unequal Sizes and Alternative Centering Choices

Unequal cluster sizes matter whenever the target is an average across observations. The grand mean weights each cluster mean by \(n_j\), because a cluster with more observations contributes more values to the overall average. An unweighted average of cluster means instead gives each cluster equal influence, regardless of size; it answers a different averaging question.

Subtracting the grand mean from every observation, \(x_{ij}-\bar{x}\), is called grand-mean centering. It shifts the origin but leaves the predictor as a mixture of within- and between-cluster variation. Cluster-mean centering, \(x_{ij}-\bar{x}_j\), isolates within-cluster deviations, while the cluster means themselves retain between-cluster variation. Cluster indicators, discussed in the earlier material on Cluster Indicators and Their Limits, represent clusters with separate indicator variables; they are a different way to represent cluster membership, not a replacement definition for these two components.

Checks, Pitfalls, and Limits

Several quick checks catch common arithmetic or data-definition errors:

• Within each cluster, the deviations should sum to zero, subject to small rounding error.
• The weighted cluster means should reproduce the overall mean: \(\sum_j n_j\bar{x}_j/N=\bar{x}\).
• The within and between sums of squares should add to the directly calculated total sum of squares.
• When using rounded means, retain adequate precision during intermediate calculations and round only the reported result.

A common mistake is to treat each cluster mean as one equally weighted observation when computing the grand mean or the between-cluster sum of squares. That ignores cluster size. In the first worked example, A and C both have mean 4, but C has four observations and A has three; their between contributions are \(64/9\) and \(16/3\), respectively. The correct weighting is by \(n_j\), not by the number of clusters alone.

Another mistake is to conclude that a large within or between sum of squares identifies an important effect. These quantities describe how the observed predictor’s variation is partitioned; their size depends on the predictor’s scale and the number of observations. They do not establish an association, a causal effect, or the suitability of a particular model. Nor does decomposition alone account for uncertainty in estimated cluster means.

For incomplete data, the arithmetic requires a defined set of predictor values for each calculation. Do not silently calculate some cluster means from observed values and others from completed values, or change the cluster membership rule partway through. In an imputation workflow, components derived from a predictor must be calculated consistently with the analysis dataset and the intended definition of the cluster mean. This tutorial defines the decomposition; it does not prescribe a model for imputing missing values or resolve the inferential consequences of estimating cluster means.