Choosing Resampling Units for Multivariate Data

Why the Resampling Unit Matters

A multivariate observation is often more than a row of numbers. It may be a vector of correlated outcomes, a set of repeated measurements on one person, or an entire tensor recorded from one experimental unit. Resampling individual entries can break the relationships that the analysis is meant to describe. Resampling whole rows can also be wrong when rows from the same cluster or nearby times are dependent.

The key decision is therefore not simply how many observations to draw. It is which pieces of the data can be treated as independent replicates for the inferential target at hand? A suitable resampling unit preserves dependence within each piece while respecting the independence, or other structure, across pieces.

These terms describe different roles, not necessarily different physical objects. A patient can be a case in a study with one record per patient, while a clinic can be the resampling unit if patients within a clinic share a source of dependence. In a longitudinal study, a patient may be the independent case and the patient’s full repeated-measurement vector the object to retain together.

Cases: Preserve the Whole Multivariate Record

Suppose \(U_i=(X_{i1},\ldots,X_{ip})^\mathsf{T}\) is the vector of \(p\) measurements on case \(i\). If the cases are independent and identically distributed, the natural empirical resampling scheme draws whole vectors \(U_i\), not separate values from each coordinate. This retains the observed association among measurements within a case.

The distinction matters even when the inferential target is a simple vector of means. For example, jointly measured blood pressure and heart rate may covary across patients. A resampling scheme that independently selects blood pressures and heart rates constructs combinations that may not reflect the joint distribution in the study population.

To verify the result, conditional on the data, \(U^*\) takes each value \(U_i\) with probability \(1/n\), so \(E[U^*\mid U_1,\ldots,U_n]=\bar U\). Substituting this conditional mean into the covariance definition gives

Under coordinate-wise selection, write the selected coordinates as \(X_j^*\) and \(X_k^*\). For \(j\ne k\), those selections are conditionally independent, so

Thus, drawing whole cases retains the empirical joint covariance, whereas independent coordinate draws erase it. This is a statement about the dependence encoded by the resampled observations; it does not by itself establish that a particular resampling approximation is accurate for every statistic or sample size. \(\square\)

Clusters: Keep Dependent Members Together

In clustered data, the rows are not necessarily independent cases. Patients in one clinic, pupils in one school, or repeated observations on one person can share unmeasured conditions. When distinct clusters are independent and sampled from the relevant population, the cluster—not an individual row within it—is the natural independent resampling unit for targets based on the cluster sampling process.

The target must be stated carefully. Giving each cluster equal weight targets an average over clusters. Giving each person equal weight targets an average over people, which can differ when cluster sizes vary. Resampling clusters does not settle this weighting choice; the statistic calculated on the resampled data must implement the intended target.

For the derivation, write \(\bar Y=G^{-1}\sum_{g=1}^G\bar Y_g\), where \(\bar Y_g=m^{-1}\sum_{j=1}^mY_{gj}\). Within cluster \(g\), this average is \(\bar Y_g=\mu+A_g+m^{-1}\sum_j\varepsilon_{gj}\). Its random terms are independent, so

The cluster averages are independent across \(g\), hence

The row-independent calculation divides the marginal variance of one observation, \(\tau^2+\sigma^2\), by \(Gm\). It incorrectly treats the shared \(A_g\) as if it were newly and independently drawn for every row. When \(\tau^2>0\) and \(m>1\), the correct variance exceeds that calculation. The source of the discrepancy is precisely the dependence that cluster-level resampling retains. \(\square\)

Time-Ordered Data: Dependence Can Cross Rows

For a time series, consecutive observations can be dependent even when each observation is itself a multivariate vector. Resampling individual time points as though they were independent removes serial association. A time-block unit retains local sequences, but block resampling requires assumptions and a block construction suited to the process. The detailed properties of particular block methods are a separate topic.

For example, if \(X_t\) is a stationary scalar process with mean \(\mu\), autocovariance \(\gamma(h)=\operatorname{Cov}(X_t,X_{t+h})\), and a finite variance for its sample mean, then

This follows by expanding the variance of the sum into all pairs of time points: there are \(n\) terms with lag zero and \(2(n-h)\) ordered contributions for each positive lag \(h\). An independent-row calculation keeps only \(n\gamma(0)\), which need not equal the displayed expression. For multivariate time series, the same issue applies to the lagged covariance matrices between vector observations.

Exchangeability, Target, and Practical Choice

A useful starting point is to write down the inferential target and the sampling structure separately. The target might be a population mean vector, a covariance feature, a within-subject contrast, or an average over clusters. The sampling structure identifies which units were independently selected or randomized, and which observations share dependence. The resampling scheme should respect both.

For ordinary case resampling, the standard justification is that the cases are independent and identically distributed draws from a population, or that an appropriate asymptotic argument makes their empirical distribution representative for the target. Exchangeability means that the joint distribution is unchanged by permutations of the units. It is a useful symmetry condition, but exchangeability alone does not guarantee that an ordinary empirical bootstrap consistently approximates every statistic or target. For clusters, the analogous premise is typically independent sampling across clusters, while arbitrary dependence is allowed within each retained cluster.

With time dependence, observations are generally not exchangeable under arbitrary permutations: reordering them changes their temporal relationships. A method that preserves local order must be justified under assumptions appropriate to the time process. Likewise, a design with treatment restrictions or matched observations may require a restricted resampling scheme rather than unrestricted rearrangement.

Common Mistakes and Why the Choice Matters

• Resampling columns independently. This replaces the observed joint distribution by separate marginal distributions and removes empirical cross-covariances.
• Treating every row as independent. Repeated measurements and clustered observations can share random variation. The number of rows then overstates the amount of independent information.
• Using clusters without defining the target weights. Equal cluster weighting and equal individual weighting are different targets when cluster sizes differ.
• Shuffling time points freely. This destroys serial order and lag dependence; time-ordered data need a scheme and assumptions that address that structure.
• Assuming the right unit guarantees validity. Unit choice is necessary, but validity also depends on the sampling design, the target, the statistic, and the relevant regularity conditions.