Indexing Clusters and Repeated Measurements

Why Cluster Indexing Matters

In clustered data, each observation belongs to a larger group: repeated measurements may belong to the same person, while patients may belong to the same clinic. A useful index must identify both which cluster an observation comes from and which observation within that cluster it is. That bookkeeping makes the observational unit, the cluster-level unit, the covariate values, and the response vector explicit before any model is written.

The two units are not interchangeable. In a study measuring blood pressure at several visits, a person is the cluster-level unit and a person-at-a-visit is the observational unit. In a multisite study of patients, the clinic may be the cluster-level unit while each patient is an observational unit. The study question and data collection determine which unit occupies each role.

Cluster and Observation Indices

Definition: Suppose a study contains $K$ clusters. Index clusters by $i=1,\ldots,K$. Let cluster $i$ contain $m_i$ observational units, indexed by $j=1,\ldots,m_i$. The response for observation $j$ in cluster $i$ is $Y_{ij}$. The total number of observational units is $N=\sum_{i=1}^{K}m_i$. If every cluster has the same size $m$, then $N=Km$; otherwise the data are unbalanced in cluster size.

The pair $(i,j)$ identifies one observational unit. The first index locates its cluster; the second identifies it within that cluster. It is often useful to record covariates with the same indexing. Write $\mathbf{x}_{ij}$ for covariates recorded separately for observational unit $(i,j)$, such as a patient's age or the visit time for a measurement. Write $\mathbf{z}_i$ for covariates that describe cluster $i$, such as a clinic's region or a hospital's size. These labels describe where a value belongs in the data; they do not, by themselves, specify how a model uses it.

A cluster-level value may be copied into each observation's row in a long-format data table. For example, a clinic's region may appear on every patient row from that clinic. Although the value is repeated in the table, the underlying characteristic is defined for the clinic, not separately measured for each patient. Conversely, an observation-level value can vary within a cluster: patients at the same clinic have different ages, and one person's visit times vary across visits.

Definition: The response vector for cluster $i$ is the ordered list of responses for its observational units: $$ \mathbf{Y}_i=(Y_{i1},Y_{i2},\ldots,Y_{im_i})^\mathsf{T}. $$ Its length is $m_i$. The superscript $\mathsf{T}$ indicates that the listed responses are arranged as a column. The vector contains the cluster's responses in a stated within-cluster order; it does not change which observational unit each response describes.

The ordering is usually chosen to match the study design. For repeated measurements, chronological order is natural. For patients within a clinic, a chosen patient order—such as study ID order—is sufficient. The order should be stated whenever it matters for reading a response vector or aligning its entries with covariates.

Worked Examples

Worked Example: Repeated Visits With Unequal Follow-Up

Three patients are measured at scheduled follow-up visits. Patient A has measurements at months 0, 3, and 6; patient B at months 0 and 3; patient C at months 0, 3, and 6. The recorded responses are systolic blood pressure in mmHg. Age is recorded for each patient, and visit month is recorded for each measurement.

Clusters and units. There are $K=3$ clusters, one per patient. The observational unit is a patient-at-a-visit, and the cluster-level unit is the patient. With the visits ordered by month, the cluster sizes are $m_1=3$, $m_2=2$, and $m_3=3$.

Responses and covariates. The response vectors are $\mathbf{Y}_1=(142,136,130)^\mathsf{T}$, $\mathbf{Y}_2=(151,145)^\mathsf{T}$, and $\mathbf{Y}_3=(128,126,122)^\mathsf{T}$. The visit-month covariates for patient A, for example, are $0,3,6$; age is a patient-level covariate and is constant across that patient's three rows. Patient B has only two observed visits, so the response vector has length two rather than three.

Observation count. The total number of measurement records is $N=3+2+3=8$. There are three clusters but eight observational units. The unequal cluster sizes are directly represented by the different lengths of the response vectors.

Worked Example: Patients Nested Within Clinics

A study enrolls patients from two clinics. Clinic North contributes three patients, and Clinic South contributes two. Each patient has one recorded response: the number of days in hospital after a procedure. For illustration, the responses are 2, 5, and 4 days at North, and 3 and 7 days at South. Each patient also has an age; the clinic has a recorded region.

Clusters and observational units. Index North by $i=1$ and South by $i=2$. The clinic is the cluster-level unit. A patient is the observational unit, so $m_1=3$ and $m_2=2$. Here $j$ indexes patients within a clinic, not visits for a patient.

Response vectors. If patients are ordered by study ID within each clinic, then $\mathbf{Y}_1=(2,5,4)^\mathsf{T}$ and $\mathbf{Y}_2=(3,7)^\mathsf{T}$. The total number of patient observations is $N=3+2=5$. The response vector for a clinic collects the responses of its patients; it is not a vector of repeated measurements on one patient.

Covariate locations. Age is recorded for each patient and can be denoted $x_{ij}$. Region describes the clinic and can be denoted $z_i$. If the clinic's region is included in every patient row, that is a repeated record of the same cluster-level value, not a different region measurement for each patient.

Worked Example: A Multisite Repeated-Measures Study

A trial recruits patients at two hospitals and measures each patient's symptom score at baseline and one month. Hospital 1 enrolls two patients and hospital 2 enrolls one. The scores are 8 and 6 for hospital 1 patient A, 5 and 4 for hospital 1 patient B, and 7 and 3 for hospital 2 patient C. Treatment group is assigned to each patient; hospital is the site.

There are two reasonable index choices, depending on the question and the intended data organization. To represent patients as clusters of repeated measurements, let $i$ index patients and $j$ index visits. Then $K=3$, $m_i=2$ for all three patients, and the response vectors are $(8,6)^\mathsf{T}$, $(5,4)^\mathsf{T}$, and $(7,3)^\mathsf{T}$. The observational unit is a patient-at-a-visit; the cluster-level unit is the patient. Hospital and treatment group are constant within each patient's two rows.

Alternatively, to organize patients as observations grouped by hospital, let $i$ index hospitals and $j$ index patients. This representation gives $m_1=2$ and $m_2=1$, but a single scalar response per patient is no longer enough to record both visits: each patient's outcome must be represented as a pair, or the records must be indexed with both patient and visit identifiers. The indexing choice must match what counts as an observation and what forms a cluster. It cannot be changed silently while keeping the same meaning for $Y_{ij}$.

Two Bookkeeping Results

The indexing scheme is useful because it gives a complete count of records and a transparent way to move between a table and cluster-specific vectors. These are bookkeeping facts, but mistakes in them can misidentify the observational unit or misalign responses and covariates.

Result: If the observational units are partitioned into $K$ nonempty clusters, with $m_i$ units in cluster $i$, then each observation has exactly one index pair $(i,j)$, and the number of observation records is $N=\sum_{i=1}^{K}m_i$.

Proof. By assumption, every observational unit belongs to one of the clusters, and no unit belongs to two different clusters. Within cluster $i$, enumerate its units once as $j=1,\ldots,m_i$. Thus every unit receives one pair $(i,j)$, and no two units receive the same pair. The clusters are disjoint, so counting all units means adding the $m_i$ units in each cluster. Therefore $N=\sum_{i=1}^{K}m_i$. $\square$

Result: Reordering observations within a cluster changes the displayed order of a response vector, but not the cluster membership or the response-covariate pairings, provided the response and all observation-level covariates are reordered together.

Proof. Fix cluster $i$ and let $\pi_i$ be a permutation of the indices $1,\ldots,m_i$. Define the reordered records by $Y'_{ij}=Y_{i,\pi_i(j)}$ and $\mathbf{x}'_{ij}=\mathbf{x}_{i,\pi_i(j)}$. Each new record therefore carries the response and observation-level covariates from the same original observational unit. Since $\pi_i$ only rearranges indices within cluster $i$, none of these records moves to another cluster. The cluster-level covariate $\mathbf{z}_i$ is unchanged. Hence the displayed ordering can change while both the membership and the within-record pairings remain intact. $\square$

This is why a response vector must be read together with its ordering convention. If responses are sorted by visit time but covariate rows are left in their original order, the displayed entries no longer refer to matching observational units. A vector alone cannot repair that mismatch.

Common Indexing Mistakes

Confusing a cluster with an observation. In a clinic-based patient study, a clinic may define the cluster while a patient is the observational unit. In a repeated-visit study, the patient may define the cluster and a patient-visit the observational unit.
Using $j$ for different things without stating the design. The within-cluster index may mean visit number or patient number, depending on the setup. State what it indexes.
Assuming all clusters have the same size. If follow-up or enrollment differs across clusters, use $m_i$ rather than a single common $m$, and compute $N$ as $\sum_i m_i$.
Treating a repeated cluster value as observation-specific. A hospital region copied down several patient rows remains a hospital-level covariate.
Reordering only part of a record. Reorder responses and observation-level covariates together; otherwise their indices describe different units.

Key takeaway: Before describing a clustered data set, state what $i$ indexes, what $j$ indexes, which unit is observed, which unit defines a cluster, and how responses and covariates are ordered. Then $\mathbf{Y}_i$ has a clear interpretation and its length $m_i$ can be checked against the study records.

Check Your Understanding

For each question, identify the indexing and units precisely.

A study measures each patient's lung function at four scheduled visits. What are the cluster-level and observational units, and what is $m_i$ if all visits are recorded?
Six clinics contribute 12, 18, 9, 15, 11, and 20 patients. If patients are observational units and clinics are clusters, write the total-count expression and calculate $N$.
In a hospital study, patient age and hospital ownership type are recorded. Which is naturally observation-level and which cluster-level if hospitals define the clusters?
A patient's symptom response vector is ordered by visit month. What must be done to the visit-time covariates if the response entries are reordered?
In a multisite study with one outcome per patient, explain why a site-specific response vector contains responses from multiple patients rather than repeated measurements from one patient.