Why the Data Structure Matters
Suppose a researcher wants to compare earnings in two years. The dataset could record the same people’s earnings in both years, or it could record earnings for different samples of people in each year. Both datasets contain an outcome and a time label, and both can be used to compare average earnings. But they do not describe the same pattern of observations: one follows units over time, while the other takes a new cross-sectional snapshot at each occasion.
This distinction affects what a comparison means. With linked observations on the same units, a change can be calculated for each unit. With different samples, the difference between averages compares the observed groups at the two occasions; it is not an average of changes for the same people. The first task is therefore to determine which units each record describes and whether those units can be linked across occasions.
A unit might be a person, firm, school, or region. A record is an observation for a unit at a particular occasion. In a common long-format layout, each row represents one unit-occasion pair, with a unit identifier, a time index, and one or more measured variables. For example, an earnings record might have the form \((i,t,y_{it})\), where \(i\) identifies a worker, \(t\) identifies an occasion, and \(y_{it}\) is that worker’s recorded earnings at that occasion.
Use Identifiers and Time Indexes Together
A time variable alone does not reveal whether data are longitudinal. A file may have many years and many rows but still consist of separate samples at each year. The diagnostic question is whether the same identifier appears at multiple occasions and is intended to refer to the same unit. Conversely, if each occasion’s records refer to different sampled units and there is no cross-occasion linkage, the data are repeated cross-sections.
Identifiers need not disclose a person’s name. A confidential survey can assign each participant a stable code, such as P104, and use that code at every interview. What matters is consistency of linkage, not whether the identifier is personally recognizable. A code newly assigned at each occasion, however, cannot establish that two records refer to the same person.
The study design is relevant as well as the table’s appearance. In repeated cross-sections, samples are drawn at separate occasions to describe the population at each occasion; a person may happen to be sampled more than once, but the design does not follow and link that person. If the researcher intentionally links and analyzes repeated records for the same units, that linked portion has a longitudinal structure. When a dataset combines designs, its structure should be described by component rather than forced into one label.
Worked Example: Identifying a Panel From Stable Worker IDs
A hypothetical firm records annual earnings, in thousands of dollars, for three workers. The records are:
| Worker ID | Year | Earnings (thousands of dollars) |
|---|---|---|
| W1 | 2024 | 50 |
| W1 | 2025 | 54 |
| W2 | 2024 | 40 |
| W2 | 2025 | 42 |
| W3 | 2024 | 60 |
| W3 | 2025 | 63 |
Classification. The same worker IDs appear in both years, and the firm’s identifier is defined to remain stable for each worker. The dataset is longitudinal: each worker is observed at multiple occasions.
Within-worker changes. W1’s change is \(54-50=4\), W2’s is \(42-40=2\), and W3’s is \(63-60=3\), each in thousands of dollars. The average change for these three observed workers is
Thus, in this hypothetical panel, the average observed earnings increase among these workers is 3 thousand dollars from 2024 to 2025. This is a descriptive statement about these three linked workers, not by itself a causal conclusion or a claim about all workers.
Repeated Cross-Sections: Different Units at Each Occasion
Repeated cross-sections provide observations for multiple occasions without following the same sampled units from one occasion to another. Each wave can describe the distribution or average of a variable among the units sampled at that time. Comparing wave-specific summaries can describe how the observed cross-sectional distributions differ, but it does not reveal each sampled unit’s individual change.
Worked Example: Classifying Two Independent Survey Waves
A hypothetical researcher surveys four residents in 2024 and four residents in 2025 about weekly hours worked. The survey assigns new IDs within each wave, and no respondents are linked across years.
| Survey ID | Year | Hours per week |
|---|---|---|
| A01 | 2024 | 30 |
| A02 | 2024 | 35 |
| A03 | 2024 | 40 |
| A04 | 2024 | 45 |
| B01 | 2025 | 32 |
| B02 | 2025 | 38 |
| B03 | 2025 | 42 |
| B04 | 2025 | 48 |
Classification. The IDs do not link respondents across years, and the survey design does not follow the same residents. These are repeated cross-sections, not a panel.
Compare the wave means. The 2024 mean is \((30+35+40+45)/4=150/4=37.5\) hours per week. The 2025 mean is \((32+38+42+48)/4=160/4=40\) hours per week. The difference in sample means is \(40-37.5=2.5\) hours per week. It describes a difference between the two observed samples. Since respondents are not linked, it is not the average change in hours for the same residents.
Balanced and Unbalanced Panels
Once the data have a longitudinal structure, ask whether the same units are observed at every occasion in the chosen set. A balanced panel is not merely one where each unit has the same number of records. The pattern of occasions matters: every selected unit must be observed at every selected occasion.
Worked Example: Equal Record Counts Do Not Guarantee Balance
Consider a hypothetical panel intended to cover occasions 1, 2, and 3 for units A and B. Its records are A at occasions 1 and 2, and B at occasions 2 and 3.
Check the required pairs. The full set of pairs is \((A,1),(A,2),(A,3),(B,1),(B,2),(B,3)\). The observed pairs are \((A,1),(A,2),(B,2),(B,3)\). The pairs \((A,3)\) and \((B,1)\) are missing.
Each unit has two records, but the panel is unbalanced over occasions 1, 2, and 3 because neither unit is observed at every occasion. Equal numbers of observations per unit are not sufficient to establish balance. If the chosen occasion set were only occasions 2 and 3, the same records would still be unbalanced for units A and B because \((A,3)\) is missing and \((B,2)\) is present; balance depends on the complete set of required unit-occasion pairs.
A duplicate pair also deserves attention. If two records both claim to be unit A at occasion 2 in a setup intended to have one record per pair, the data are not balanced as defined. The duplicate may be an error or may signal that the unit definition needs refinement—for example, if the rows actually represent two different transactions by A. Diagnose the record structure before labeling the panel.
Why Within-Unit Changes Are Different
A defining feature of linked observations is that a change can be computed for each unit observed at both occasions. For a balanced panel of \(N\) units at occasions 0 and 1, let \(y_{i0}\) and \(y_{i1}\) be the measured outcome for unit \(i\). The difference between the two panel means equals the average of the units’ changes:
The equality follows by combining the sums over the same \(N\) units and distributing the subtraction term by term. It depends on comparing the same units with the same weights at both occasions. In an unbalanced panel, a comparable within-unit change can still be calculated for units observed at both occasions, but the units included in that calculation must be stated; it need not represent all units in the original panel.
For repeated cross-sections, the two means generally average over different sampled units. Their difference can reflect changes in outcomes, differences in which kinds of units were sampled, or both. It cannot be rewritten as an average of individual changes unless the same units are linked and observed at both occasions. This is a distinction about what the records identify; it does not, on its own, establish why an outcome changed.
A Practical Diagnostic
A careful classification can be made in a few steps. First identify the observational unit and the occasions in scope. Then verify the meaning and stability of the unit identifier. Finally, list the expected unit-occasion pairs and compare them with the records actually present.
Specify what one record describes and what the time index labels. A person-year and a firm-quarter are different record structures.
If a stable identifier deliberately refers to the same unit at multiple occasions, the linked records have longitudinal structure. If samples are taken separately and units are not followed, the design is repeated cross-sectional.
For a panel, compare records with the chosen set of unit-occasion pairs. Every pair exactly once means balanced; missing or multiply recorded pairs mean the one-record-per-pair panel is unbalanced or requires clarification.
Common Mistakes and Why the Distinction Matters
- Assuming that a time column implies longitudinal data. Years or survey waves label occasions; they do not prove that the same units are followed. Check stable unit identifiers and the design.
- Calling a panel balanced because each unit has the same record count. Units can have equal counts but be observed in different occasions. Balance requires every chosen unit-occasion pair to be present exactly once.
- Interpreting a repeated-cross-section difference as individual change. A change in sample means compares two observed groups. Without links between units, it does not show how any particular person’s outcome changed.
- Treating an ID code as proof of identity. Codes assigned separately by wave may look like identifiers but do not link records. Confirm how the code was generated and whether it is stable.
These distinctions matter when defining a research question and interpreting its empirical summaries. A question about average outcomes in a population at two occasions can be addressed descriptively with repeated cross-sections. A question about changes for the same units requires linked observations for those units. Neither structure automatically solves questions of sample representativeness, measurement, or causal identification; those require their own assumptions and evidence.
Check Your Understanding
Answer each question using the unit identifiers and occasions, not just the presence of a time variable.
- A survey interviews the same 200 households in 2023 and 2024 using stable household IDs. What is the data structure?
- Two annual surveys sample different residents and assign new IDs at each wave. Why is the dataset not a panel, even though it has two years of observations?
- For units A and B and occasions 1, 2, and 3, list the pairs required for a balanced panel.
- Unit A is observed at occasions 1 and 2; unit B is observed at occasions 2 and 3. Each has two records. Is the panel balanced over occasions 1–3? Explain.
- What can the difference between two repeated-cross-section means describe, and what individual-level interpretation does it not support?