Why an Exclusion Record Matters
A reported analysis sample size is the endpoint of a sequence, not a self-explanatory number. In clustered and repeated-measures data, “sample size” could refer to people, clusters, rows, or the individual trials represented by grouped-binomial rows. These counts answer different questions. A clear accounting record shows how the data moved from the source dataset to the records actually analyzed, and makes it possible to check that the reported analysis matches the stated eligibility rules.
This accounting is a data-management and reporting task; it does not itself determine whether an exclusion is statistically harmless. Eligibility criteria should be defined from the study question and applied consistently. Incomplete observations should be identified separately from people or records that were never eligible. A record may be incomplete for one analysis but usable for another, so the analysis-specific rules need to be explicit.
Define the Unit and the Accounting Sequence
Before counting, state what one row represents and what counts as a person and a cluster. In a long-format repeated-measures dataset, one row may represent one person at one visit; the same person therefore appears on several rows. In grouped-binomial data, one row may represent several trials summarized as a number of successes and a denominator. Counting rows alone does not recover either the number of people or the number of outcome trials.
A useful audit applies rules in a stated order. At step \(k\), let \(N_{k-1}\) be the number of units under consideration just before the step and let \(e_k\) be the number removed by that step. If each unit is assigned to the first rule that removes it, then the remaining count is
This subtraction is valid for the count of the unit being tracked, such as rows or people, provided the removed units are distinct at that step. If reasons are counted independently and can overlap, their counts cannot simply be added. For two overlapping sets of records \(A\) and \(B\), for example, the number removed for either reason is
A sequential log avoids this ambiguity by recording each record’s first applicable removal reason, while retaining additional flags when useful. Counts of unique people and clusters must still be recalculated from the retained rows; they cannot generally be obtained by subtracting the number of removed rows.
Record the initial number of unique people, clusters, and outcome rows, and explain what one row represents.
Report how many people or observational units fail each prespecified rule, and show which clusters remain.
Distinguish units with no usable outcome records from individual rows with an unobserved outcome.
Count incomplete rows under the stated analysis rule, then recount the unique people and clusters represented by the remaining rows.
Check that each count follows from the preceding stage and describe the final data structure and denominator.
Worked Examples of a Transparent Audit
Worked Example: Repeated Outcomes and Complete Records
Hypothetical data. A study enrolls 96 people from 12 clinics, with three scheduled visits per person. Four people fail a prespecified eligibility rule, leaving 92 eligible people and \(92 \times 3 = 276\) scheduled outcome rows. Three eligible people have no observed outcome at any visit. The other 89 people have at least one observed outcome. Among their \(89 \times 3 = 267\) potential visit rows, eight outcomes are missing. Of the 259 rows with an observed outcome, seven lack a predictor required for this particular analysis.
The row count after the outcome requirement is \(267 - 8 = 259\). Applying the predictor requirement leaves \(259 - 7 = 252\) complete analysis rows. Suppose an identifier check shows that exactly two of the 89 people with an observed outcome have no row containing both an observed outcome and the required predictor. Then \(89 - 2 = 87\) unique people are represented in the final rows. If all 12 clinics have at least one such row, the final data include 12 clinics, 87 people, and 252 visit rows.
The participant count does not follow from the seven incomplete rows alone. It follows from counting unique person identifiers among the 252 final rows. In particular, knowing only that at least one participant has no complete row would not justify subtracting exactly one from 89: several people could have no complete row. The number of unique people in the final records must be counted directly.
The accounting separates three different outcomes of filtering: four people were ineligible; three eligible people had no observed outcome; and seven outcome-observed rows lacked a required predictor. Those are not interchangeable descriptions, even though each can reduce the amount of information available to the analysis.
Worked Example: A Cluster Disappears From the Outcome Dataset
Hypothetical data. Six wards each contribute 10 people, for 60 people total. Four people from Ward 6 are ineligible, leaving 56 eligible people. The remaining six eligible people in Ward 6 have no observed outcomes. Each of the other 50 eligible people has two scheduled visits, so there are \(50 \times 2 = 100\) potential outcome rows among people with at least one observed outcome. Five of those rows have missing outcomes, leaving \(100 - 5 = 95\) outcome-observed rows.
Three of the 95 rows lack a predictor required for the analysis. Thus the final row count is \(95 - 3 = 92\). Suppose none of these three rows is the only otherwise complete row for its person, and every one of the 50 people has at least one complete row. Then the final dataset contains 50 people. Ward 6 contributes no outcome-observed rows, so the final cluster count is five, not six.
This example also illustrates why the count of scheduled rows needs a clear base. The 56 eligible people would have \(56 \times 2 = 112\) scheduled visits. The 12 visits for the six Ward 6 people with no outcomes are not among the 100 potential rows described above; they are accounted for as people with no observed outcome. Alternatively, a report may show all 112 scheduled visits first, then separately report the 12 rows with no outcome and the five missing outcomes among the other 100. Either presentation is defensible if the stages and denominators are explicit and the counts reconcile.
Worked Example: Grouped-Binomial Rows and Trial Denominators
Hypothetical data. Six clinics contribute four clinic-period rows each. Each row summarizes 10 binary outcome trials, so the source data contain \(6 \times 4 = 24\) rows and \(24 \times 10 = 240\) trials. Suppose the 24 rows contain 48 successes in total. Two rows fail a prespecified eligibility criterion; their success counts are 2 and 3. Removing them leaves \(24 - 2 = 22\) rows, \(240 - 20 = 220\) trials, and \(48 - (2 + 3) = 43\) successes.
Three of the remaining rows lack a required predictor. Their success counts are 1, 2, and 3. Removing them leaves \(22 - 3 = 19\) rows and \(220 - 30 = 190\) trials. The final success total is \(43 - (1 + 2 + 3) = 37\). If all six clinics still have at least one retained row, the final dataset has 19 grouped rows, six clinics, and a binomial denominator of 190 trials.
The number 19 is the number of data rows, not the number of binary trials. For grouped-binomial data, reporting only “19 observations” could conceal that those rows represent 190 trials. Conversely, 190 is not a count of independent clusters. State the row count and the trial denominator separately.
Checks, Comparisons, and Limitations
A transparent flow table is useful for the main report, while a row-level exclusion log is useful for reproducibility and audit. A flow table emphasizes the sequence and totals; a log can preserve identifiers, rule flags, and reasons for individual records. These approaches complement one another. Reporting only the final sample size is shorter but makes it difficult to detect accidental filtering, overlapping reasons, or loss of a cluster.
Several checks catch common data-accounting failures:
- Check uniqueness at the right level. Count distinct person and cluster identifiers, not rows, whenever reporting people or clusters.
- Check the row definition. Confirm whether a row is a visit, a person, or a grouped outcome summary before interpreting any count.
- Check the sequence. For each sequential stage, verify that the prior count minus the newly removed count equals the next count.
- Check cluster representation. List clusters with zero retained rows. A cluster can disappear even when many people remain overall.
- Check outcome denominators. For grouped-binomial rows, verify that each success count is between zero and its trial denominator, and sum retained denominators separately from row counts.
- Check missingness by variable and stage. A missing outcome row and a row with a missing predictor are different data states; do not label both simply “excluded” without detail.
An exclusion ledger documents what was analyzed; it does not establish that the retained data represent the intended population or that missingness has no effect on estimates. Complete-case restrictions can alter the composition of the analysis sample. The accounting should therefore report the rule and its consequences without implying that a reduced dataset is automatically unbiased. Decisions about handling missing data or assessing sensitivity require separate statistical reasoning.
Common Mistake
Another error is to add exclusion-reason counts that overlap. If the same row has both a missing outcome and a missing predictor, it must not be counted twice in a total removed-row count. Use mutually exclusive first-removal categories for the sequential flow, or report overlapping reason flags with a clear warning that their totals are not additive.
Check Your Understanding
Answer each question, then compare your work with the solutions below.
- A repeated-measures dataset has 40 people with two scheduled visits each. After the outcome requirement, 73 rows remain. A predictor filter removes 5 rows, and the final rows contain 36 unique people from 8 clinics. How many final rows are there, and why can the number of people lost not be inferred by subtracting five from 40?
- Two exclusion reasons flag 9 and 7 rows, with 3 rows flagged for both. How many distinct rows are flagged by either reason?
- A grouped-binomial dataset contains 15 retained rows, each with denominator 12. State the row count and the total trial denominator. What additional count should be reported for a clustered analysis if there are 4 represented clinics?
- In one clinic, all people are excluded or have no usable outcome. What happens to the clinic count in the final outcome dataset, and what should be checked before reporting it?
Solutions. (1) The final row count is \(73 - 5 = 68\). Five removed rows might belong to fewer than five people, and some affected people might still have another retained row; the 36 unique people must be counted from the final records. (2) The union count is \(9 + 7 - 3 = 13\) distinct rows. (3) There are 15 grouped rows and \(15 \times 12 = 180\) trials; report four clinics as the cluster count. (4) The final cluster count decreases by one if that clinic has no retained outcome rows. Verify the clinic identifiers and confirm that the absence is due to the stated rules rather than an identifier mismatch or unintended filter.)