Why Sample Counts Need an Audit Trail
A regression result describes a model fitted to particular observations, not automatically every record in the source file. Records may be removed because they are duplicates, do not meet the study’s eligibility rules, or lack a value needed for a specified model. If the report gives only the starting sample size or only the final model size, readers cannot tell how the analysis sample was formed.
A useful account therefore connects each preparation stage to a rule and a count. It identifies the starting set, states how many records were removed at each stage and why, and gives the number of observations actually used to fit the model. These counts help readers check the arithmetic, understand the analysis population, and distinguish a stated eligibility decision from a data-availability limitation.
Define What Is Being Counted
Before listing exclusions, identify the unit represented by one record. It might be one patient, one visit, one school, or one other observational unit. Do not assume that a count of rows is also a count of distinct people: a person may contribute multiple visits, or a file may contain duplicate rows. State whether the starting count refers to rows, distinct units, or another clearly defined set.
The starting count should refer to a specified source or set of records, rather than an imprecise phrase such as “all data.” Then define the target analysis set in ordinary language: for example, eligible clinic visits with an observed outcome and the predictors required by the model. A preparation-stage count and a model’s sample size answer different questions. The former describes what remains after a stated set of rules; the latter describes how many observations contributed to a particular fit.
An exclusion count should also be attached to a reason and a denominator. “Seven records excluded” is not enough by itself. Readers need to know whether the seven were removed from the original records or from the records remaining after an earlier rule. For a sequential accounting, say explicitly that each stage acts on the records retained from the previous stage. This makes the arithmetic interpretable and avoids counting the same record twice.
A Transparent Count-by-Stage Record
A compact table or flow of counts can show the audit trail. At each step, report the rule, how many records it removes, and how many remain. If two exclusion reasons are assessed against the same original set and can apply to the same record, report their overlap or report the number of distinct records excluded. Do not simply add overlapping category totals.
Give its source, record unit, and count, such as “240 submitted visit records.”
For each rule, report the number removed from the current set and the number retained afterward.
State which outcome and predictors are required for the fitted model and how many records remain with those values available.
Confirm that the sequential counts reconcile and report the model’s actual observation count, not just an intended target.
A useful accounting identity for sequential steps is: the number retained after a step equals the number entering that step minus the number removed at that step. If every starting record is either retained or removed exactly once, these stages reconcile from beginning to end. When categories overlap, a different calculation is needed because category counts may include the same record.
Worked Examples
Worked Example: Building a Sequential Count From Source Records
Suppose a hypothetical clinic file contains 240 visit rows. The analysis plan retains eligible visits from adults and fits a regression model requiring an outcome and two predictors. During preparation, 9 rows are identified as duplicate entries and removed first. Of the remaining rows, 14 are from visits that do not meet the adult eligibility rule. Among the records still retained, 11 lack the outcome and 7 lack at least one required predictor. The last two groups are counted in that order, after the earlier exclusions.
Start with 240 rows. Removing 9 duplicate rows leaves \(240-9=231\). Removing 14 ineligible visits from those 231 leaves \(231-14=217\). Removing 11 records without the outcome leaves \(217-11=206\). Finally, removing 7 of those 206 records without a required predictor leaves \(206-7=199\).
The reported model sample size is \(n=199\) visit records. A concise account would say that the file began with 240 visit rows; 9 duplicate rows and 14 ineligible visits were removed; then 11 records without the outcome and 7 without a required predictor were excluded; the regression model was fitted to 199 visits. Because each later count is explicitly taken among records remaining, the four exclusion counts refer to distinct rows and sum to \(9+14+11+7=41\). This agrees with \(240-199=41\).
This example assumes that the file’s rows are visit records and that the stated sequential rules were applied in that order. The result should not be described as 199 patients unless each retained visit corresponds to a different patient.
Worked Example: Correcting for Overlapping Exclusion Reasons
Imagine a hypothetical set of 100 records assessed for two reasons at the same time: 10 lack an outcome value, and 8 lack an exposure value. Two records meet both conditions. How many records remain if both values are required by the regression model?
Let \(A\) be the set of records lacking the outcome and \(B\) the set lacking the exposure. Adding the category counts gives \(10+8=18\), but this counts each of the 2 records in both sets twice. The number of distinct excluded records is therefore \(10+8-2=16\), leaving \(100-16=84\) records for the model.
Thus the model sample size is \(n=84\), not 82. This calculation assumes the two reason counts refer to the same original set of 100 records and that the overlap count of 2 is known. A report can make this transparent by stating that 10 records lacked the outcome, 8 lacked the exposure, 2 lacked both, and 16 distinct records were excluded in total.
Why the Overlap Formula Works
The overlap adjustment follows by dividing the records in \(A\cup B\) into three non-overlapping groups: records in \(A\) but not \(B\), records in \(B\) but not \(A\), and records in both. When \(|A|\) and \(|B|\) are added, the first group is counted once, the second group once, and the overlap twice. Subtracting \(|A\cap B|\) removes one extra count for every record in the overlap. Therefore the total number of distinct records in either exclusion group is \(|A|+|B|-|A\cap B|\). Subtracting that union count from the starting total gives the number that meets neither exclusion condition.
If records are instead classified by a sequential procedure, later counts can be made among the records still present, as in the first example. In that setup, a record meeting more than one possible condition is removed at the first applicable step and is not counted again later. Both approaches can be reported clearly; the key is to state which counting approach was used.
Worked Example: Reporting Different Model Sample Sizes
Suppose a hypothetical eligible data set contains 160 records. A first regression model requires an outcome, an exposure, and age. Among the 160 records, 4 lack the outcome; among the 156 remaining, 2 lack the exposure; among the 154 remaining, none lack age. The model therefore uses \(160-4-2=154\) records. A second model adds a required laboratory predictor. Of the 154 records used by the first model, 7 lack that predictor, so the second model uses \(154-7=147\) records.
The correct report gives \(n=154\) for the first model and \(n=147\) for the second. It also explains the reason for the difference: the second model requires an additional predictor that is unavailable for 7 records otherwise used by the first model. These counts assume the stated losses are sequential and that the models are fitted to the records with all variables each model requires.
Reporting one sample size for both models would conceal that their estimates are based on different sets of records. Conversely, reporting only the smaller count would incorrectly imply that the first model used fewer observations than it actually did. Model-specific counts make the comparison auditable without implying that sample size alone determines which model is preferable.
What a Good Exclusion Account Includes
A clear report gives enough detail to reconstruct the arithmetic without turning the results into an unstructured list of every record. It should distinguish rules decided in advance, such as eligibility criteria, from preparation issues discovered in the records, such as duplicate rows. It should also distinguish exclusions made before a model is specified from exclusions that follow from the variables required by a particular model.
For each stage, include the following where relevant:
- The starting denominator: what set was being considered at that stage.
- The reason: the rule or missing required value that led to removal.
- The number removed and remaining: with a clear statement of whether counts are sequential or overlapping.
- The final model count: the number of observations actually used for each fitted model.
- The record unit: for example, visits rather than people, if that distinction matters.
Keep descriptions specific enough to be checked. “Data were cleaned” does not identify a rule. “Nine duplicate visit rows were removed, leaving 231 rows” is more informative. If an exclusion category is based on a subjective or ambiguous criterion, describe the criterion rather than presenting the resulting count as self-explanatory. The report should state what was done, not suggest that removing records automatically eliminates bias or guarantees a valid regression model.
Common Mistake: Adding Overlapping Counts
A frequent error is to add the counts for separate exclusion reasons without checking whether the groups overlap. For example, from 100 records, 10 may lack the outcome and 8 may lack the exposure, while 2 records lack both. Adding 10 and 8 and then reporting \(100-18=82\) assumes no record belongs to both groups. Since 2 records are counted twice, there are only \(10+8-2=16\) distinct exclusions, so \(100-16=84\) records remain.
The correction is to clarify whether counts are sequential or assessed on the same set. For sequential counts, report the denominator at each step and remove each record only once. For simultaneous reason categories, report the overlap or calculate the distinct union. If the overlap is unknown, do not claim a precise total distinct exclusion count by simply adding the categories; report the available category counts and explain that they can overlap.
Check Your Understanding
For each question, state the counting logic as well as the result.
- A file has 300 visit rows. Sequentially, 12 duplicate rows are removed, then 20 ineligible visits, then 15 records without the outcome. How many rows remain, and what assumption makes adding the exclusions valid?
- Of 90 records, 9 lack a required outcome and 7 lack a required predictor; 3 records meet both conditions. How many distinct records are excluded, and how many remain?
- Why is “the analysis included 120 patients” potentially inaccurate if each row in the fitted data represents a clinic visit and some patients have multiple visits?
- A second regression model uses an additional predictor, and its sample size is smaller. What information should the report provide to explain the difference?
- If two exclusion-reason counts are calculated on the same starting set but their overlap is unknown, why is their sum not necessarily the number of distinct excluded records?