Reporting the Analyzed Sample in a GLMM Study

Why the Analyzed Sample Needs Its Own Report

A GLMM may be fitted to a different set of data records from the set first assembled for a study. Some records may be outside the eligibility criteria; others may lack an outcome or a model predictor. A report that gives only the original enrollment count can therefore leave readers unsure how much information entered the fitted model. A report that gives only the final number can hide how the analysis sample was formed.

The central task is to describe the analysis sample: the records that contributed to the fitted model, the people or other observational units represented by those records, and the groups represented in each grouping structure. These counts answer different questions. In repeated-measures data, for example, 597 analyzed records might come from 304 patients. In a crossed participant-item design, one participant can contribute records involving many items, and each item can appear in records from many participants.

Definition: The analyzed sample is the set of observations or records actually used in fitting the reported model. The record count is the number of such observations or records. A group count is the number of distinct levels of a grouping factor represented among those records. These counts should be reported separately from the number initially eligible or enrolled.

For this purpose, “record” means a unit of data entering the model likelihood, as represented in the analysis data. It does not automatically mean one unique person or one independent piece of information. With repeated measurements, a person can contribute several records. With grouped data, a record can summarize outcomes for multiple trials. State the unit explicitly whenever “observation” could be misunderstood.

How to Reconcile the Counts

Start by identifying the data that were eligible for the intended analysis, then list the reasons records did not enter the fitted model. Reasons may include ineligibility, withdrawal before any usable measurement, an unavailable outcome, a missing predictor required by the model, or a data-quality rule. Give the number excluded for each reason and say whether reason counts overlap. When exclusions overlap, their counts cannot simply be added.

A concise way to verify the arithmetic is to label each eligible record with an inclusion indicator. Let \(I_i=1\) if record \(i\) entered the fitted model and \(I_i=0\) otherwise. If \(M\) records were eligible for this model, then the number analyzed is the sum of the indicators. For two exclusion conditions, the number excluded for at least one condition is the size of their union, not necessarily the sum of their sizes.

N_{\text{analyzed}}=\sum_{i=1}^{M} I_i, \qquad |A\cup B|=|A|+|B|-|A\cap B|.

The second equality is a short accounting justification: records in both exclusion sets \(A\) and \(B\) appear in each separate count, so subtracting the overlap once leaves each excluded record counted exactly once. With more than two reasons, a record-level flow or an explicitly defined primary reason for exclusion can prevent confusion.

Group counts require a separate calculation. For a grouping variable \(g\), count the distinct levels among records with \(I_i=1\), rather than counting levels in the original data file. In a model with more than one grouping structure, report the count for each structure. For a nested design, make clear which level is being counted; for crossed effects, count the represented levels of each factor separately. A group with no analyzed records does not contribute to the fitted model and should not be included in the reported analyzed group count.

Worked Example: Repeated Clinical Records After Exclusions

Suppose a hypothetical study screened 268 people. Of these, 18 did not meet eligibility criteria and 10 eligible people did not enroll, leaving 240 enrolled participants. The intended repeated-measures GLMM requires a follow-up outcome and a baseline predictor. Among the 240 enrolled participants, 14 lack the follow-up outcome, 9 lack the baseline predictor, and 4 are missing both.

The two model-variable exclusion sets overlap by 4 people. Thus the number of distinct participants excluded from the model is \(14+9-4=19\), not 23. The analyzed participant count is \(240-19=221\). Equivalently, \(240-14-9+4=221\).

Suppose all 221 participants have two scheduled records, one at each of two follow-up times, and the model requires both records. Then the model uses \(221\times2=442\) records. Suppose the participants were enrolled at 12 clinics, but all seven participants from one clinic were among the 19 excluded. The fitted model therefore represents 11 clinics, not 12.

A suitable summary is: “Of 240 enrolled participants, 221 had the outcome and baseline predictor required for the model (19 excluded; 4 had both values missing). The GLMM included 442 repeated-measures records from 221 participants at 11 clinics.” This statement distinguishes the enrollment count, the analysis count, and the clinic count, and the arithmetic reconciles.

Worked Examples for Different Data Structures

Worked Example: A Crossed Participant-Item Design

Consider a hypothetical experiment with 84 enrolled participants and 30 planned items per participant. Four participants withdrew before providing any usable responses. For the remaining participants, 65 planned responses are unavailable; all other responses have the outcome and predictors required by the model.

The number of participants represented is \(84-4=80\). Those 80 participants could have contributed \(80\times30=2{,}400\) participant-item records. Removing the 65 unavailable responses gives \(2{,}400-65=2{,}335\) analyzed records. The data contain 80 represented participant levels and 30 represented item levels, assuming each item has at least one analyzed response.

A report might say: “The model used 2,335 participant-item records from 80 participants and 30 items. Four of 84 enrolled participants provided no usable responses; a further 65 of the 2,400 planned responses from the remaining participants were unavailable.” The participant and item counts are not added together or treated as record counts; each describes a different grouping factor in the crossed structure.

Worked Example: Patients, Visits, Wards, and Hospitals

In a hypothetical longitudinal study, 312 eligible patients were assigned to wards in six hospitals, with two planned visits per patient. Eight patients had no outcome at either visit and were excluded. Among the remaining patients, 11 of the planned visit records lack an outcome required by the model. Assume each of the other records has the required variables.

After excluding the eight patients with no usable outcome, \(312-8=304\) patients remain. They contribute \(304\times2=608\) planned visit records. Removing the 11 unavailable records leaves \(608-11=597\) analyzed records. If all 11 records belong to different patients, all 304 patients remain represented. If some patients lack both visits, the represented-patient count must instead be counted directly from the 597 records; it cannot be inferred from the record total alone.

Suppose there were 18 wards initially, one of which had only three patients, all among the eight excluded patients. Then 17 wards are represented in the analyzed records. If each of the six hospitals still has at least one represented ward, the fitted model represents six hospitals. A clear statement gives all relevant units: “The model used 597 visit records from [the directly counted number of] patients, nested within 17 represented wards and six hospitals.” The bracketed count must be verified from the analysis data rather than assumed.

Check the Analysis Data Before Writing

The final counts should be obtained from the same model-ready data used for the reported fit, not reconstructed from memory or from the original enrollment file. A model may omit records because a required variable is missing or because a coding rule removes a value. It may also use a filtered subset specified in the analysis. The analyst should verify that the reported count agrees with the data actually passed to the fitting procedure and, where possible, the model’s reported number of observations.

A useful check is to make one record-level indicator for inclusion in the fitted model and then calculate both the total and the distinct levels of each grouping variable among included records. For each stated exclusion, confirm that the record meets the stated condition. If a single participant contributes multiple records, distinguish exclusions of participants from exclusions of individual records. The count of excluded participants is not generally the same as the count of excluded records.

Define the denominator.
State whether the starting count is screened, eligible, enrolled, or otherwise available for the planned analysis.

Reconcile exclusions.
List model-relevant reasons and account for overlap, or give mutually exclusive primary reasons whose counts sum to the number excluded.

Count records and groups in the fitted data.
Verify analyzed records and distinct represented levels for every grouping structure from the model-ready data.

Write the statement in context.
Name the record unit, the number of people or other observational units, the grouping factors, and the principal exclusions.

A flow diagram can communicate several stages and reasons for exclusion efficiently, especially in a large study. A sentence or a small table may be clearer when the sample flow is simple. These are presentation alternatives, not different definitions of the analysis sample. Whichever format is used, the final number and the unit counted must be explicit.

Common Errors, Diagnostics, and Limits

Reporting only the enrollment count. Enrollment describes who entered the study, not necessarily who contributed data to the fitted model. Give the analyzed count as well.
Calling repeated records participants. If 442 records come from 221 people, do not report “442 participants.” State both counts with their units.
Counting groups before exclusions. A site, ward, participant, or item with no analyzed records is not represented in that model fit. Recalculate group counts after applying the model’s inclusion rules.
Adding overlapping exclusion counts. If one record has two missing variables, it belongs to both reason-specific counts but is only one record excluded. State the overlap or use exclusive categories.
Assuming a record count reveals a group count. The same number of records can arise from many different distributions across participants or sites. Count distinct levels directly.
Using “complete cases” without defining the term. The relevant completeness is relative to the variables and records required for the reported model. Identify what was required and how the included sample was determined.

These counts document what was analyzed; they do not, by themselves, establish that exclusions were harmless. In particular, a transparent count does not show that the observed records are representative or that missingness has no effect on the estimate. The analysis rationale for missing data and the implications for interpretation should be considered separately. Avoid implying that a complete-record analysis is automatically unbiased merely because its sample flow is fully reported.

A practical diagnostic is arithmetic reconciliation: the starting count minus the number uniquely excluded should equal the analyzed count. A second diagnostic is structural: every reported group count should match the distinct levels present in the model-ready records. If either check fails, common causes include counting rows in one stage and people in another, overlooking overlap, retaining an empty group in a preliminary data table, or reporting counts from a different model specification.

Key takeaway: A useful GLMM sample statement reports the number of analyzed records and identifies their unit, gives the number of represented participants or other observational units when relevant, reports represented groups for each grouping structure, and reconciles exclusions with the population initially considered.

Check Your Understanding

Answer each question, then compare with the solutions below.

A repeated-measures model uses 360 visit records from 120 patients at 8 clinics. What are the record count, participant count, and clinic count?
Of 150 eligible records, 12 lack an outcome, 10 lack a predictor, and 3 lack both. How many records are excluded for at least one reason, and how many are analyzed?
A crossed model uses 1,140 records from 38 participants and 20 items. Can the record count be described as 1,140 participants? Explain.
A study initially includes 9 wards, but one ward has no records after exclusions. How many wards should be reported as represented in the fitted model?
Why does reporting the final sample count not establish that missing records caused no bias?

Solutions. (1) There are 360 analyzed records, 120 participants, and 8 represented clinics; these are distinct counts with distinct units. (2) The union of the exclusion sets is \(12+10-3=19\), so \(150-19=131\) records are analyzed. (3) No. The model uses 1,140 participant-item records from 38 distinct participants and 20 distinct items; records are not unique participants. (4) Eight wards are represented, because the ninth contributes no records to the fitted model. (5) Sample counts describe which records entered the analysis, but they do not establish how missingness relates to outcomes or model variables; representativeness and possible bias require separate assessment.