Choosing Figures for Cohort Flow and Diagnostics

Why These Figures Matter

A comparative study can report participant counts, propensity-score diagnostics, and covariate-balance results in text or tables, yet still leave readers unsure how the analysis took shape. A well-designed figure makes the sequence and diagnostics easier to inspect: who entered each analysis, how treatment groups overlap in estimated propensity scores, and which measured covariates remain imbalanced after matching or weighting.

These figures are descriptive reporting tools. A flow diagram does not establish that exclusions were harmless, and favorable overlap or balance plots do not prove the absence of unmeasured confounding. Their value is more precise: they make analysis decisions, sample changes, and measured-data diagnostics visible. The plotted groups and counts must correspond to the analysis being reported, not to an earlier or different sample.

Designing a Reconciled Cohort-Flow Figure

Start with the population that was actually identified, then show the steps that produced the analysis sample. Give each box a concise description and a count, and label branches so that the reader can tell which group each count represents. If participants can be excluded for more than one reason, state whether the reasons are mutually exclusive or whether a participant can appear in multiple exclusion counts. Otherwise, readers may incorrectly add the exclusions and conclude that the counts do not reconcile.

The figure should distinguish people from analysis-specific quantities. In a weighted analysis, for example, the number of included participants is a count of people; it is not the sum of their weights. If an effective sample size is reported, label it as such and do not present it as a participant count. Likewise, if matching leaves some eligible participants unused, show that fact rather than making the matched sample appear to include everyone.

If separate analyses use different samples—for example, a matched analysis and a weighting analysis—show their final sample counts separately. Do not combine their participants into one final box unless the groups are genuinely part of the same analysis. A diagram should also make the timing of each exclusion understandable: an exclusion before treatment-group classification is different from loss after eligibility or exclusion from only one analysis.

Showing Propensity-Score Overlap

The propensity score is the estimated probability of receiving treatment conditional on the covariates included in the specified model. An overlap figure typically plots that estimated probability, which ranges from 0 to 1, separately for treated and comparator participants. Its purpose is to show the parts of the score distribution where the groups have support in common and the parts where one group is sparse or absent.

Use a common horizontal scale from 0 to 1 and a common vertical scale for the groups. Overlaid or adjacent histograms are both possible; distinguish groups with clearly labeled colors or patterns and include a legend. State whether bars show counts, proportions, or density. Counts emphasize sample size, but can obscure the shape of the smaller group. Proportions or densities make distribution shapes easier to compare, but should not be mistaken for numbers of participants. If density is used, each group’s histogram has total area 1.

Use identical bin boundaries for both groups. A truncated horizontal axis can exaggerate apparent overlap, so show the full score range or clearly disclose any restricted range. If the analysis trims or restricts the sample, mark the restriction on the figure and report how many participants in each group remain. The plotted distributions should be identified as pre-restriction or post-restriction; if both are useful, label them as distinct panels rather than blending them.

If a restriction is used, make its rule visible without implying that a universal cutoff exists. For instance, a vertical line can mark a prespecified score boundary, but the caption should state the rule, which group-specific participants fall outside it, and the sample retained. A threshold chosen after inspecting the figure should not be presented as if it were fixed in advance. The graphic should describe the restriction actually used in the reported analysis.

Making Covariate Balance Visible

A balance plot often places covariates on the vertical axis and standardized mean differences on the horizontal axis. Plot a point for each covariate before and after matching or weighting, connect corresponding points if that helps show change, and clearly distinguish the two analysis stages. Include every covariate used in the balance assessment, not only those with large initial differences. Label the measure, the groups being compared, and the sample represented at each stage.

For a continuous covariate, a common standardized mean difference is the difference in group means divided by a standard deviation used as a reference. For a binary covariate, a common version divides the difference in group proportions by the square root of the average of the two groups’ Bernoulli variances. The standardization convention should remain consistent across pre- and post-adjustment points; state the reference denominator in the caption or methods. This matters because changing the denominator can make apparent improvement or deterioration difficult to interpret.

A vertical reference line at an absolute standardized mean difference of 0.1 is often used as a descriptive heuristic. It is not a significance threshold, a test of confounding, or proof that bias is absent. A signed horizontal scale shows direction as well as magnitude; an absolute-value scale shows magnitude alone. Label the scale accordingly. If standardized differences are displayed as absolute values, do not describe the plot as showing which group has the larger mean or prevalence.

Checking Figures Against the Analysis

Before finalizing a figure, compare it with the reported methods, sample description, and baseline and balance tables. The same treatment labels, analysis population, matching or weighting status, and covariate definitions should appear throughout. If a table reports balance after weighting but a plot shows balance after matching, label those as separate analyses. If a flow figure gives the number matched, that number should agree with the group counts used for the matched balance plot and effect estimate.

A useful check is to ask whether the main visual conclusion changes when the scale or display is made explicit. For overlap, inspect whether bins and vertical scales are shared. For balance, confirm whether the axis is signed or absolute and whether the threshold is labeled as a heuristic. For flow, recompute each branch total. These checks catch errors that may be hard to notice when a figure is reviewed only for appearance.

Common Mistakes and Alternatives

• Unreconciled flow counts. A diagram gives final counts that cannot be obtained from the preceding boxes. Recalculate each branch and define overlapping exclusion categories.
• Misleading overlap scales. Different histogram bins or vertical scales make group distributions hard to compare. Use common boundaries and clearly label counts, proportions, or density.
• Unlabeled restrictions. A plot omits score regions without stating that the analysis excluded them. Mark the restriction and report retained and excluded group counts.
• Selective balance display. The plot shows only covariates that improved. Include the full set assessed and state any relevant denominator convention.
• Overstating a threshold. A reader treats the 0.1 reference line as a formal pass/fail test. Describe it as a heuristic and report the actual standardized differences.
• Equating balance with no confounding. Small plotted differences do not establish exchangeability or rule out unmeasured confounding. Interpret balance as a measured-covariate diagnostic.

Tables remain useful when exact participant counts or covariate summaries are needed, while figures make patterns and changes easier to scan. Neither replaces the other in every report: a compact flow diagram can clarify sample construction, and a balance plot can reveal which covariates changed most, while a table provides precise values. A figure that is too crowded may be better split into labeled panels or supported by a table; simplification should not omit important groups or covariates.