Why Match-Count Audits Matter
A matching output can look plausible while its reported ratio, number of controls, or matched-sample size is wrong. A common cause is confusing the number of control assignments with the number of distinct control units. Another is reporting one overall ratio as though every matched set used that ratio. These are not cosmetic bookkeeping errors: they can change which observations are analyzed and how the design is described.
This tutorial focuses on auditing allocations after matching. Assume treatment is binary, and each matched set contains one or more treated units and one or more control units. A unit is an individual in the study; an assignment is that unit's placement in a matched set. Unless a design explicitly permits reuse, a unit should appear in at most one set. If reuse is allowed, the allocation must make that explicit. The audit also assumes the treatment labels and set membership are recorded correctly; a count check cannot establish that matching was appropriate on covariates.
The order in the phrase treated-to-control matters. For example, \(1:3\) means one treated unit for three control appearances, not three treated units for one control. In a set with two treated units and four controls, the ratio is \(2:4\), which reduces to \(1:2\). Always state the order in words as well as with numbers when reporting results.
Three Counts That Must Not Be Confused
An allocation can be audited at three levels. The number of appearances counts every placement in a set, even if the same person occurs more than once. The number of unique units counts each person once across the allocation. The number of sets counts the groups, not their members. These quantities agree only in limited designs. With no reuse and no unit appearing in multiple sets, total appearances equal total unique matched units. With reuse, control appearances can exceed unique controls.
For set \(j\), let \(t_j\) and \(c_j\) denote the numbers of treated and control appearances. Then the overall treated-to-control ratio, calculated from appearances, is \(\sum_j t_j:\sum_j c_j\). This overall ratio need not be the ratio in any particular set. In a variable-ratio design, reporting only the overall ratio conceals how allocation varies. Report set-level counts or a clear summary of their distribution as well.
For controls, let \(u_C\) be the number of distinct control units across all sets. Then control reuse has occurred precisely when the total number of control appearances exceeds \(u_C\). This comparison detects repeated appearances, but the allocation itself must be inspected to identify which control was reused and where. In a design without replacement, every control identifier should occur in at most one set.
Worked Example: Checking a Fixed 1:2 Allocation
Suppose a hypothetical study matches four treated patients to two controls each, without replacement. The allocation lists treated units \(T_1,T_2,T_3,T_4\) and control pairs \((C_1,C_2)\), \((C_3,C_4)\), \((C_5,C_6)\), and \((C_7,C_8)\). Check the ratio and counts.
Each set has one treated appearance and two control appearances, so each set is \(1:2\). There are \(J=4\) sets. Thus the treated appearances are \(\sum_j t_j=1+1+1+1=4\), and the control appearances are \(\sum_j c_j=2+2+2+2=8\). The overall ratio is \(4:8=1:2\).
The eight control identifiers are all distinct, so \(u_C=8\). Control appearances and unique controls agree: \(8=8\). There are also four unique treated units, each appearing in one set. The allocation is consistent with fixed 1:2 matching without replacement: four treated units, eight distinct controls, and twelve unique matched units in total.
If a report instead claimed that this allocation used “two controls per treated unit” but listed only six control appearances, the claim and allocation would conflict. Either a set was omitted, fewer than two controls were assigned to some treated units, or the count was wrong; the discrepancy must be resolved before analysis.
Fixed Ratios, Variable Ratios, and Full Matching
A fixed ratio is a set-level requirement, not merely an overall average. In fixed 1:2 matching, every included treated unit should have two control appearances under the specified design. If matching constraints leave a treated unit with only one eligible control, the allocation is not fixed 1:2 for that unit. It may be a variable-ratio result, or the unit may have been excluded, depending on the stated design. The analyst should not silently count a partial set as if it met the target ratio.
In variable-ratio matching, different sets can have different ratios, such as 1:1 for one treated unit and 1:3 for another. The overall ratio can still be calculated by summing appearances, but it does not describe each set. Full matching is also naturally audited set by set: a set may contain multiple treated and multiple control units. Its ratio is based on those counts, and a single global treated-to-control ratio does not turn the individual sets into fixed-ratio sets.
A reliable check proceeds in this order: verify set membership and treatment status; count treated and control appearances in every set; check each set against the design's ratio rules; count unique identifiers separately; then compute the overall appearance ratio. If reuse is permitted, mark repeated control identifiers and state the number of appearances and distinct controls. Do not infer a fixed ratio from the pooled total alone.
Worked Example: Variable Ratios With Reused Controls
Consider a hypothetical variable-ratio allocation with three treated units. Set 1 contains \(T_1\) and controls \(C_1,C_2\); Set 2 contains \(T_2\) and controls \(C_2,C_3\); Set 3 contains \(T_3\) and controls \(C_4\). Reuse is permitted. Find the set ratios, overall ratio, and distinct-control count.
The set-level counts are \((t_1,c_1)=(1,2)\), \((t_2,c_2)=(1,2)\), and \((t_3,c_3)=(1,1)\). The ratios are therefore \(1:2\), \(1:2\), and \(1:1\). Since these differ, this is variable-ratio matching, not fixed 1:2 matching.
The total treated appearances are \(1+1+1=3\), and total control appearances are \(2+2+1=5\). The overall ratio is \(3:5\), or 0.600 treated appearances per control appearance (equivalently, \(1:1.667\) when expressed as treated-to-control). The distinct controls are \(C_1,C_2,C_3,C_4\), so \(u_C=4\). There are five control appearances but only four distinct controls because \(C_2\) appears in Sets 1 and 2.
Thus the data support a report of three treated appearances, five control appearances, and four distinct controls, with reuse of \(C_2\). Reporting “five controls” without qualification would be ambiguous and could incorrectly imply five different people. Reporting a fixed 1:2 ratio would also be wrong because Set 3 has only one control appearance.
A Short Justification for the Reuse Check
Suppose the control identifiers appearing across sets are listed as \(C_1,\ldots,C_m\), with control \(C_i\) appearing \(r_i\) times. The total control appearances count each placement, so it equals \(\sum_{i=1}^{m}r_i\). The unique-control count is \(m\). Since every listed control appears at least once, \(r_i\geq1\), and therefore:
The difference is zero exactly when every \(r_i=1\), meaning no control is reused. A positive difference proves that at least one control appears in multiple sets. It does not show that the matching was erroneous: reuse may be an intended feature. The design specification determines whether it is allowed.
Worked Example: Finding a Misreported Full-Matching Ratio
A hypothetical full-matching allocation has three sets: Set A contains two treated and one control unit; Set B contains one treated and two control units; Set C contains one treated and three control units. All identifiers are distinct. A summary claims that the design is fixed 1:2. Audit the claim.
The set counts and ratios are: A, \(2:1\); B, \(1:2\); and C, \(1:3\). Thus the sets do not share a common ratio, so the fixed 1:2 claim is false. The overall treated appearances are \(2+1+1=4\), while control appearances are \(1+2+3=6\). The pooled ratio is \(4:6=2:3\), not 1:2.
Because all identifiers are distinct, the unique counts are four treated units and six control units. Total appearances also equal total unique units: \(4+6=10\). This confirms there is no reuse, but it does not rescue the fixed-ratio claim. The correct description is that the three full-matching sets have ratios \(2:1\), \(1:2\), and \(1:3\), with an overall appearance ratio of \(2:3\).
Diagnostics and Common Failure Modes
A ratio audit is a bookkeeping diagnostic, not a test of covariate balance or treatment-effect validity. It can reveal inconsistencies in the matching output, but it cannot show that the chosen distance, caliper, or matching design is scientifically adequate. Check the match allocation alongside the design specification and the earlier material on matching structures.
- Reversing the ratio. Writing “2:1” without saying which group comes first invites mistakes. State “two treated appearances per control appearance” or use the explicit order treated:control.
- Using the overall ratio to claim a fixed ratio. A pooled count can average over sets with different compositions. Check every set against the fixed-ratio requirement.
- Calling appearances unique people. A control reused in three sets contributes three appearances but only one distinct control. Report both counts when reuse occurs.
- Assuming more rows mean more independent units. A reused control can generate multiple matched-set records, but it remains the same person. Analyses and uncertainty calculations must account for the actual design; duplicating a record as though it were a new independent person can understate uncertainty.
- Silently repairing a partial set. If a target fixed ratio was not met, document whether the set was excluded, retained under a variable-ratio rule, or handled by another prespecified rule. Do not relabel it after seeing the outcome.
When reuse is found, investigate whether it was allowed by the matching design and whether the same identifier truly represents the same unit. When no reuse was intended, repeated identifiers indicate an implementation or data-management error to resolve. When a set has an unexpected count, compare the allocation with the specified ratio constraints and record the disposition of any unmatched units. These checks make the allocation interpretable, but they do not replace examination of balance, overlap, or the target causal contrast.
Comparison and Limits of the Audit
A fixed-ratio design offers a simple and uniform set structure, so its audit is direct: count the controls in every set and verify the target. Variable-ratio matching permits the number of controls to vary, so the audit must retain set-level counts rather than rely on one summary ratio. Full matching permits more general set compositions and likewise requires explicit counts of both treatment groups in every set. With replacement, unique-unit counts and appearance counts must be reported separately; without replacement, a repeated control identifier is a warning that the allocation may violate its stated rule.
No one of these count summaries establishes that one design is preferable. Ratio and reuse decisions depend on the study design and the available matches. This audit also does not determine how to resolve ties, nor does it prescribe an estimator or variance formula. Its purpose is narrower: ensure that the matched structure being described is the one actually encoded in the allocation.
Check Your Understanding
Answer each question by distinguishing the set-level allocation from the pooled counts. Brief solutions follow.
- Two treated units each have three distinct controls, with no control reused. What are the total treated appearances, control appearances, and overall treated-to-control ratio?
- Three sets each contain one treated unit. Their control counts are 1, 2, and 3. Is the design fixed-ratio? What is the overall ratio?
- A matching file has six control appearances but five distinct control identifiers. What can be concluded about reuse, and what cannot be concluded from these counts alone?
- Why is it incorrect to describe a reused control as a new independent person each time it appears in a matched-set table?
Solutions. (1) There are \(2\) treated appearances and \(3+3=6\) control appearances, so the overall ratio is \(2:6=1:3\); the allocation is fixed 1:3 if both sets meet that rule. (2) The set ratios are \(1:1\), \(1:2\), and \(1:3\), so the design is variable-ratio. The pooled ratio is \(3:(1+2+3)=3:6=1:2\). (3) At least one control is reused, because six appearances exceed five distinct controls. The counts alone do not identify which control was reused or whether reuse was permitted. (4) Repeated appearances refer to the same unit, not new people. Treating them as independent observations can misrepresent the sample size and produce invalid uncertainty calculations.