Why Test Invariants After Imputation?
A completed dataset should not merely look plausible: it must also obey the structural rules that give its variables meaning. A visit count cannot be negative or fractional if it represents a count of visits. A body-mass index (BMI) stored as a derived field should agree with the weight and height from which it is calculated. Such requirements are invariants: conditions that must hold for every applicable record in every completed dataset.
This tutorial concerns checks after imputation. The earlier material on Validating Input Data with Automated Schema Checks addresses checks on input data; here the key question is whether the completed datasets continue to satisfy essential constraints after missing values have been filled. These checks do not assess whether the imputation model is statistically appropriate, whether the imputations are plausible in distribution, or whether results are robust to assumptions. They test structural validity.
An invariant can apply to individual values, relationships within a record, or the structure of a whole dataset. For example, a range condition can be evaluated one record at a time; a rule about category indicators applies to several columns in the same record; and a uniqueness requirement may concern an identifier across the whole dataset. State the condition precisely enough that two analysts would reach the same pass-or-fail result.
Define the Rule Before Writing the Check
A useful check starts with a substantive rule, not a convenient property of the observed numbers. Specify the variables involved, the eligible records, the allowed values, and how missingness or rounding is handled. The rule should reflect the data’s meaning and measurement process. It should not be added merely because a pattern happened to hold in the observed records.
For a row-level rule \(C_i\), the requirement is that it hold for each eligible record \(i\) in each completed dataset. A dataset-level rule \(G\) instead evaluates an entire completed dataset, such as whether all participant identifiers are unique. In either case, test the \(M\) datasets separately. Pooling values across completed datasets could conceal a violation in one of them.
The conjunction means “and”: every dataset must pass. This is a logical quality-control criterion, not a statistical test. There is no p-value or sampling uncertainty attached to whether a declared structural rule is satisfied by the stored values.
Worked Example: Valid Ranges and Integer Counts
Suppose a hypothetical completed dataset contains four records. Before examining the imputations, the analysis team specifies that age must be from 18 through 100 years, inclusive, and that the number of clinic visits in the past year must be a nonnegative integer. The checks are therefore \(18\leq\text{age}\leq100\) and \(\text{visits}\in\{0,1,2,\ldots\}\).
| Record | Age (years) | Visits (count) | Result |
|---|---|---|---|
| A | 34 | 0 | Passes both rules |
| B | 101 | 2 | Age violates the upper bound |
| C | 52 | 2.5 | Visits are not an integer |
| D | 27 | -1 | Visits violate the nonnegative requirement |
Record B fails because \(101>100\), even though its visit count is valid. Record C fails because \(2.5\) is not an integer, and record D fails because \(-1<0\). Thus three records have at least one violation, involving three different rule failures. A useful report would identify each record, the rule, and the observed value; it would not report only “range check failed.”
A range should be an actual structural requirement, not a guess at what values are common. If ages above 100 are possible in the study population, then 100 is not an appropriate hard upper bound just because such values are rare. Likewise, a count rule is justified by the variable’s definition as a count, not by the appearance of a particular imputed value.
Check Relationships and Derived Fields
Some invariants involve more than one column. They can be expressed as formulas or logical conditions, then evaluated for each applicable record. If a field is a deterministic function of other fields, a strong practice is to calculate it from its defining inputs in each completed dataset and compare it with the stored value. This check is meaningful only if the formula, units, and rounding convention are specified.
Worked Example: A Derived BMI Field
Consider a hypothetical completed dataset with weight measured in kilograms, height in metres, and a stored BMI field rounded to one decimal place. The defining formula is \(\text{BMI}=\text{weight}/\text{height}^2\). Because the stored value is rounded to one decimal place, the team specifies a tolerance of \(0.05\) BMI units when comparing it with the unrounded calculation.
For record A, the weight is 70 kg, the height is 1.75 m, and the stored BMI is 22.9. The calculation is \(70/(1.75^2)=70/3.0625=22.8571\). Its absolute discrepancy is \(|22.9-22.8571|=0.0429\), which is no greater than \(0.05\); the check passes.
For record B, the weight is 64 kg, the height is 1.60 m, and the stored BMI is 24.2. The calculated value is \(64/(1.60^2)=64/2.56=25.0000\). The discrepancy is \(|24.2-25.0|=0.8\), which exceeds \(0.05\); the check fails.
The tolerance is tied to the stated rounding rule, rather than selected after seeing the mismatch. If BMI is generated within the workflow, it is often safer to recompute it from the completed weight and height than to impute BMI as a separate, potentially inconsistent field. A check still has value: it can detect that recomputation was omitted, used different units, or used a wrong formula.
Worked Example: Mutually Exclusive Category Indicators
Suppose a hypothetical dataset represents smoking status with three binary indicators: current, former, and never. For every record with a known status, exactly one indicator should equal 1 and the other two should equal 0. The invariant is that the three indicators are binary and their sum is 1.
Record A has values \((1,0,0)\). Each value is either 0 or 1, and the sum is \(1+0+0=1\), so it passes. Record B has values \((0,1,0)\), with sum \(0+1+0=1\), and also passes. Record C has values \((1,1,0)\), with sum \(1+1+0=2\), so it fails: the record is simultaneously marked current and former smoker.
The rule must match the coding scheme. If the study allows an unknown or unclassified status, the team must specify how that status is represented and which records are subject to the “exactly one” condition. It would be incorrect to apply the rule blindly to a legitimate missing or structurally inapplicable status.
A Practical Audit and Clear Failure Reports
An invariant check is most useful when its result is actionable. A failure report should make it possible to locate the problem and understand what rule was broken. At minimum, record the completed-dataset number \(m\), a stable record identifier, the name of the invariant, and the observed value or values. For a derived-field check, include the expected value and the discrepancy; for a dataset-wide check, state the relevant scope and count of failures.
Record the variables, units, eligible records, and any tolerance or missingness rule before evaluating completed data.
Do not check only the first dataset or combine values across imputations. A violation in any completed dataset must remain visible.
Include the dataset number, record identifier, rule name, observed value, and expected condition. If there are no violations, record that all specified checks passed.
Determine whether the cause is a coding error, a unit mismatch, a flawed formula, an unsuitable imputation model, or an incorrectly specified invariant. Preserve the original result while resolving the cause.
A failed check does not establish which part of the workflow is wrong. For example, a negative count could signal an inappropriate model for a count variable, an error in post-processing, or an incorrect interpretation of the variable. Simply replacing the value with zero may hide the problem and distort the completed dataset. Use the violation as a diagnostic lead, not as permission to edit an imputation without a documented, justified procedure.
What Invariants Can and Cannot Establish
Structural checks are complementary to other forms of validation. Passing a range check shows only that values lie within the specified range; it does not show that the distribution is credible, that associations are preserved, or that the imputation model is adequate. Conversely, a statistically unusual value is not necessarily a structural violation. A valid extreme value should not be rejected solely because it is rare.
There are several recurring failure modes:
- Checking only one completed dataset. A condition that passes in \(D^{(1)}\) can fail in another \(D^{(m)}\). Evaluate all \(M\) separately.
- Rounding away a violation. A fractional count is not made valid by rounding it after the fact. Use an imputation and data-generation strategy consistent with the count’s meaning, and preserve the original completed values for diagnosis.
- Using a tolerance without a measurement reason. Tolerances should follow stated precision or numerical computation needs, not be widened to make a failing check pass.
- Enforcing an empirical pattern as a universal rule. A relationship that happens to hold in observed records is not automatically an invariant. Establish that it must hold from the data definitions or study design.
- Confusing validity with plausibility. A value can pass every structural rule and still warrant review as statistically implausible. That is a different diagnostic question.
Invariants also have limits. Real data may include explicit “unknown” categories, structural missingness, special codes, or precision differences. The check must reflect those definitions rather than treating every nonstandard value as an error. Rules can also be too strict: for example, a hard age maximum unsupported by the study definition could flag valid records. Document the rule and its rationale so a future reviewer can distinguish a genuine defect from a mistaken constraint.
Check Your Understanding
For each question, work out the answer before reading the solution below.
- There are \(M=5\) completed datasets. A nonnegative-integer count check passes in four datasets and finds one negative value in the fifth. Does the invariant pass overall? What should the report include?
- A stored field is defined as twice a value \(x\), rounded to the nearest whole number. If \(x=3.2\) and the stored field is 7, does a tolerance of 0.5 permit the check to pass?
- Three indicators are intended to encode exactly one category for each applicable record. A record has \((0,0,0)\). What does the check conclude, and what should be verified before labeling the record erroneous?
- A completed dataset passes all specified range and consistency checks. What can be concluded, and what cannot be concluded?
Solutions. (1) No. The invariant passes overall only when it holds in all five datasets. Report the fifth dataset’s number, the record identifier, the count value, and the nonnegative-integer rule it violates. (2) The expected value before rounding is \(2(3.2)=6.4\); the stored value 7 differs by \(|7-6.4|=0.6\), which exceeds 0.5, so the check fails. (3) The sum is \(0+0+0=0\), not 1, so it fails the stated rule. First verify that the record is applicable and that an unknown or unclassified status is not legitimately represented by all zeros. (4) The specified structural constraints hold in that dataset. This does not establish that the imputation model is appropriate, that distributions or associations are preserved, or that the imputations are substantively plausible.