Why Verify Deterministic Identities?
In a completed dataset, a derived variable may be defined exactly by other variables: a total is the sum of its components, an interaction is their product, and a quadratic term is a square. These are not approximate statistical relationships. They are deterministic identities: once the inputs are specified, the derived value is fixed.
That exactness makes identity checks useful. If a completed dataset contains \(z=x_1+x_2\), then every row with valid inputs should satisfy that equation. A single discrepancy can indicate that a derived term was not refreshed after an input was imputed, that rows or columns were misaligned, or that a formula was implemented differently from its intended definition. Conversely, passing the checks establishes only that the derived columns agree with their definitions; it does not establish that the imputations or analysis model are appropriate.
The aim here is a row-by-row verification procedure for every completed dataset, including continuous sums and products, categorical recodes, and transformations. The check is structural: compare each stored derived value with the value calculated from its inputs using the stated defining rule.
Definitions and Verification Rule
For exact arithmetic, the identity holds if and only if \(r_i=0\). Computer arithmetic often stores decimal values approximately, so a check for continuous variables usually permits a small, declared tolerance. One common rule is to flag row \(i\) when
where \(\text{atol}\) is an absolute tolerance and \(\text{rtol}\) is a relative tolerance. The absolute part matters when the expected value is near zero; the relative part scales the allowance for larger values. Tolerances should reflect the numerical precision and scale of the stored data, not be enlarged until unwanted discrepancies disappear. For integer-valued sums, products, and recodes stored exactly, exact equality is generally appropriate.
The check applies only where the defining inputs and the stored derived value are available. In a properly completed dataset, every variable required by the rule should ordinarily have a value for each analysis row. If a value is missing, do not silently treat it as zero or let software omit the row without recording that omission: first establish whether the row is outside the intended analysis or whether completion failed.
Worked Checks
Worked Example: Verifying a Passive Sum
Suppose a hypothetical completed dataset has two component scores, \(x_1\) and \(x_2\), and a total score \(s\) defined by \(s=x_1+x_2\). The values and recomputed totals are:
| Row | \(x_1\) | \(x_2\) | Stored \(s\) | Recomputed \(x_1+x_2\) | \(r_i\) |
|---|---|---|---|---|---|
| 1 | 12 | 8 | 20 | 12 + 8 = 20 | 0 |
| 2 | 9 | 11 | 20 | 9 + 11 = 20 | 0 |
| 3 | 14 | 7 | 22 | 14 + 7 = 21 | 1 |
For row 3, the discrepancy is \(r_3=22-(14+7)=22-21=1\). Thus the stored total fails its defining identity by one score point. Rows 1 and 2 pass. The correct response is to investigate row 3 and, if the specified definition is indeed a sum, recompute its total as 21. The check does not tell us whether either component score was imputed well; it only identifies a conflict between the stored total and its inputs.
Worked Example: Verifying a Product Term
Consider a hypothetical completed dataset with an interaction \(w=ab\), where \(a\) and \(b\) are continuous predictors. The product must be calculated using the values in the same completed row.
| Row | \(a\) | \(b\) | Stored \(w\) | Recomputed \(ab\) | Discrepancy |
|---|---|---|---|---|---|
| 1 | 2 | 3 | 6 | 2 × 3 = 6 | 0 |
| 2 | −1 | 4 | −4 | (−1) × 4 = −4 | 0 |
| 3 | 0.5 | 6 | 2.9 | 0.5 × 6 = 3.0 | −0.1 |
For row 3, \(r_3=2.9-(0.5)(6)=2.9-3.0=-0.1\). If \(w\) is stored to one decimal place, this discrepancy is not explained by rounding to one decimal place: 3.0 is the one-decimal representation of the computed product. The row fails the identity check. By contrast, if a product were stored with a small floating-point difference, such as \(2.999999999\) instead of \(3\), a suitably chosen numerical tolerance could treat that as computational rounding rather than a meaningful mismatch.
This check must use the current row’s \(a\) and \(b\), after completion. A product calculated once before imputing a missing value in \(a\) is not guaranteed to remain valid afterward.
Worked Example: Checking a Recode and a Nonlinear Transformation
Suppose a hypothetical dataset defines \(I=1\) when \(x\geq 10\) and \(I=0\) otherwise, and defines \(q=x^2\). For \(x=8\), the recomputed values are \(I=0\) and \(q=8^2=64\). For \(x=12\), they are \(I=1\) and \(q=12^2=144\). If the stored values are \(I=1,q=64\) at \(x=8\), and \(I=1,q=144\) at \(x=12\), then the quadratic term passes for both rows, but the indicator fails at \(x=8\): the stored indicator is 1 while its defining rule gives 0.
Now suppose a transformation is \(u=\log(x)\), using the natural logarithm. It is defined only for \(x>0\). If a row has \(x=2\) and stored \(u=0.6931\), then the recomputed value is \(\log(2)\approx0.693147\), so the difference is approximately \(0.000047\). That may pass a pre-specified tolerance of \(0.0001\). A row with \(x=0\), however, has no finite real value under this definition; it is not a case where the identity check should substitute an arbitrary value. It requires checking the variable’s domain and the intended transformation rule.
Why the Row-Level Check Is Sufficient
For a deterministic definition, the dataset-wide claim that the identity holds is equivalent to the collection of row-level claims. This gives a simple justification for checking every row rather than relying on a summary statistic.
The forward direction follows by subtracting \(g(x_{i1},\ldots,x_{ik})\) from both sides of the defining equality, giving \(r_i=0\) in every row. For the reverse direction, if \(r_i=0\), then \(z_i-g(x_{i1},\ldots,x_{ik})=0\), and adding \(g(x_{i1},\ldots,x_{ik})\) to both sides gives \(z_i=g(x_{i1},\ldots,x_{ik})\). Applying this argument to each row proves the dataset-wide identity. A mean discrepancy of zero is not enough: positive and negative row-level errors could cancel. Verification therefore requires checking the individual residuals, or an equivalent count of failing rows.
Applying Checks Across Completed Datasets
Multiple imputation produces multiple completed datasets. A deterministic identity is required within each one, not merely in an average dataset or in one selected imputation. For each completed dataset, recompute every derived term from that dataset’s inputs, compare stored and recomputed values, and record the number and location of failures. The following sequence makes the check explicit:
State the exact formula, coding rule, transformation, and relevant domain. For example, specify whether an indicator uses \(x\geq10\) or \(x>10\).
Use the corresponding row’s values in the current completed dataset; do not reuse a derived column calculated from earlier, incomplete inputs.
Calculate residuals or apply a declared tolerance, then identify the dataset and rows that do not satisfy the rule.
Check the formula, variable coding, row alignment, missing values, and transformation domain. Rebuild the derived term from its inputs when the definition is confirmed.
For categorical recodes, verify the exact mapping for each possible input category, including any explicitly defined reference or missing category. For transformations, verify both the formula and its domain. For centered or scaled terms, the rule also depends on which constants were used; the values used to define the transformation must be recorded and applied consistently. If centering constants vary across completed datasets, verify each dataset against the constants specified for that dataset’s definition rather than assuming one rule.
Interpretation, Alternatives, and Limitations
A failed identity check is a concrete data-structure discrepancy, but it does not identify its cause by itself. Possible causes include a stale passive term, an incorrect threshold, an unintended category code, row misalignment, rounding beyond the declared tolerance, or an input outside the transformation’s domain. Once located, the discrepancy should be resolved against the intended definition rather than hidden by broadening the tolerance or dropping the row without explanation.
Direct recomputation is the primary check because it tests the defining equation itself. A visual inspection of a few rows is faster but can miss an isolated failure. A correlation between a derived term and its inputs is not a substitute: a high correlation does not imply exact equality, and even a perfect-looking plot may conceal small or localized discrepancies. Summary checks such as a zero average residual can also miss errors that cancel. These approaches can help explore data, but the row-level identity check answers the structural question directly.
There are important limits. Passing all checks does not show that the imputation model is well specified, that imputations preserve the substantive analysis model, or that the chosen transformation is scientifically appropriate. It confirms only that each stored derived value matches the stated function of its inputs, within the declared numerical tolerance. If the definition itself is wrong, a flawless check will faithfully confirm the wrong definition. This is why the formula and coding rule must be set out before running the check.
Common Mistakes
- Checking only one completed dataset. An identity may hold in one dataset and fail in another. Repeat the check for every completed dataset.
- Checking an average rather than each row. Positive and negative residuals can cancel. Inspect row-level discrepancies or the count of failing rows.
- Using the wrong rule at a boundary. The indicator \(x\geq10\) differs from \(x>10\) when \(x=10\). Verify the exact inequality and test boundary values.
- Treating any nonzero difference as substantive, or any small difference as harmless. Use a stated tolerance appropriate to storage precision and scale, and investigate discrepancies that exceed it.
- Assuming a passing check validates the imputation. It verifies consistency of derived columns with their inputs, not the quality of those inputs or the statistical adequacy of the imputation model.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A derived total is defined as \(t=a+b\). In one row, \(a=6\), \(b=5\), and stored \(t=12\). Compute the residual and state the result of the check.
- A product is defined as \(v=xy\). For \(x=-2\), \(y=3\), and stored \(v=-6\), does the row pass? Show the calculation.
- An indicator is defined as \(J=1\) when \(x>4\), and \(J=0\) otherwise. What is the correct value at \(x=4\), and why is the boundary rule important?
- Why can a dataset have a mean residual of zero while some rows fail the identity check?
- What does it establish if every deterministic identity passes in every completed dataset, and what does it not establish?
Solutions. (1) \(r=12-(6+5)=1\), so the stored total fails the identity by 1. (2) Recompute \(xy=(-2)(3)=-6\); thus \(r=-6-(-6)=0\), and the row passes. (3) At \(x=4\), the condition \(x>4\) is false, so \(J=0\). Using \(x\geq4\) instead would give 1 at the boundary and define a different recode. (4) Residuals such as \(+1\) and \(-1\) can cancel in the mean even though neither row has residual zero; the row-level values must be checked. (5) Passing establishes that stored derived values agree with their stated definitions within the chosen tolerance. It does not establish that the imputations are appropriate, that the definition is scientifically sensible, or that the analysis model is valid.