What Makes a Variable Derived?
Many analysis variables are constructed from other recorded quantities: a total from component items, a product term from two predictors, or a rate from a numerator and denominator. The central question is whether the construction is an exact definition or merely a useful relationship. That distinction matters in imputation: an exact definition leaves no independent value to draw once its inputs are fixed, whereas a separately modeled target can vary conditionally on those inputs.
Here, “deterministic” means that the same specified inputs, under the same definition, always produce the same output. It does not mean that the source measurements are error-free representations of some underlying reality. It means only that the derived value is fixed by the source values as recorded and by the stated calculation.
The definition requires more than a strong association. A regression prediction of one variable from another is not, by itself, a deterministic definition: actual values may differ from predictions. Nor does a variable become deterministic merely because it was calculated in a spreadsheet. The formula, source values, units, timing, and applicable rules must define its value without an additional random component.
Deterministic Values and Separately Modeled Targets
A separately modeled imputation target is treated as a quantity with its own conditional distribution given the variables in its model. That distribution may have variation even after the predictors are fixed. A deterministic derived variable has a different conditional structure: given its complete source vector, its conditional distribution places all probability on the one value \(g(X_i)\).
Consequently, for a deterministic definition, the conditional variance is zero:
This is not an assertion that the sources are known without uncertainty. If one or more source values are missing, then the output may not yet be computable from the available record. The defining relationship still applies to a complete source vector, but different possible completions of missing inputs can produce different derived values. Uncertainty about the inputs is not independent uncertainty in the function after the inputs have been fixed.
A separately modeled target instead permits values other than \(g(X_i)\), unless its model or support explicitly rules them out. This can be appropriate when the target is not an exact function, when it was measured independently with error, or when a different scientific quantity is intended. It is not an equivalent way to represent an exact calculated field: it changes the assumed data structure.
Worked Example: Body Mass Index From Recorded Measurements
Suppose a hypothetical record gives a weight of 72 kg and a height of 1.80 m. Define the derived variable \(D_i\) as weight divided by squared height, with units kg/m². Check the definition and compute the value.
The stated source values and function are \(W_i=72\) kg, \(H_i=1.80\) m, and \(g(W_i,H_i)=W_i/H_i^2\). The definition gives
For these fixed inputs, 22.22 kg/m² is the defined value, subject only to the chosen rounding rule. A separate model that could assign 21.7 or 23.0 kg/m² while keeping this same weight, height, and definition would not enforce the stated calculation. If height or weight is missing, however, the value cannot be calculated from the incomplete source record alone; different possible input values can yield different outputs.
This exact identity concerns the recorded measurements and the stipulated formula. It does not claim that weight and height perfectly measure every scientific construct someone might associate with body size.
How to Establish That a Relationship Is Deterministic
Before treating a variable as a function of other columns, establish the definition and its scope. In particular, verify that the same inputs and rule apply to every unit where the identity is claimed. A calculation may depend on a fixed conversion factor, a time-specific convention, or a rule that changes across groups. If that information is omitted, the function is not fully specified.
State which recorded quantities are inputs and which column is claimed to be derived from them.
Include units, constants, coding rules, and any relevant definition choices so the output is uniquely determined.
Confirm that the function is defined for the source values under consideration. A formula may fail for inadmissible inputs, such as a zero denominator.
For rows with complete inputs, recompute the function and examine discrepancies, allowing only the documented rounding or recording tolerance.
The justification is direct. If the function is fully specified and the input vector equals \(x\), the definition assigns \(D_i=g(x)\), not a range of possible values. Therefore the conditional distribution given \(X_i=x\) is a point mass at \(g(x)\); its conditional variance is zero. If an alleged value can differ while the same inputs and definition remain fixed, at least one premise is false: the function was not fully stated, the inputs do not capture all relevant information, or the target is not an exact deterministic function.
Worked Example: A Total From Component Items
Consider a hypothetical questionnaire with four items, each scored from 0 to 5. The defined total is the sum of the four item scores. For one complete record, the item values are 3, 4, 2, and 5. Determine the total and assess a separately stored total of 13.
The source vector is \(X_i=(3,4,2,5)\), and the function is \(g(X_i)=X_{i1}+X_{i2}+X_{i3}+X_{i4}\). Thus,
A separately stored value of 13 does not satisfy the exact definition for these inputs. It could reflect a data-entry error, a different scoring rule, a missing item handled by an undocumented convention, or a total that is not actually defined as this sum. It should not be treated as an equally valid draw from an unconstrained conditional model while retaining the claim that the total is exactly the sum.
The item range also provides a useful check: for four complete items each between 0 and 5, the total must be between 0 and 20. That range alone does not prove the exact sum identity—13 lies in the range—but checking the function does establish whether the particular value is correct. If an item is missing, the total is not determined by the observed items unless a further rule defines how to handle that missing component.
Examples of Relationships That Are Not Exact Definitions
A useful way to avoid overclaiming is to separate an identity from a statistical model. Suppose two measurements tend to increase together. A model for one conditional on the other can describe that tendency, but it does not make one measurement a deterministic function of the other. Likewise, a fitted regression value is a prediction or conditional mean under a model, not generally the observed value itself.
Worked Example: Product Term Versus Separate Outcome
In a hypothetical analysis, two recorded predictors have values \(A_i=2\) and \(B_i=3\). The analyst defines a product term \(D_i=A_iB_i\) for use in an analysis model. A separate model for a target \(Z_i\) given \(A_i\) and \(B_i\), however, allows residual variation. Distinguish the two quantities.
For the defined product term,
With \(A_i\) and \(B_i\) fixed at 2 and 3, the product term must equal 6. By contrast, a stochastic model for \(Z_i\) may assign positive probability to multiple values of \(Z_i\) at those same predictor values. That is a model for a separate target, not another way to define the product. Even if \(Z_i\) is strongly associated with the product, association does not establish the identity \(Z_i=A_iB_i\).
The example does not prescribe how a product term should be handled during an imputation algorithm. It establishes the prior distinction needed to reason correctly: a function of sources has one algebraically required value when the sources are fixed; a separately modeled target can have its own conditional variation.
Diagnostics, Failure Modes, and Alternatives
A deterministic relationship is useful because it supplies a sharp consistency check. For each record with complete inputs, compare the stored value with the value calculated from the stated function. Discrepancies can reveal coding differences, inconsistent units, stale calculated columns, transcription errors, or an incorrect claim that the target is derived. A repeated pattern of discrepancies may indicate a systematic rule difference rather than isolated mistakes.
Interpret discrepancies against the data’s precision. If a recorded value is rounded, exact equality at full machine precision is not expected; the relevant question is whether it agrees with the function to the stated rounding convention. Likewise, if the inputs themselves were rounded before the calculation, recomputing from those rounded inputs may not reproduce a value originally calculated from higher-precision inputs. Document which values and precision define the identity.
When the relationship is not exact, the main alternative is to define the target as its own variable and specify a model suited to it. That model should reflect the target’s meaning and allowed values. This alternative is not a repair for an unexplained mismatch in an exact derived column: first determine whether the definition, source values, or recorded target is wrong. A third situation occurs when an independently measured value is intended to estimate a quantity also calculable from sources. Measurement error or differing measurement procedures can make the two records disagree; then the scientific model must represent that distinction rather than silently asserting exact equality.
| Situation | What fixes the value? | Interpretation |
|---|---|---|
| Exact calculated field | Specified source values and function | No independent conditional variation after inputs are fixed |
| Separately modeled target | A conditional model, with its own possible variation | Not necessarily equal to a function of the predictors |
| Independent measurement of a related quantity | A measurement process that may differ from a calculation | Disagreement may be meaningful and requires an explicit account |
A common mistake is to reason, “The derived column is incomplete, so it should be imputed as another variable with its own conditional model.” That conclusion does not follow if the column is exactly defined by other variables. If the inputs are complete, the definition already fixes the value. If some inputs are missing, the output may be unresolved because its inputs are unresolved; allowing an independent value can break the identity rather than resolve that uncertainty. The correction is to establish whether the field is genuinely deterministic, identify its inputs and rule, and distinguish missing inputs from an independently missing target.
The distinction also has limits. Deterministic calculation cannot resolve ambiguity in the definition, recover missing source information by itself, or guarantee that the chosen function is scientifically appropriate. It only enforces the stated relationship. Whether that relationship belongs in a substantive analysis is a separate modeling decision.
Check Your Understanding
Answer each question by identifying the definition or modeling distinction involved.
- A hypothetical score is defined as the sum of three recorded items: 2, 1, and 4. What is the score, and could an exact sum definition also assign it a value of 8?
- A variable is predicted from age by a regression model, but observations vary around the fitted line. Is the variable a deterministic function of age? Explain.
- A rate is defined as a numerator divided by a denominator. The numerator is 12 and denominator is 4. What is the rate, and what issue arises if the denominator is zero?
- Two complete records have identical source values and the same stated function but different stored derived values. Give two possible explanations and describe the first check to make.
Solutions. (1) The defined score is \(2+1+4=7\), so an exact sum definition cannot also assign 8 to those same inputs. (2) No. Regression describes a conditional relationship with residual variation; it is not an exact function unless the observed value is guaranteed to equal the prediction. (3) The rate is \(12/4=3\), in the units implied by the numerator and denominator. If the denominator is zero, division is undefined, so the function needs a domain rule for that case. (4) Possible explanations include a recording error or an unstated difference in units, precision, or calculation rules. First recompute both values from the complete sources using the claimed function and compare under the documented precision convention.