Why Redundant Passive Predictors Matter
A passive variable is computed deterministically from other variables. For example, a total score may be the sum of several item scores. When such a variable is included as a predictor in a MICE conditional model, it can provide a useful summary. But including the total together with every item used to calculate it creates an exact linear dependency among the predictors. The model cannot uniquely estimate all of their coefficients.
This is a property of the design matrix for a particular conditional model, not a general objection to computing or storing passive variables. A passive total may still be required for the substantive analysis or useful in other models. The key question is whether the predictors selected for a given fit contain redundant columns.
How a Total Creates a Linear Dependency
Suppose \(T\) is passively defined as the sum of \(p\) component variables \(X_1,\ldots,X_p\), and that identity holds for every case in the model fit. If the design includes the components and the total, its columns satisfy
In design-matrix terms, the vector with coefficient \(-1\) on each component, \(+1\) on \(T\), and \(0\) on the other columns is nonzero but is mapped to the zero vector by \(X\). Thus the design is rank-deficient. An intercept does not remove this dependency: the relation has zero intercept.
The usual full-rank condition for unique ordinary least-squares coefficient estimates is that the columns of \(X\) be linearly independent. Under this condition, \(X^{\mathsf T}X\) is invertible. When \(X\) is rank-deficient, \(X^{\mathsf T}X\) is singular, so the usual formula \((X^{\mathsf T}X)^{-1}X^{\mathsf T}y\) cannot produce a unique coefficient vector.
Worked Example: A Total and All Its Items
In a hypothetical imputation model, a total \(T\) equals \(X_1+X_2\). Four completed cases have the following predictor values:
| Case | \(X_1\) | \(X_2\) | \(T\) |
|---|---|---|---|
| 1 | 1 | 0 | 1 |
| 2 | 0 | 1 | 1 |
| 3 | 1 | 1 | 2 |
| 4 | 2 | 1 | 3 |
With an intercept, the design matrix is
The last column is the sum of the second and third columns, so \(Xv=0\) for \(v=(0,-1,-1,1)^{\mathsf T}\), which is nonzero. The design has four columns but cannot have rank four. In fact, the first three columns are independent, so the rank is three.
If the conditional mean is written as \(\beta_0+\beta_1X_1+\beta_2X_2+\beta_TT\), then substituting \(T=X_1+X_2\) gives
Only the combined coefficients \(\beta_1+\beta_T\) and \(\beta_2+\beta_T\) are identified by the fitted mean, not the three individual slopes. For any number \(c\), replacing \((\beta_1,\beta_2,\beta_T)\) by \((\beta_1-c,\beta_2-c,\beta_T+c)\) leaves every fitted value unchanged.
Check the Identity Before Changing the Model
A singularity warning is a reason to investigate the design, not an instruction to delete a variable immediately. First establish whether the dependency is expected. For a passive total, verify its definition and recompute it from the intended component values in each completed data set. The identity must hold on the rows and values actually supplied to the conditional model.
In MICE, a passive variable may need to be recalculated after component values are drawn. If a stored total is stale, the intended identity may fail in the data used for fitting. That can hide the expected rank deficiency, or create a different design whose meaning is unclear. Conversely, if the total is correctly recomputed and all components and the total are predictors, exact dependence is expected.
Worked Example: A Stale Total Can Hide the Problem
Suppose a hypothetical total should satisfy \(T=X_1+X_2\). The first three rows below are consistent, but the fourth total was not updated after a component changed.
| Case | \(X_1\) | \(X_2\) | Stored \(T\) | \(T-X_1-X_2\) |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
| 2 | 1 | 0 | 1 | 0 |
| 3 | 0 | 1 | 1 | 0 |
| 4 | 1 | 1 | 3 | 1 |
For these stored values, the design with columns \((1,X_1,X_2,T)\) is
Its determinant is \(1\), so this particular square design has full rank. For example, subtracting the first row from the other rows leaves a \(3\) by \(3\) minor with determinant \(1\). A software fit may therefore show no singularity warning. But the nonzero residual \(3-1-1=1\) in case 4 shows that the stored total does not follow the stated passive definition. Full rank here is not evidence that the data are correct.
The appropriate check is to recalculate the total from the components and investigate why the stored value differed. If the total is then included with both components, the exact dependency reappears, as it should. Do not preserve an inconsistent total merely to make a model fit.
Choose a Nonredundant Predictor Set
Once the passive definitions and completed values are verified, decide which information should enter a particular conditional model. Common choices are to use the component items without the total, or to use the total without all of its components. The choice depends on the model’s purpose and the scientific role of the predictors; rank alone does not determine which representation is best.
A total and only some of its components do not automatically create the same exact linear dependency. They may still be highly associated, and the total may be inappropriate if it incorporates the variable being imputed. In particular, if \(X_1\) is the target and \(T=X_1+X_2\), using a currently completed \(T\) as a predictor can feed information about \(X_1\) back into its own imputation model. Rank deficiency and this form of target leakage are distinct concerns; avoiding an exact singularity does not establish that the conditional model is sensible.
Worked Example: Retaining a Summary for a Different Target
Consider the four consistent cases from the first example, but now suppose the imputation target is a separate variable \(Z\), not either component. The proposed predictors are an intercept, \(X_1\), and \(T\), while \(X_2\) is omitted. The first three rows give the square submatrix
Its determinant is \(-1\), so these three columns are linearly independent; adding the fourth row cannot reduce their rank. This set is not rank-deficient, even though \(T=X_1+X_2\), because the design does not include every component alongside \(T\).
This calculation only addresses linear dependence. Whether the total is useful for imputing \(Z\), whether omitting \(X_2\) is scientifically defensible, and whether this predictor set is compatible with the intended analysis require separate consideration. For a target that is itself part of the total, the same choice would require special caution because the total depends on the target.
Diagnosing and Responding to Singularity
Warning signs include a singular or non-invertible matrix message, aliased coefficients, a predictor automatically dropped by software, or estimates that change markedly when a redundant term is removed. The most direct diagnosis is to inspect the actual model matrix for the affected conditional fit and check its rank. For an exact total identity, verify the row-wise residual \(T-\sum_jX_j\). For other terms, look for duplicate columns or other exact linear combinations, including dependencies created by how categorical variables are coded.
Near-dependence is different from exact rank deficiency. Columns can be technically independent but so similar that small numerical changes lead to unstable coefficient estimates. Scaling may affect numerical conditioning, but it cannot remove an exact algebraic identity. A model can also have problems unrelated to rank deficiency, such as separation in a logistic regression. A full-rank design does not rule those problems out.
A practical response is to verify the passive computation, identify which predictors are redundant in the specific conditional model, and select a scientifically defensible nonredundant set. Other responses require care:
- Dropping the total retains the individual component predictors and removes this particular dependency.
- Dropping one or more components may retain a useful summary, but changes which item-level information the model uses.
- Automatically dropping a column can allow computation to continue, but the resulting predictor set may depend on software conventions and may not match the intended model.
- Penalization or coefficient priors can produce estimates under some singular designs, but they do not make the original likelihood identify the separate coefficients. The resulting estimates may depend on the penalty or prior.
The right choice depends on the conditional model’s role. A predictor set used to impute an item need not be identical to a set used for a different target. Review it in context rather than applying a blanket rule that passive variables must always be excluded.
Common Mistake and Key Takeaway
A common mistake is to reason, “The total is a useful predictor, so I should include it alongside every item; if the software drops a variable, the problem is solved.” The correction is that usefulness does not make redundant coefficients identifiable. First confirm that the total is correctly computed; then choose which summary or component information belongs in that conditional model. Automatic deletion may hide both the exact dependency and a data-processing error.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- If \(T=X_1+X_2+X_3\), what nonzero coefficient vector demonstrates rank deficiency when the design contains an intercept, all three components, and \(T\)?
- A total and all components are included, but the software reports full rank. Name one data check that should be made before interpreting this result.
- If a model includes an intercept, \(X_1\), and \(T=X_1+X_2\), but omits \(X_2\), is exact rank deficiency guaranteed by the total identity?
- Why is an automatically dropped predictor not necessarily a satisfactory scientific solution?
Solutions. (1) In the column order \((1,X_1,X_2,X_3,T)\), use \(v=(0,-1,-1,-1,1)^{\mathsf T}\). It is nonzero and \(Xv=0\) because \(T-X_1-X_2-X_3=0\) on every row. (2) Recompute the total row by row and check whether \(T-X_1-X_2-X_3=0\) on the values used in the fit; a stale or inconsistent total can break the identity. (3) No. The displayed identity involves the omitted \(X_2\), so it does not by itself establish a dependency among the included columns; the actual design must still be checked for other dependencies. (4) Software may drop a column according to an internal convention, potentially changing the intended predictor set or concealing an incorrectly computed passive variable. The retained variables should be chosen for the model’s purpose.