Why Encoding Matters for Redundancy Checks
A categorical predictor such as payment frequency or vehicle class is not a single numeric column. To use it in many predictive models, an analyst represents its levels with indicator columns. That representation can create exact relationships among columns, even when the underlying categorical variable contains no unexpected information. A redundancy check that ignores how the encoding was constructed can therefore mistake a predictable structural relationship for a new finding about the data.
The central question is not simply whether two encoded columns are correlated. It is whether the columns in the design matrix are linearly dependent, what caused any dependency, and whether the dependency would remain if the encoding or intercept convention changed. The distinction matters when screening predictors, assessing matrix rank, and preparing a model matrix.
How the Encoding Creates a Dependency
Suppose a factor has levels \(A\), \(B\), and \(C\). Write its full set of indicators as \(D_A\), \(D_B\), and \(D_C\). Every observation belongs to exactly one of the levels, so exactly one indicator is 1. For every row, therefore, the indicators sum to 1.
The right-hand side is precisely the intercept column. Consequently, a design matrix that includes an intercept and all three indicators has an exact linear dependency. This is a property of the encoding, not evidence that one category predicts another in a surprising way. In reference-level encoding with \(A\) as the reference, the model matrix instead contains the intercept and \(D_B,D_C\). The omitted reference indicator can be reconstructed as \(D_A=1-D_B-D_C\), but that reconstruction does not make the included columns dependent when all three levels are observed.
This statement assumes each observation belongs to exactly one level and that the relevant levels occur in the data. Missing category membership, multiple memberships, or unobserved levels change the indicator pattern and must be checked rather than assumed. In particular, a category absent from a sample produces an all-zero column if it is nevertheless retained in the encoding.
Worked Examples: Inspecting Encoded Columns
Worked Example: A Three-Level Factor With an Intercept
A hypothetical insurer groups six policies by billing arrangement. The observed levels, in order, are \(A,B,C,A,B,C\). Determine the encoded columns and their rank under (1) full one-hot encoding with an intercept and (2) reference-level encoding with \(A\) as the reference. State what a redundancy check should conclude.
Method. Write the indicator values row by row, then use the defining sum of the indicators to identify any exact dependency. The common mistake avoided here is treating every full dummy column and the intercept as independent predictors.
Under full one-hot encoding, the six rows for \((D_A,D_B,D_C)\) are \((1,0,0),(0,1,0),(0,0,1),(1,0,0),(0,1,0),(0,0,1)\). The intercept column is \((1,1,1,1,1,1)\). Each row satisfies \(D_A+D_B+D_C=1\), so the intercept is the sum of the three indicator columns.
The three full indicator columns are independent: the rows \((1,0,0),(0,1,0),(0,0,1)\) form a \(3\)-by-\(3\) identity matrix, whose rank is 3. Adding the intercept does not increase the rank, so the four-column matrix has rank 3 and one exact dependency.
With \(A\) as reference, the columns are the intercept, \(D_B\), and \(D_C\). The rows are \((1,0,0),(1,1,0),(1,0,1),(1,0,0),(1,1,0),(1,0,1)\). The first three distinct rows form a square matrix with determinant 1, so these columns have rank 3. There is no exact linear dependency among the three included columns.
Check. The full encoding has four columns but rank 3; the reference encoding has three columns and rank 3. The reduction in rank for the full encoding is exactly the intercept-versus-indicators relationship, not an extra discovery about the policies.
Worked Example: An Unobserved Category
A hypothetical claims team defines four service tiers: Basic, Standard, Premium, and Legacy. In a sample of seven records, the tiers are Basic three times, Standard twice, Premium twice, and Legacy zero times. The team creates full one-hot columns for all four tiers. Find the rank of the four indicator columns, then find the rank after adding an intercept. Explain the screening implication.
Method. Count the observed indicator patterns and check for both a zero column and the sum-to-one relationship. The common mistake avoided is interpreting an all-zero column as a meaningful, independent predictor.
The Basic, Standard, and Premium columns each have at least one 1. The Legacy column contains seven zeros, because no sampled record is in that tier. For every row, \(D_{\text{Basic}}+D_{\text{Standard}}+D_{\text{Premium}}=1\), and \(D_{\text{Legacy}}=0\).
The three nonzero columns are independent: the sample contains each of the patterns \((1,0,0),(0,1,0),(0,0,1)\) among those three columns. Thus their rank is 3. Adding the all-zero Legacy column leaves the rank at 3. Adding an intercept also leaves the rank at 3 because the intercept equals the sum of the three nonzero columns. The five-column matrix (four indicators and intercept) therefore has rank 3.
The Legacy column contributes no information in this sample; it is zero for every row. The other three indicators already have the structural relationship with the intercept. A rank check should report the deficiency, while inspection of the columns explains its two causes. The screening conclusion is about the observed sample: it does not show that Legacy can never occur in a future sample. If the known tier is retained for future scoring, its encoding and treatment should be planned consistently rather than treating the zero sample column as evidence of a new effect.
Check. The number of nonzero category columns is three, and each has a distinct observed pattern, so the rank remains 3 both before and after adding the intercept.
Worked Example: Negative Correlation Is Not Pairwise Redundancy
Eight hypothetical accounts fall into exactly one of three contact-preference categories. The counts are \(A=3\), \(B=3\), and \(C=2\). Calculate the correlation between the \(A\) and \(B\) indicators, then determine whether those two columns are identical or exact linear multiples. Finally, identify the structural relation involving all three indicators and an intercept.
Method. Treat each indicator as a binary variable and calculate its empirical mean, variance, and covariance using averages over the eight observations. The common mistake avoided is dropping an indicator merely because mutually exclusive categories have negative correlation.
The empirical means are \(\bar D_A=3/8\) and \(\bar D_B=3/8\). No account belongs to both categories, so the mean of \(D_AD_B\) is 0. The empirical covariance is therefore \(0-(3/8)(3/8)=-9/64\).
For either indicator, the empirical variance is \(p(1-p)=(3/8)(5/8)=15/64\). Thus the correlation is
The columns are not identical: an \(A\) row has values \((1,0)\), while a \(B\) row has values \((0,1)\). They are not scalar multiples either, since a nonzero multiple of the first column cannot change its nonzero entries from the \(A\) rows to the \(B\) rows. Their correlation of \(-0.6\) describes mutual exclusivity and the observed category frequencies; it is not an exact pairwise linear dependency.
Across all three indicators, \(D_A+D_B+D_C=1\). Hence the three indicator columns have rank 3, but adding an intercept creates a dependency. Pairwise correlation does not by itself reveal that full-column relationship.
Check. The covariance calculation gives \(-9/64\), each variance is \(15/64\), and the correlation is \(-0.6\). The two columns are distinct and not scalar multiples; the exact structural relationship involves all three indicators and the intercept.
A Reliable Redundancy-Check Workflow
A redundancy check should separate the encoding’s known structure from other relationships in the data. A useful first step is to document the factor’s intended levels, the levels actually represented in the sample, the reference level if one is used, and whether an intercept is included. Then inspect the actual encoded columns, not only the original factor label.
For each factor, identify whether the representation is full one-hot or reference-level, and state the reference level when applicable.
Count each level in the analysis sample. Flag all-zero columns and levels represented by no rows before interpreting a rank result.
Assess whether any encoded columns are exact linear combinations of the others. In R, a model matrix can be examined with `qr(X)$rank`; compare that rank with the number of columns.
Test whether the cause is the full-indicator sum with an intercept, an unobserved level, or another relationship. Do not label a structural encoding identity as a newly discovered predictor effect.
Rank is a property of the realized design matrix: it depends on which rows and columns are present. Correlation is a pairwise summary and does not test every possible linear relationship among several columns. A correlation close to 1 or \(-1\) may be a useful warning, but exact redundancy is a linear-algebra question. Conversely, a structural dependency involving several columns can exist without any single pair having correlation 1 or \(-1\).
For a factor with a reference level, the omitted level is represented by all included indicators being zero. That pattern has an interpretation only if the reference level is actually observed and the encoding convention is known. A zero vector might otherwise reflect missingness, a data-processing problem, or a category not included in the fitted sample. Check the data definition before assigning it a reference-level meaning.
Common Mistakes and Why the Distinction Matters
- Keeping every full indicator with an intercept and calling the resulting dependency new information. The exact sum-to-one identity explains the dependency; it is a consequence of the encoding.
- Calling negatively correlated category indicators redundant based on correlation alone. Indicators for mutually exclusive levels are often negatively associated, but that does not make two columns exact copies or scalar multiples.
- Assuming reference-level encoding is always full rank. That conclusion requires the relevant levels to be observed and the columns to have the expected patterns. A missing level, a zero column, or another exact relationship can change the rank.
- Interpreting an absent level as evidence of no possible future membership. A sample with no records in a category tells you that its column is zero in that sample, not that the category is impossible in the population.
- Using a correlation screen as a substitute for checking the model matrix. Pairwise correlations cannot establish or rule out every exact linear dependency among multiple columns.
For predictive work, the practical goal is to encode categories consistently and recognize which relationships are built into the representation. A full indicator set plus an intercept contains a known dependency; reference-level encoding usually removes that particular dependency. An absent level or another rank deficiency calls for separate investigation. These checks support reliable screening without treating the mechanics of encoding as additional predictive information. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each problem, distinguish the structural encoding relationship from what the observed sample can establish.
- A four-level factor has all levels represented. It is encoded with four one-hot columns and an intercept. How many columns are there, what is the rank, and how many independent linear dependencies are present? (Answer: 5 columns, rank 4, and 1 dependency.)
- A three-level factor is reference-coded with an intercept, but the reference level has no observations. The other two levels both occur. What is the pattern of the two dummy columns and what is the rank of the three-column matrix including the intercept? (Answer: the two dummy columns sum to the intercept, so the matrix has rank 2 and 1 dependency.)
- In a sample of 10 accounts, two mutually exclusive categories each occur 4 times. What are the empirical covariance and correlation of their indicators, using averages over the 10 records? Are the columns scalar multiples? (Answer: covariance \(=-0.16\), correlation \(=-2/3\approx-0.6667\), and the columns are not scalar multiples.)
- A factor has levels Red, Blue, and Green, with Red chosen as the reference. An observation has dummy values \(D_{\text{Blue}}=0\) and \(D_{\text{Green}}=0\). Which level does it represent, assuming every row has exactly one valid level? (Answer: Red, the reference level.)
- A one-hot encoding has indicators for North, Central, and South. In the sample, South is never observed, while North and Central are both observed. What is the rank of the three indicator columns alone, and what is the rank after adding an intercept? (Answer: rank 2 for the three indicator columns, and rank 2 after adding the intercept; the South column is zero and the intercept is the sum of the North and Central columns.)