Why Preprocessing Consistency Matters
A cluster assignment or principal-component score is not determined by a record’s raw values alone. It also depends on how those values were imputed, centered, and scaled before the clustering or PCA was fitted. If an assessment record is processed differently from the training records, the resulting derived feature may no longer have the meaning assigned to it by the fitted model.
This matters whenever a predictive model uses clusters or component scores as predictors. A missing value must be handled using a defined rule, and the scaling parameters used to create a derived feature must be the same ones used when the feature was fitted. The earlier material on designing a training-only transformation pipeline establishes the general principle: estimate preprocessing quantities from training data, then reuse them without refitting on assessment data.
Imputation and Scaling Define the Geometry
For predictor \(j\), let \(m_j\) be an imputation value estimated from observed training values. A missing value is replaced by \(m_j\). After this imputation, let \(\bar{x}_j\) and \(s_j\) be the training mean and sample standard deviation of the completed predictor. When \(s_j>0\), the standardized value is
The order matters: first estimate the imputation value from observed training values and fill training-data gaps; then calculate the center and scale from the completed training predictor. For a new record, apply the stored imputation value and then the stored center and scale. Do not calculate new values from the assessment data.
For Euclidean clustering, standardization changes the distance between two completed records \(a\) and \(b\) to
The training mean cancels when taking a difference, but the scale does not. Thus a one-unit difference in a predictor with a small training standard deviation contributes more to squared distance than a one-unit difference in a predictor with a large standard deviation. Imputation also changes distances because it supplies a numerical value where a value was absent.
Worked Example: A Missing Value Changes a Cluster Distance
A hypothetical training set contains four records on predictors \(x\) and \(y\): A is \((0,0)\), B is \((2,0)\), C is \((0,2)\), and D is \((\text{missing},0)\). Use the median of observed training values to impute \(x\), then standardize both predictors using their completed-training means and sample standard deviations. Two fitted cluster centroids in standardized coordinates are \(C_1=(-0.5,-0.5)\) and \(C_2=(1.5,-0.5)\). Find D’s assignment and assign a new assessment record E with \(x\) missing and \(y=2\).
Method. Estimate the imputation and scaling quantities from the training records, transform those records, and use the same quantities for E. This avoids calculating a different geometry for assessment data.
The observed training values of \(x\) are \(0,2,0\). In sorted order they are \(0,0,2\), so the median imputation value is \(m_x=0\). After imputation, the \(x\) values are \(0,2,0,0\), with mean \(\bar{x}=0.5\). Their squared deviations sum to \(0.25+2.25+0.25+0.25=3\). The sample variance is \(3/(4-1)=1\), so \(s_x=1\).
The \(y\) values are \(0,0,2,0\), with mean \(\bar{y}=0.5\). Their squared deviations also sum to \(3\), so their sample variance is \(3/3=1\), and \(s_y=1\). D is imputed to \((0,0)\), and its standardized coordinates are \(((0-0.5)/1,(0-0.5)/1)=(-0.5,-0.5)\).
The squared distance from D to \(C_1\) is \((-0.5+0.5)^2+(-0.5+0.5)^2=0\), so the distance is \(0\). The squared distance to \(C_2\) is \((-0.5-1.5)^2+(-0.5+0.5)^2=4\), so the distance is \(2\). D is assigned to \(C_1\).
For E, use the stored \(x\) imputation value \(0\). Its completed raw values are \((0,2)\), giving standardized coordinates \((-0.5,1.5)\). Its squared distance to \(C_1\) is \(0^2+2^2=4\), so the distance is \(2\). Its squared distance to \(C_2\) is \((-2)^2+2^2=8\), so the distance is \(\sqrt{8}\), approximately \(2.828\). E is assigned to \(C_1\). The distances confirm the assignment: \(2<2.828\).
Common mistake avoided: Recomputing the median, mean, or standard deviation from E or from a batch of assessment records changes the transformation that defines the fitted centroids. The assignment would then not be based on the same coordinate system.
Using Training Preprocessing for PCA Scores
PCA also depends on the data representation used when it is fitted. Suppose training predictors are centered and scaled, and PCA produces a loading vector \(v_k=(v_{1k},\ldots,v_{pk})\) for component \(k\). For a record with completed and standardized coordinates \(z_1,\ldots,z_p\), its component score is
The loading vector and the standardization parameters belong together: the loading vector was calculated using training data expressed under that preprocessing rule. To score an assessment record, first apply the training imputation values, then the training centers and scales, and finally the training loadings. A score computed using different centers or scales is not the score on the fitted component axis.
The following calculation isolates this issue. Treat the training centers, scales, and loading vector as fitted quantities. The example does not estimate PCA anew from assessment records.
Worked Example: Applying a Fitted Component to New Records
A two-predictor PCA was fitted after standardizing with training means \((10,20)\) and training sample standard deviations \((2,4)\). Its first loading vector is \(v_1=(1/\sqrt{2},1/\sqrt{2})\). Calculate the first-component score for an assessment record \((14,24)\). Then show the error from standardizing a two-record assessment batch consisting of \((14,24)\) and \((10,20)\) using that batch’s own means and sample standard deviations.
Method. Apply the training transformation before projecting onto the fitted loading vector. This preserves the scale and origin of the component scores.
For the record \((14,24)\), the training-standardized values are \(z_1=(14-10)/2=2\) and \(z_2=(24-20)/4=1\). Therefore,
Now calculate what the incorrect batch-based transformation would do. The assessment-batch means are \((12,22)\). For each predictor, the two values differ from their mean by \(-2\) and \(2\). The sample standard deviation for either predictor is \(\sqrt{(4+4)/(2-1)}=\sqrt{8}\), approximately \(2.8284\). Thus the first record’s batch-standardized coordinates are \(2/\sqrt{8}=1/\sqrt{2}\) for both predictors. Projecting these incorrect coordinates onto the stated loading vector gives \(1/\sqrt{2}\cdot1/\sqrt{2}+1/\sqrt{2}\cdot1/\sqrt{2}=1\).
The batch-based answer, \(1\), differs from the correct training-based score, \(2.1213\). The check is direct: the correct score uses training-standardized coordinates \((2,1)\), while the incorrect score uses \((1/\sqrt{2},1/\sqrt{2})\). The latter describes a different scaling, not the fitted training transformation.
Common mistake avoided: Re-centering or re-scaling assessment records as a batch can make scores depend on which other records happen to be assessed at the same time. It also makes those scores incomparable with training scores calculated under the original transformation.
A Fixed Pipeline for Missing Assessment Values
A practical pipeline needs a rule for a missing value in a new record as well as a rule for missing training values. For example, if the training median is used to fill gaps, an assessment gap receives that same median. The completed value is then standardized using the training mean and standard deviation. It is not standardized first, and it is not replaced by an assessment-set median.
For PCA, this sequence produces a standardized vector that can be projected onto the fitted loading vectors. For clustering, it produces a vector that can be compared with fitted centroids in their training coordinate system. These operations are valid only when the relevant predictors are available and the stored transformations are defined. If a training predictor has zero standard deviation, division by \(s_j=0\) is undefined; that predictor requires a separately specified treatment, such as excluding it from the scaled calculation.
For each predictor, estimate any imputation value from observed training values. Impute training gaps, then estimate the training center and scale. Fit the clustering or PCA using the resulting representation.
Retain the imputation values, centers, scales, and—where applicable—cluster centroids or PCA loadings. These quantities define how a record becomes a derived feature.
Fill missing values with the stored training imputation values, apply the stored training centers and scales, and then calculate distances or component scores using the fitted model.
Confirm that the same predictors, units, imputation rules, scaling parameters, and fitted cluster or PCA quantities were used. A difference at any of these stages can change the derived feature.
Worked Example: Impute and Score an Assessment Record
A training set has two predictors \(x\) and \(y\). After imputing its training gaps, the fitted values are \(m_x=0\), \(\bar{x}=0.5\), \(s_x=1\), \(\bar{y}=0.5\), and \(s_y=1\). The first PCA loading vector fitted on this training representation is \(v_1=(1/\sqrt{2},1/\sqrt{2})\). An assessment record has \(x\) missing and \(y=2\). Calculate its first-component score using the fixed pipeline.
Method. Use the training imputation value before applying the training standardization and PCA loading.
The missing \(x\) value is replaced with \(m_x=0\). The completed record is \((0,2)\). Standardizing gives \(z_x=(0-0.5)/1=-0.5\) and \(z_y=(2-0.5)/1=1.5\). The score is
The arithmetic checks: the two weighted coordinates sum to \(( -0.5+1.5)/\sqrt{2}=1/\sqrt{2}\). Replacing the missing value with an assessment-set statistic instead would change \(z_x\), and therefore change the score on the same fitted component.
Common mistake avoided: An imputed value is not automatically a standardized value. The record must first receive the training imputation value and then be centered and scaled using the training parameters.
Common Traps and Practical Checks
The most consequential error is to let assessment data influence preprocessing quantities. Even without using assessment outcomes, calculating assessment-set medians, means, or standard deviations changes the derived features and means that the fitted transformation is not being applied as specified. The assessment data should be used only to receive the fixed transformation.
Another trap is treating scaling as a cosmetic step. For clustering, scaling changes relative distances and may alter which centroid is nearest. For PCA, scaling changes the coordinates that are projected onto the loadings. The exact numerical effect depends on the fitted data and transformation; it is not safe to assume that cluster labels or component scores remain unchanged.
Before interpreting a derived feature, check that missing-value rules were fixed from training data, that every scale is positive and defined, and that new records used the stored preprocessing quantities. This distinction is especially important when batches arrive at different times: the score for a record should not change merely because it was processed alongside different assessment records.
Check Your Understanding
For each problem, use the stated training quantities as fixed values unless the question asks you to identify a mistake.
- A training predictor has mean \(50\) and sample standard deviation \(10\). An assessment value is \(70\). What is its standardized value? (Answer: \(2\))
- The observed training values for predictor \(x\) are \(2,4,9\), and the fitted imputation rule is the training median. What value replaces a missing \(x\) in an assessment record? (Answer: \(4\))
- Training centers are \((5,20)\), training standard deviations are \((2,5)\), and a fitted cluster centroid is \((0,0)\) in standardized coordinates. Find the Euclidean distance from an assessment record \((7,15)\) to that centroid. (Answer: \(\sqrt{2}\approx1.4142\))
- A fitted PCA loading vector is \((0.6,0.8)\). Training means are \((10,5)\), training standard deviations are \((2,4)\), and an assessment record is \((14,9)\). Find its component score. (Answer: \(2\))
- Two assessment records have values \(14\) and \(10\) for a predictor. A candidate proposes using their batch mean \(12\) and batch sample standard deviation \(\sqrt{8}\) instead of the stored training mean \(10\) and standard deviation \(2\). For the record with value \(14\), what are the correct training-standardized value and the candidate’s batch-standardized value? (Answer: Correct \(2\); batch-based \(1/\sqrt{2}\approx0.7071\))