Why Use Cluster Membership in a Prediction Model?
A clustering method assigns records to groups using patterns in predictor variables, without using the outcome being predicted. Those assignments can then be supplied to a supervised model as a new categorical predictor. The model can estimate whether records in different learned groups tend to have different outcomes.
The cluster numbers are labels, not measurements. Cluster 3 is not necessarily “more” than cluster 2, and the numbering can change if the clustering procedure returns the same groups in a different order. The useful information is which group a record belongs to, not the numerical value of its cluster ID.
This predictor can summarize several related characteristics of a record in a compact form. It may be useful when records fall into distinct profiles, but it does not guarantee better predictions. The appropriate comparison is predictive performance on data not used to fit the supervised model, with the clustering and other preprocessing steps handled consistently.
Encoding Membership as a Factor
In R, store the cluster assignment as a factor before including it in a model. For example, if a data frame named dat has a column named cluster, use dat$cluster_factor <- factor(dat$cluster). A linear model with a separate fitted mean for each cluster can be written as lm(y ~ 0 + cluster_factor, data = dat). The 0 + requests a no-intercept parameterization in which each coefficient corresponds to a cluster-specific mean.
For a generalized linear model, the same factor can enter the linear predictor. For example, glm(event ~ 0 + cluster_factor, family = binomial, data = dat) fits cluster-specific log odds under a logistic model. The model family and link determine what its fitted values and coefficients mean.
Do not leave IDs as ordinary numbers in the formula. A model using the numeric value of cluster treats a one-unit increase from cluster 1 to cluster 2 as comparable to an increase from cluster 2 to cluster 3. That imposes an ordering and a particular trend across the labels, neither of which follows from clustering.
Worked Example: Fitting Cluster-Specific Means
A hypothetical training sample contains ten records assigned to three clusters by a procedure that did not use the response. The observed continuous outcomes are: cluster A, 4, 6, 8; cluster B, 10, 12, 14; and cluster C, 5, 7, 9, 11. Fit the cluster-only ordinary least squares model and calculate its fitted outcome for each group. Assume independent errors with a common mean of zero within each cluster; for standard small-sample inference, also assume normally distributed errors with common variance. No distributional assumption is needed simply to calculate the least-squares fitted means.
Method. Treat cluster as a factor and fit a separate mean for each group. In R, the model is lm(y ~ 0 + cluster_factor, data = dat).
Cluster A. Its outcome total is \(4+6+8=18\), and its size is 3. Its fitted mean is \(18/3=6\).
Cluster B. Its total is \(10+12+14=36\), and its size is 3. Its fitted mean is \(36/3=12\).
Cluster C. Its total is \(5+7+9+11=32\), and its size is 4. Its fitted mean is \(32/4=8\).
The model therefore predicts 6 for a record in A, 12 for a record in B, and 8 for a record in C. The estimated differences are B minus A, \(12-6=6\); C minus A, \(8-6=2\); and B minus C, \(12-8=4\), in the outcome’s units.
As a check, the residuals within A are \(-2,0,2\), which sum to 0. Within B they are \(-2,0,2\), also summing to 0. Within C they are \(-3,-1,1,3\), again summing to 0. This is consistent with each fitted value being its group’s least-squares mean. The common mistake avoided here is treating the labels A, B, and C as if they were an ordered numeric scale.
Why the Least-Squares Fit Is a Group Mean
For a cluster-only linear model with an intercept for each group, the fitted value for every record in cluster \(c\) is a parameter \(\mu_c\). Ordinary least squares chooses the parameters to minimize the residual sum of squares. If cluster \(c\) has \(n_c\) observations, its contribution to that sum is:
Different clusters have separate parameters, so minimizing the total residual sum of squares can be done separately for each cluster. Differentiate the cluster contribution with respect to \(\mu_c\), set the derivative to zero, and solve:
The second derivative is \(2n_c\), which is positive when the cluster contains at least one observation. Thus the stationary point is the unique minimum. This explains why the no-intercept linear model with a factor produces the sample mean in each cluster. The result is specific to the least-squares model with a separate mean per group; it does not mean that every model’s coefficient is a raw group average.
Interpreting a Logistic Model
For a binary outcome, suppose observations are independent Bernoulli trials within clusters, and the probability of an event is constant within each cluster. A cluster-only logistic model expresses each probability \(p_c\) through its log odds, \(\log(p_c/(1-p_c))\). In a no-intercept factor model, each cluster has its own log-odds coefficient. The predicted probability is obtained by applying the inverse-logit transformation to that coefficient.
Worked Example: Comparing Event Probabilities Across Clusters
In a hypothetical sample, the event occurs for 18 of 30 records in cluster A, 12 of 30 in cluster B, and 6 of 30 in cluster C. Fit a cluster-only logistic model and compare the predicted event probabilities and the odds for A versus C. Assume independent Bernoulli outcomes and a common event probability within each cluster. Each group contains both event and non-event observations, so its estimated probability is not 0 or 1.
Method. Fit glm(event ~ 0 + cluster_factor, family = binomial, data = dat). In this no-intercept parameterization, the cluster coefficients are the estimated log odds.
Cluster A. The estimated event probability is \(18/30=0.6000\). Its odds are \(0.6/(1-0.6)=0.6/0.4=1.5\), so its log-odds coefficient is \(\log(1.5)\), approximately \(0.4055\).
Cluster B. The estimated probability is \(12/30=0.4000\). Its odds are \(0.4/0.6=2/3\), so its log-odds coefficient is \(\log(2/3)\), approximately \(-0.4055\).
Cluster C. The estimated probability is \(6/30=0.2000\). Its odds are \(0.2/0.8=0.25\), so its log-odds coefficient is \(\log(0.25)\), approximately \(-1.3863\).
The inverse-logit of each coefficient returns its group probability: for example, the inverse-logit of \(\log(0.25)\) is \(0.25/(1+0.25)=0.2000\). The predicted probability in A exceeds that in C by \(0.6000-0.2000=0.4000\). The odds ratio comparing A with C is \(1.5/0.25=6\). Thus the estimated odds of the event are six times as high in A as in C in this fitted model; this is not a statement that the probability is six times as high. The common mistake avoided is interpreting a logistic coefficient or odds ratio as a probability difference.
Why Factor Coding Matters
A factor tells the model to treat the labels as categories. By contrast, if the same group IDs are supplied as numbers, a linear model fits a numerical relationship between the ID and the outcome. That can hide real group differences or create a trend that has no substantive basis. The following example makes the distinction explicit.
Worked Example: Avoiding a False Trend From Numeric IDs
A hypothetical data set has two outcomes in each of three clusters: cluster 1 has 8 and 12; cluster 2 has 18 and 22; cluster 3 has 8 and 12. Compare the fitted values from a model using cluster as a factor with those from ordinary linear regression using the numeric cluster ID. Assume the usual independent, mean-zero error condition for interpreting the linear fits as estimates of conditional means.
Factor model. The group means are \((8+12)/2=10\), \((18+22)/2=20\), and \((8+12)/2=10\). Therefore, the factor model predicts 10, 20, and 10 for clusters 1, 2, and 3, respectively.
Numeric-ID model. Across the six records, the mean cluster ID is \((1+1+2+2+3+3)/6=2\). The mean outcome is \((8+12+18+22+8+12)/6=80/6=40/3\). For each cluster ID, the two outcomes average to 10, 20, and 10. The centered cross-product across IDs is proportional to \((1-2)(10-40/3)+(2-2)(20-40/3)+(3-2)(10-40/3)\), which equals \(10/3+0-10/3=0\). Thus the estimated numeric-ID slope is 0, and its fitted value is the overall mean \(40/3\), or approximately 13.333 for all three clusters.
The numeric model misses the higher mean in cluster 2 because its straight-line trend cannot represent the pattern 10, 20, 10. The factor model retains the separate group means. In R, encode the IDs as factor(cluster) rather than relying on their numeric storage type. The mistake avoided is assuming that consecutive cluster labels represent equal steps along a meaningful scale.
Interpretation, Limitations, and Workflow Checks
A difference between cluster-specific predictions is an association produced by the fitted model. It does not establish that membership causes an outcome to change: cluster membership is a summary derived from observed predictors, not a randomized treatment. Also, a cluster-only model predicts the same value for every record in a given cluster. It cannot distinguish records within that group unless additional predictors are included.
Before interpreting results, check that each factor level has training observations and that the model has enough information to estimate the requested quantities. Sparse or absent groups can make estimates unstable or impossible. For binary outcomes, a group with no events or no non-events can lead to a boundary estimate or separation in logistic regression, so the simple finite log-odds calculation in the preceding example would not apply.
Cluster construction must also respect the training and validation design. Learn the clustering transformation using training data, then use the established transformation consistently when constructing predictors for validation data. Do not use validation outcomes to define clusters or revise the supervised fit. This keeps outcome information from leaking into model development and makes a validation comparison meaningful.
Confirm that they are group identifiers, not measurements with a meaningful numeric ordering.
Use a categorical predictor in R so the supervised model does not impose a trend across arbitrary IDs.
For a least-squares group model, predictions are group means; for a logistic model, coefficients are log odds and fitted probabilities require the inverse-logit.
Report predicted outcome contrasts as model-based associations, and assess predictive value on data kept separate from model fitting.
Using cluster membership as a factor is a direct way to ask whether unsupervised groups correspond to different predicted outcomes. The method is aligned with the SOA Exam PA (Predictive Analytics) syllabus, where sound encoding and careful interpretation matter as much as fitting the model.
Check Your Understanding
For each problem, identify the appropriate model or calculation and check whether the interpretation respects the categorical nature of cluster labels.
- A continuous response has values 3, 5, 7 in cluster R and 9, 11 in cluster S. Under a cluster-only least-squares model, what are the fitted values for R and S, and what is S minus R? (Answer: R = 5, S = 10, S minus R = 5.)
- A binary outcome occurs 8 times among 20 records in cluster M. Under a cluster-only logistic model, what is the fitted probability and the estimated log odds? (Answer: probability = 0.4000; log odds = \(\log(2/3)\), approximately \(-0.4055\).)
- In R, the cluster IDs are stored as integers 1, 2, and 3, with no natural ordering. What should be done before using them as a categorical predictor? (Answer: convert the IDs to a factor, for example with factor(cluster).)
- Two clusters have fitted event probabilities 0.50 and 0.25. Calculate the difference in probabilities and the odds ratio for the first cluster versus the second. (Answer: probability difference = 0.25; odds ratio = \((0.50/0.50)/(0.25/0.75)=3\).)
- A clustering transformation is learned using training predictors. A validation record is then processed using information from its validation outcome to decide its cluster. What is the problem? (Answer: outcome leakage; the validation outcome must not be used to construct the predictor.)