Why Feature-Importance Stability Matters
A model can rely on different predictors in different samples, even when its overall predictive performance appears similar. That matters when a feature-importance summary is used to explain a model, prioritize data-quality checks, or support a business decision. A ranking that changes substantially across samples may be a fragile summary rather than a dependable account of which predictors the model uses.
The first task is to make the comparison fair. Feature-importance scores depend on how importance is defined, which features are included, and the data on which importance is evaluated. A change in scores is interpretable only after these conditions have been considered. Even then, feature importance describes a model’s predictive behavior; it does not, by itself, tell us that changing a predictor would cause an outcome to change.
Different importance methods answer different questions. For example, a permutation-based score assesses how much a chosen performance measure changes when a predictor’s values are permuted. A model’s internal split or coefficient summary uses a different definition. These scores should not be treated as interchangeable. For a meaningful stability comparison, use the same importance method and a consistent performance measure, target definition, feature set, and score direction. If the model or evaluation population differs, report that context rather than implying the scores are directly equivalent.
Comparing Two Complete Rankings
One summary of agreement between two complete rankings is Spearman’s rank correlation. It is the ordinary correlation between the two sets of ranks. A value near 1 indicates similar ordering; a value near -1 indicates nearly reversed ordering; a value near 0 indicates little overall rank association. It summarizes the ordering, not the size of score differences.
The formula follows from the definition of correlation. With no ties, each ranking contains the numbers \(1,2,\ldots,p\), whose mean is \((p+1)/2\). The sum of squared deviations from that mean is \(Q=p(p^2-1)/12\). Both rank vectors therefore have the same centered sum of squares \(Q\). Their Pearson correlation is their centered cross-product divided by \(Q\). Also, $$ \sum_{j=1}^{p}(r_j-s_j)^2 =\sum_{j=1}^{p}\left[(r_j-\bar r)-(s_j-\bar s)\right]^2 =2Q-2\sum_{j=1}^{p}(r_j-\bar r)(s_j-\bar s), $$ because the rank means are equal. Dividing the last expression by \(2Q\) gives \(1-\rho_s\). Substituting \(Q=p(p^2-1)/12\) yields the stated formula. The no-ties condition is what allows this particular shortcut from squared rank differences.
Worked Example: Comparing Rankings Across Two Samples
A hypothetical insurer fits the same model specification on two samples and calculates permutation importance using the same evaluation metric. The normalized scores are as follows. Higher scores mean greater importance.
Sample 1: claim frequency 0.36, claim severity 0.29, policy tenure 0.21, region 0.14.
Sample 2: claim frequency 0.31, claim severity 0.18, policy tenure 0.24, region 0.27.
Step 1: Check the scores. Each sample’s scores sum to 1: \(0.36+0.29+0.21+0.14=1.00\), and \(0.31+0.18+0.24+0.27=1.00\). The problem states that the same importance method and metric were used.
Step 2: Assign ranks, with rank 1 for the largest score. In Sample 1, the order is claim frequency, claim severity, policy tenure, region. In Sample 2, it is claim frequency, region, policy tenure, claim severity.
Step 3: Calculate rank differences. In the feature order just listed, the ranks and differences \(d_j=r_j-s_j\) are:
Claim frequency: \(1-1=0\). Claim severity: \(2-4=-2\). Policy tenure: \(3-3=0\). Region: \(4-2=2\). Thus \(d=(0,-2,0,2)\) and \(\sum d_j^2=0^2+(-2)^2+0^2+2^2=8\).
Step 4: Calculate and interpret Spearman’s correlation. Here \(p=4\), so $$ \rho_s=1-\frac{6(8)}{4(4^2-1)} =1-\frac{48}{60} =0.20. $$ The rankings have only modest rank agreement: claim frequency stays first and policy tenure stays third, but the relative ordering of claim severity and region reverses. This is evidence of ranking instability across these two samples, not proof that either feature’s causal influence changed.
Method and mistake avoided. The method is to rank scores within each sample and compare the ranks, rather than compare feature names or rank numbers informally. A common mistake would be to conclude that region became the most important predictor overall; it moved to second, not first, in Sample 2.
Stability Across Repeated Fits
A comparison of just two rankings can conceal how much a feature’s rank varies across repeated samples. One practical approach is to refit the same model and importance procedure on multiple resamples, then summarize each feature’s rank distribution. Useful summaries include the mean or median rank, the range of ranks, and the proportion of fits in which a feature appears among the top few. These summaries describe the fits that were run; they do not guarantee how a feature will rank in every future population.
Worked Example: Summarizing Ranks Across Resamples
A hypothetical pricing team obtains rankings from five bootstrap resamples, using the same features and importance method each time. The resulting ranks are shown below; rank 1 is most important.
| Feature | Fit 1 | Fit 2 | Fit 3 | Fit 4 | Fit 5 |
|---|---|---|---|---|---|
| Exposure | 1 | 1 | 2 | 1 | 2 |
| Claim frequency | 2 | 3 | 1 | 2 | 3 |
| Territory | 3 | 2 | 3 | 4 | 1 |
| Sales channel | 4 | 4 | 4 | 3 | 4 |
Step 1: Calculate mean ranks. Exposure has mean rank \((1+1+2+1+2)/5=7/5=1.4\). Claim frequency has \((2+3+1+2+3)/5=11/5=2.2\). Territory has \((3+2+3+4+1)/5=13/5=2.6\). Sales channel has \((4+4+4+3+4)/5=19/5=3.8\).
Step 2: Count top-ranked appearances. Exposure is rank 1 in three of five fits, so its top-rank proportion is \(3/5=0.60\). Claim frequency and territory are each rank 1 once, giving \(1/5=0.20\) each. Sales channel is never rank 1, giving \(0/5=0\).
Step 3: Interpret both summaries. Exposure has the best mean rank and is most often ranked first, but it does not rank first in every fit. Territory’s mean rank of 2.6 hides a wide range, from rank 1 to rank 4. Reporting both average rank and variability is more informative than presenting one preferred ordering as if it were fixed.
Method and mistake avoided. The method is to summarize ranks across resamples, not to treat the five resamples as five independent populations or as proof of a particular causal ordering. With only five fits, these proportions are descriptive and coarse.
Comparing Importance Across Time
For a time-based comparison, specify the periods, population, model, and importance procedure. If a model is evaluated in each period, changes in feature scores may reflect changes in predictor distributions, outcome patterns, or relationships among predictors. Earlier material on covariate shift and concept drift distinguishes changes in predictor distributions from changes in the relationship between predictors and outcomes. Importance rankings add a model-focused diagnostic, but do not by themselves identify which explanation is responsible.
Worked Example: A Ranking Shift with Similar AUC
A hypothetical insurer evaluates the same classification approach in two periods. It calculates permutation importance using the same metric and reports the following values: age has importance 0.30 in Period 1 and 0.22 in Period 2; renewal gap has 0.24 and 0.29; payment history has 0.16 and 0.15. The model’s AUC is 0.78 in Period 1 and 0.77 in Period 2.
Step 1: Rank the importance values. In Period 1, age ranks 1, renewal gap ranks 2, and payment history ranks 3. In Period 2, renewal gap ranks 1, age ranks 2, and payment history ranks 3.
Step 2: Compare rankings. In feature order age, renewal gap, payment history, the rank differences are \(1-2=-1\), \(2-1=1\), and \(3-3=0\). Thus \(d=(-1,1,0)\), and \(\sum d_j^2=1+1+0=2\). With \(p=3\), $$ \rho_s=1-\frac{6(2)}{3(3^2-1)} =1-\frac{12}{24} =0.50. $$ The rankings show a change in the leading feature, while payment history remains third. The AUC decreases by \(0.78-0.77=0.01\); the ranking shift therefore occurs alongside similar, though not identical, measured discrimination.
Step 3: State what can be concluded. The results support reporting that age and renewal gap exchanged ranks under this importance procedure, while AUC changed by 0.01. They do not show that age has less causal influence on the outcome or that renewal gap has more. The scores describe how the fitted model’s predictive attribution behaves under the specified procedure and data; causal claims require evidence and assumptions beyond these rankings.
Method and mistake avoided. The method is to report the rank comparison alongside the performance context and the limits of the measure. A common mistake is to interpret rank 1 as the feature with the greatest causal effect. Rank 1 means only that the feature received the largest score under the stated importance method in that period.
Common Traps and Responsible Interpretation
Importance scores are conditional summaries. Correlated predictors can share or substitute for predictive information, so one feature may rank higher in one sample and a related feature higher in another. A ranking change can therefore occur even when prediction quality is similar. Conversely, stable rankings do not establish that a model is accurate, fair, or suitable for a decision.
Normalization also needs care. Suppose one period has raw importance scores that sum to 10, with a feature’s score equal to 4; its normalized share is \(4/10=0.40\). In another period the total is 20, while that feature’s raw score remains 4; its normalized share is \(4/20=0.20\). The wrong calculation is to say, “The feature’s causal effect fell by \((0.40-0.20)/0.40=50\%\).” The arithmetic describes a 50% reduction in the feature’s normalized share, but that share changed because the total score changed. It does not demonstrate a change in absolute predictive contribution, and it certainly does not estimate a causal effect.
When presenting stability results, state the importance method, the features compared, the sample or period, and the summary used. If ties occur, say how they were handled and use a method appropriate to tied ranks. If the feature set differs between comparisons, identify which features were not comparable rather than silently assigning them ranks. A practical summary often combines a ranking comparison with score values and model-performance context.
Name the samples or periods and define the importance method, metric, score direction, and feature set.
Use an appropriate rank comparison for two rankings, or report rank distributions and top-rank frequencies across repeated fits.
Describe which model-importance summaries changed, then state that the change alone does not establish a causal change.
Check Your Understanding
For each problem, use rank 1 for the largest importance and state only what the calculation supports.
- Three features have ranks A=1, B=2, C=3 in one sample and A=2, B=1, C=3 in another. Calculate Spearman’s rank correlation. (Answer: \(d=(-1,1,0)\), \(\sum d_j^2=2\), and \(\rho_s=0.50\).)
- Four features have first-sample ranks A=1, B=2, C=3, D=4 and second-sample ranks A=3, B=1, C=2, D=4. Calculate Spearman’s rank correlation. (Answer: \(d=(-2,1,1,0)\), \(\sum d_j^2=6\), and \(\rho_s=0.40\).)
- Across five repeated fits, a feature has ranks \(1,2,1,3,1\). Find its mean rank and proportion of fits in which it ranks first. (Answer: mean rank \(=8/5=1.6\); top-rank proportion \(=3/5=0.60\).)
- A normalized importance score for a feature changes from 0.30 to 0.24. Calculate its relative percentage decrease, then state whether that calculation establishes a change in causal influence. (Answer: \((0.30-0.24)/0.30=0.20\), a 20% relative decrease in the normalized score; no causal change is established.)
- Two periods have rankings with Spearman correlation 0.90, but a model’s AUC falls from 0.82 to 0.74. What does the correlation describe, and does it establish stable performance? (Answer: it describes strong agreement in feature ordering; no, it does not establish stable performance, since the reported AUC decreased by 0.08.)