Why Trees Need Surrogate Splits
A tree makes a prediction by routing an observation through a sequence of splits. At a node, a primary split might send records with \(x < 50\) to the left child and the remaining records to the right. If \(x\) is missing for a new record, that rule cannot be applied directly. A surrogate split offers an alternative route based on another predictor that tended to send the same training records in the same direction.
A surrogate is a routing substitute, not a replacement primary split and not a separate prediction model. It is selected for how well its branches reproduce the primary split among records where both predictors are observed. Its agreement describes similarity to the primary routing decision; it does not directly measure whether the tree predicts the outcome correctly.
Measuring Agreement
For a binary split, let \(p_L\) and \(p_R\) be the proportions assigned left and right by the primary split, and let \(q_L\) and \(q_R\) be the corresponding proportions assigned by a proposed surrogate. These proportions are computed on the same eligible records. The surrogate can be oriented either way: its left branch may be mapped to the primary left branch, or to the primary right branch. Use the orientation with greater agreement.
Write \(A\) for the observed proportion of eligible records whose surrogate assignment agrees with the primary assignment, after choosing the better orientation. The chance-agreement reference \(E\) is the agreement expected if the primary and surrogate assignments were independent while retaining their observed branch proportions. It is a benchmark, not a claim that the assignments in the data actually are independent.
A common adjusted-agreement measure is
This adjustment sets chance-level agreement to zero and scales the difference against the theoretical agreement of 1. That theoretical endpoint need not be attainable with the observed branch proportions. The formula is undefined if \(E=1\); this occurs when both splits assign every eligible record to the same branch, so the data provide no variation for assessing agreement.
Worked Example: Adjusting Agreement When Perfect Agreement Is Impossible
At a node, 10 eligible records are split primarily into 6 left and 4 right. A candidate surrogate sends 4 records left and 6 right. After choosing its orientation, 6 of the 10 records agree with the primary split. Calculate raw agreement, chance agreement, adjusted agreement, and the largest agreement possible with these fixed branch proportions.
Method. Use the observed margins to calculate \(E\), then apply the adjusted-agreement formula. The primary proportions are \(p_L=6/10=0.6\) and \(p_R=4/10=0.4\). The surrogate proportions are \(q_L=4/10=0.4\) and \(q_R=6/10=0.6\).
Raw agreement is \(A=6/10=0.60\). The chance reference is \(E=(0.6)(0.4)+(0.4)(0.6)=0.24+0.24=0.48\). Therefore, \[ \frac{A-E}{1-E}=\frac{0.60-0.48}{1-0.48} =\frac{0.12}{0.52}=0.2308 \] to four decimal places.
To find the largest possible agreement given these margins, match as many records as possible within each branch. At most \(\min(0.6,0.4)=0.4\) can match in the left branch and \(\min(0.4,0.6)=0.4\) in the right branch. Thus the largest attainable agreement is \(0.4+0.4=0.8\), not 1. Its adjusted value would be \((0.8-0.48)/(1-0.48)=0.32/0.52=0.6154\), to four decimal places.
Check. The observed agreement, 0.60, is below the attainable maximum, 0.80; its adjusted value is consequently below 0.6154. The common mistake is to call 1 the largest possible agreement for these fixed margins. In the adjustment formula, 1 is a theoretical reference endpoint, not necessarily an attainable agreement.
Why the Adjustment Has This Form
The chance reference follows from a simple calculation. If primary and surrogate assignments were independent, the probability that both assign a record left would be \(p_Lq_L\). The probability that both assign it right would be \(p_Rq_R\). These are the only two agreeing outcomes, so adding their probabilities gives \(E=p_Lq_L+p_Rq_R\).
The adjusted measure then rescales agreement above this reference. Subtracting \(E\) makes the value zero when observed agreement equals the chance reference. Dividing by \(1-E\) makes the value one if \(A=1\), provided \(E<1\):
These endpoint properties do not mean that the value 1 is always attainable for the observed margins. In general, with branch proportions \(p_i\) and \(q_i\), the largest possible agreement is \(\sum_i \min(p_i,q_i)\): each branch can match no more records than appear in the smaller of its two margins. When that maximum is less than 1, the largest possible adjusted agreement is also less than 1. This distinction matters when interpreting a surrogate score.
How rpart Uses Surrogates
In rpart, surrogate search is controlled by maxsurrogate, which limits how many surrogate splits are retained at a node, and usesurrogate, which controls how they are used for routing. A typical classification-tree fit can request several surrogates and majority-direction fallback as follows:
For a record with a missing primary predictor, the tree tries the available surrogate splits in their stored order. The first usable surrogate determines the child. With usesurrogate = 2, if the primary predictor and all available surrogates are missing, the record is sent to the primary split’s majority direction. With usesurrogate = 1, that majority-direction fallback is not used when all the split predictors are missing. Setting maxsurrogate = 0 prevents surrogate splits from being retained.
The printed tree and its summary can be used to inspect primary and surrogate rules. Surrogate information typically reports a direction for the rule and values such as agree and adj. Interpret those as split-routing statistics on eligible records, not as measures of classification accuracy or regression error. The model’s prediction for a record is still determined by the terminal node it reaches.
Worked Example: Tracing rpart Routing at a Node
A primary rule sends \(x < 50\) left and \(x \geq 50\) right. Among 12 eligible training records, 7 go left and 5 go right. Surrogate A uses \(a < 20\) for left and \(a \geq 20\) for right. Its cross-tabulation against the primary rule is shown below. Surrogate B uses \(b < 40\) for left and \(b \geq 40\) for right.
| Primary assignment | A left | A right | B left | B right |
|---|---|---|---|---|
| Left | 6 | 1 | 5 | 2 |
| Right | 2 | 3 | 2 | 3 |
Method. Compare the candidate surrogates by calculating adjusted agreement from their margins, then apply the first available rule. For A, the surrogate margins are 8 left and 4 right, and the diagonal agreement is \(6+3=9\), so \(A_A=9/12=3/4\). Its chance reference is \[ E_A=(7/12)(8/12)+(5/12)(4/12) =76/144=19/36. \] The adjusted agreement is \[ \frac{3/4-19/36}{1-19/36} =\frac{27/36-19/36}{17/36} =\frac{8}{17}=0.4706 \] to four decimal places.
For B, the surrogate margins are 7 left and 5 right, and its diagonal agreement is \(5+3=8\), so \(A_B=8/12=2/3\). Its chance reference is \[ E_B=(7/12)(7/12)+(5/12)(5/12)=74/144=37/72. \] Its adjusted agreement is \[ \frac{2/3-37/72}{1-37/72} =\frac{48/72-37/72}{35/72} =\frac{11}{35}=0.3143 \] to four decimal places. A has the higher adjusted agreement.
With usesurrogate = 2, consider four records at this node. If \(x\) is missing and \(a=14\), A sends the record left. If \(x\) and \(a\) are missing but \(b=52\), B sends it right. If \(x\), \(a\), and \(b\) are all missing, the primary split’s majority direction is left because 7 of 12 eligible records went left. Finally, if \(x=61\), the primary rule sends the record right even if A would send it left: the primary rule takes precedence when it is available.
Check. The calculations rank A above B, and the four routes follow the primary-first, surrogate-next, majority-fallback order. A common mistake is to use a surrogate even when the primary predictor is observed.
Assessing Surrogate Routing Carefully
Surrogate agreement is conditional on having both predictors observed at the node. If missingness changes which records are eligible, the calculated agreement may not describe the records that will need surrogate routing. For example, records with a missing primary predictor could differ systematically from those with a recorded value. A high training agreement is therefore not a guarantee of equally high agreement among cases that will be missing in use.
A useful diagnostic is to evaluate routing on held-out records whose primary predictor is initially known: temporarily hide that value, route using the surrogate rules, and compare the resulting branch with the branch the primary rule would have taken. This measures agreement with the primary routing decision under the test’s masking setup. It is not the same as testing the tree’s outcome predictions, and the masking setup should reflect the missingness pattern expected in practice as closely as possible.
Worked Example: Separating Routing Agreement From Outcome Accuracy
A fitted tree has a primary split with left and right directions. For a held-out routing audit, analysts know the original primary direction for 10 records, then mask the primary predictor. The first surrogate is available for 8 records. Its assignments agree with the original primary direction for 7 of those 8. For the remaining 2 records, all surrogate predictors are missing; the tree’s majority-direction fallback sends both left. The original primary directions for those two records were one left and one right. Calculate surrogate coverage, agreement conditional on coverage, the adjusted agreement among records with a usable surrogate, and total routing agreement after fallback.
Method. Keep the denominators separate: surrogate coverage uses all 10 records, while conditional agreement uses only the 8 with an available surrogate. Coverage is \(8/10=0.80\). Conditional raw agreement is \(7/8=0.875\).
Among the 8 records with a usable surrogate, suppose the cross-tabulation is: primary-left records include 4 surrogate-left and 1 surrogate-right; primary-right records include 0 surrogate-left and 3 surrogate-right. The primary margins are therefore \(p_L=5/8\), \(p_R=3/8\), and the surrogate margins are \(q_L=q_R=4/8\). The observed agreement is \(A=(4+3)/8=7/8\). The chance reference is \[ E=(5/8)(4/8)+(3/8)(4/8)=32/64=0.5. \] Thus adjusted agreement among the 8 usable cases is \[ \frac{7/8-1/2}{1-1/2} =\frac{3/8}{1/2}=0.75. \]
For the two records without a usable surrogate, the majority fallback is correct for one of them. Total agreement with the original primary direction after masking is therefore \(7+1=8\) out of 10, or \(0.80\). The result checks because the total consists of seven correct surrogate routes and one correct fallback route. The common mistake is to call \(8/10\) outcome accuracy: it is only routing agreement against the original primary split.
Practical Interpretation and Common Traps
Surrogates can preserve a tree’s ability to route records when predictors are missing, but their usefulness depends on the quality and availability of alternative predictors. Interpret them at the node where they are defined: agreement at one node does not imply the same rule or performance elsewhere in the tree.
- Do not equate adjusted agreement with the fraction routed correctly. Raw agreement is \(A\); adjusted agreement subtracts a chance reference and rescales it.
- Do not assume the theoretical maximum is attainable. With fixed margins, the maximum is \(\sum_i\min(p_i,q_i)\), which may be below 1.
- Do not calculate agreement using records missing either split variable. They cannot establish whether the surrogate matches the primary assignment.
- Do not overlook the fallback rule. When no split predictor is available, the specified rpart setting determines whether the record is sent to the majority direction.
- Do not read routing agreement as outcome performance. A surrogate imitates a branch assignment; the tree’s prediction and its outcome accuracy are separate questions.
Check Your Understanding
For each problem, identify the relevant margins and distinguish agreement with a primary split from prediction accuracy.
- A primary split sends 70% of eligible records left and 30% right. A surrogate sends 50% left and 50% right, with observed agreement \(A=0.65\). Calculate \(E\), adjusted agreement, and the maximum attainable raw agreement from these margins. (Answer: \(E=0.50\), adjusted agreement \(=0.3000\), maximum raw agreement \(=0.80\).)
- At a node, 9 of 12 records go primary-left and 3 go primary-right. A surrogate has the same margins and agrees on 10 records. Calculate the chance reference and adjusted agreement. (Answer: \(E=(9/12)^2+(3/12)^2=0.625\); adjusted agreement \(=(10/12-0.625)/(1-0.625)=0.5556\), to four decimal places.)
- A fitted rpart node uses
usesurrogate = 2. The primary predictor is missing, the first surrogate is missing, and the second surrogate is observed and routes right. Which child is used, and does the majority fallback apply? (Answer: The second surrogate routes the record right; the majority fallback does not apply because a usable surrogate is available.) - At a node, 14 training records go primary-left and 6 go primary-right. A new record is missing the primary predictor and every retained surrogate. With majority-direction fallback, which child receives it? If an available surrogate instead routes it right, which rule takes precedence? (Answer: It goes left under fallback; an available surrogate routes it right, so the fallback is not used.)
- In an eligible sample, both the primary split and a surrogate send every record left. State \(A\), \(E\), and whether adjusted agreement is defined. Explain what additional feature of the data would be needed to assess split agreement meaningfully. (Answer: \(A=1\), \(E=1\), and adjusted agreement is undefined because the formula is \(0/0\); the eligible records need variation in the branch assignments.)