Handling Tied And Nearly Tied Candidates

Why Ties Need an Explicit Rule

Subset selection compares candidate models using a score. Sometimes one candidate has a uniquely best score; sometimes two or more candidates share the best score, or have scores so close that a practical rule treats them as equivalent. The numerical comparison alone may then fail to identify a unique model.

This matters because candidate models can contain different predictors and different numbers of predictors. If two candidates tie, a stated secondary rule—such as choosing the smaller model—can determine both which subset is reported and its size. If the rule is not stated, two analysts can report different selections while agreeing on every score.

Definition: Let \(\mathcal{C}\) be the finite set of candidate models being compared, and let \(Q(M)\) be the score for candidate model \(M\). Assume the selection procedure is defined to prefer smaller scores. An exact tie for the best score occurs when two or more candidates attain the minimum \(Q_{\min}=\min_{M\in\mathcal{C}}Q(M)\). A near tie occurs when candidates have different scores, but a specified tolerance treats their differences from the best score as negligible for the selection decision.

The direction of preference must be clear before comparing numbers. If a score is defined so that larger values are better, select a maximum instead; do not apply a minimum-score rule automatically. In the examples below, smaller scores are preferred. Model size \(k(M)\) means the number of predictors in \(M\), excluding the intercept.

Exact Ties: The Secondary Rule Determines the Report

For a score that is minimized, the set of exact best-scoring candidates is

\mathcal{B}=\{M\in\mathcal{C}:Q(M)=Q_{\min}\}.

If \(\mathcal{B}\) contains more than one model, the primary score does not distinguish its members. A tie-breaking rule supplies an additional decision. Common choices include selecting the candidate with fewer predictors or using a predetermined ordering of predictors or candidate models. A rule should be selected before interpreting the tie, applied consistently, and reported with the result.

Key result: A tie-breaker selects among the primary score’s best candidates; it does not make one tied candidate better according to that score. If the rule is “choose the fewest predictors,” select a model \(M\in\mathcal{B}\) having the smallest \(k(M)\). If multiple such models remain, a further rule is needed to choose one uniquely.

The justification is direct. For every \(M\in\mathcal{B}\), \(Q(M)=Q_{\min}\). Therefore, choosing one tied model rather than another leaves the best attainable primary score unchanged. The tie-breaker acts only on the set of primary minimizers. It can nevertheless change which predictors appear in the reported model and, if tied candidates have different sizes, the reported model size.

M_1,M_2\in\mathcal{B} \quad\Longrightarrow\quad Q(M_1)=Q(M_2)=Q_{\min}.

Worked Example: An Exact Tie With Different Model Sizes

A risk analyst compares three candidate models using a score for which smaller is better. The candidates and their full-precision scores are: \(M_A=\{x_1,x_2\}\), score \(18.60\); \(M_B=\{x_1,x_3\}\), score \(18.60\); and \(M_C=\{x_1,x_2,x_3\}\), score \(18.60\). Apply the rule “among exact best-score ties, choose the model with the fewest predictors.” State the selected subset, its size, and its score.

Method. First find the minimum score. Each candidate has score \(18.60\), so \(Q_{\min}=18.60\), and all three candidates belong to the exact best-scoring set.

Compare sizes. \(k(M_A)=2\), \(k(M_B)=2\), and \(k(M_C)=3\). The smallest size in the tied set is 2, so \(M_C\) is not selected under the stated rule. The rule leaves \(M_A\) and \(M_B\) tied.

Resolve the remaining tie. The stated rule does not distinguish \(M_A\) from \(M_B\). If the analyst’s predetermined secondary rule is to choose the candidate listed first, select \(M_A=\{x_1,x_2\}\). The reported model has two predictors and score \(18.60\).

Check and common mistake avoided. The selected score equals the minimum, and the selected size is the smallest size among the best-scoring candidates. Choosing \(M_C\) because it contains both \(x_2\) and \(x_3\) would invent a preference not present in the score or the stated tie-breaker.

Near Ties: Define the Tolerance Before Applying It

Unlike exact equality, a near tie requires a practical convention. There is no universal difference that makes two scores “close enough”: a difference that is negligible on one score scale may be meaningful on another. A tolerance should therefore be stated in the context of the score and used consistently.

One common convention combines an absolute tolerance \(\tau_{\mathrm{abs}}\) with a relative tolerance \(\tau_{\mathrm{rel}}\). For a candidate \(M\), define its difference from the best score as \(\Delta(M)=Q(M)-Q_{\min}\). With lower scores preferred, a candidate is treated as near-tied with the best if

0\leq \Delta(M) \leq \max\!\left\{\tau_{\mathrm{abs}}, \tau_{\mathrm{rel}}\max\!\left(|Q(M)|,|Q_{\min}|\right)\right\}.

The absolute tolerance is useful when score differences have a meaningful fixed scale. The relative tolerance allows the threshold to grow with the magnitude of the scores. Both tolerances may be set to zero when only exact ties count. The rule must be specified; a reader should not infer a near-tie threshold from the reported numbers alone.

Once the near-tied set has been formed, a secondary rule can select within it, for example the candidate with fewer predictors. This can mean selecting a model whose score is slightly worse than the numerical minimum. Report both facts: the candidate is within the declared tolerance, and its score is not the exact minimum.

Worked Example: Applying an Absolute Near-Tie Tolerance

A pricing team compares three candidate models, with smaller scores preferred. Their scores are \(Q(M_1)=0.8426\), \(Q(M_2)=0.8431\), and \(Q(M_3)=0.8460\). Their sizes are 4, 2, and 1 predictors, respectively. The selection rule treats candidates within an absolute difference of \(0.0010\) from the minimum as near-tied, then selects the smallest model in that set. Find the selected model and distinguish its score from the minimum.

Method. The minimum is \(Q_{\min}=0.8426\), attained by \(M_1\). Compute each candidate’s difference from that minimum:

\Delta(M_1)=0.8426-0.8426=0, \qquad \Delta(M_2)=0.8431-0.8426=0.0005, \qquad \Delta(M_3)=0.8460-0.8426=0.0034.

Apply the tolerance. The threshold is \(0.0010\). Since \(0\leq0.0010\), \(M_1\) is included. Since \(0.0005\leq0.0010\), \(M_2\) is included. Since \(0.0034>0.0010\), \(M_3\) is excluded. The near-tied set is therefore \(\{M_1,M_2\}\).

Apply the tie-breaker. \(M_1\) has 4 predictors and \(M_2\) has 2. The rule selects \(M_2\), the smaller candidate in the near-tied set. Its score is \(0.8431\), which is \(0.0005\) above the exact minimum of \(0.8426\).

Check and common mistake avoided. The selected score is within \(0.0010\) of the minimum, as required, and the selected size is 2. Calling \(M_2\) the exact score minimum would be incorrect; it is selected because the declared tolerance and secondary rule permit it.

Tolerance comparisons should be made against the best score, not by informally comparing candidates in a chain. For example, one candidate might be close to a second candidate, and that second candidate close to a third, while the third is not close to the minimum. Defining the near-tied set by each candidate’s difference from \(Q_{\min}\) makes the decision reproducible.

Precision, Rounding, and Reporting

A displayed score may have fewer digits than the value used in the selection. Two displayed values that look identical can differ at full precision; conversely, two full-precision scores that differ may round to the same displayed value. Unless the procedure explicitly defines selection on rounded scores, compare the unrounded values available to the selection procedure, then round only for presentation.

Worked Example: A Tie Created by Display Rounding

Two candidate models are scored, with smaller preferred. Model \(M_P\) has 3 predictors and score \(7.36444\); model \(M_Q\) has 2 predictors and score \(7.36449\). The report displays scores to three decimal places, and the rule says to choose the exact minimum, using fewer predictors only for an exact tie. Which model is selected?

Compare full-precision scores. The difference is

$$ 7.36449-7.36444=0.00005>0. $$

Thus \(M_P\), not \(M_Q\), has the exact minimum. The exact-tie rule is not activated, even though \(M_Q\) has fewer predictors.

Round for display. To three decimal places, \(7.36444\) rounds to \(7.364\), and \(7.36449\) also rounds to \(7.364\). The displayed values appear tied, but the full-precision values are not. The selected model is \(M_P\), with 3 predictors and score \(7.36444\); the displayed score is \(7.364\).

Check and common mistake avoided. \(M_P\) has the lower full-precision score. Selecting \(M_Q\) by treating the rounded display as an exact tie would apply the secondary rule to a tie that does not exist under the stated procedure.

A clear report names the selected subset, its size, its score at an appropriate precision, and the decision rule. For a near tie, also report the tolerance and the difference from the minimum. These details allow another analyst to reproduce the decision rather than guess how “close” was interpreted.

Reporting checklist: State whether smaller or larger scores are preferred; identify the exact minimum; give the exact-tie or near-tie rule, including any tolerance; name any further rule needed to resolve a remaining tie; and report the selected subset and its size. Do not describe a near-tied model as the exact best-scoring model.

Common Exam Traps

Using an unstated tie-breaker. A rule such as “choose the smaller model” should not be assumed. Apply it only when given or explicitly established as the decision convention.
Confusing a tie in the displayed values with an exact tie. Check the available full-precision scores before using an exact-tie rule.
Calling a near tie an exact tie. A nonzero score difference remains nonzero even when it falls within a declared tolerance.
Leaving the model size ambiguous. If a tie-breaker does not separate candidates of the same size, report that another rule is needed rather than claiming a unique selection.
Forgetting the score direction. A minimum is best only when smaller values are preferred.

The practical lesson is that a score comparison and a selection decision are not always the same thing. The score identifies the best value; a declared rule resolves ambiguity when multiple candidates attain that value or fall within an accepted tolerance. When that distinction is kept explicit, the chosen subset and its reported size are defensible and reproducible. This treatment is aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Check Your Understanding

For each problem, follow the stated score direction and use only the tie-breaking rule given.

Smaller scores are preferred. \(M_A=\{x_1\}\) has score \(5.20\), \(M_B=\{x_2\}\) has score \(5.20\), and \(M_C=\{x_1,x_2\}\) has score \(5.18\). Which model is selected if the rule is to choose the exact minimum? (Answer: \(M_C\), with 2 predictors and score \(5.18\).)
Smaller scores are preferred. \(M_D\) has 3 predictors and score \(10.04\), while \(M_E\) has 1 predictor and score \(10.07\). The rule selects the exact minimum and uses model size only to resolve an exact tie. Which model is selected? (Answer: \(M_D\), with 3 predictors and score \(10.04\); there is no exact tie.)
Smaller scores are preferred. The minimum score is \(0.5010\), and a candidate with 4 predictors has score \(0.5018\). Under an absolute near-tie tolerance of \(0.0010\), does this candidate qualify as near-tied with the best? (Answer: Yes; its difference is \(0.5018-0.5010=0.0008\leq0.0010\).)
Smaller scores are preferred. Two exact best-scoring models each have score \(22.0\); one has 2 predictors and the other has 3. The stated rule is to select the smallest model among exact best-score ties. Which model size is selected, and does the rule identify a unique model if there are two tied 2-predictor candidates? (Answer: Size 2 is selected; no, a further rule is needed to distinguish the two 2-predictor candidates.)
Smaller scores are preferred. \(M_R\) has score \(3.1186\) and 2 predictors; \(M_S\) has score \(3.1192\) and 1 predictor. The rule treats models within \(0.0010\) of the minimum as near-tied, then chooses the fewest predictors. Which model is selected, and what is its difference from the minimum? (Answer: \(M_S\), with 1 predictor; its score is \(3.1192-3.1186=0.0006\) above the minimum.)

What You'll Learn