Interpreting Class Fractions as Local Probabilities

From Neighbor Votes to Local Probabilities

In K-nearest-neighbor classification, the labels of the $k$ observations nearest to a query point determine its prediction. A majority vote gives a hard class prediction, but the neighbor labels also contain more information: their proportions describe how the classes are represented around the query point. These proportions can be interpreted as local estimates of class probabilities.

For example, if 7 of 10 nearby observations belong to class A, the neighborhood provides an estimated local probability of 0.7 for class A. This does not mean that the query point has a known, exact 70% chance of class A. It means that class A accounts for 70% of the selected neighbors, which is used as an estimate of how likely class A is among observations with similar predictor values.

Definition: Let $N_k(x_0)$ be the set of the $k$ nearest training observations to a query point $x_0$, using the specified distance measure and predictors. For class $c$, let $n_c(x_0)$ be the number of those neighbors whose observed label is $c$. The KNN local class-probability estimate is $$ \hat{P}(Y=c\mid X\text{ near }x_0)=\frac{n_c(x_0)}{k}. $$ Here, “near” refers to the neighborhood selected by the KNN rule; the expression is an empirical estimate, not a guarantee that the true conditional probability equals this fraction.

For a classification problem with $C$ possible classes, every neighbor has exactly one label. Therefore, the counts sum to $k$, and the estimated class probabilities sum to one:

\sum_{c=1}^{C} n_c(x_0)=k \qquad\Longrightarrow\qquad \sum_{c=1}^{C}\hat{P}(Y=c\mid X\text{ near }x_0)=1.

The class with the largest estimated proportion is the majority-vote prediction. The estimates retain additional information: a neighborhood with proportions 0.9 and 0.1 gives a different indication of class composition from one with proportions 0.55 and 0.45, even though both favor the same class. A class-probability estimate is thus a description of the local evidence, not just another way to state the winning class.

Why a Neighbor Fraction Estimates a Local Probability

A short probability argument explains the interpretation. Fix a neighborhood containing $k$ observations, and define $I_i=1$ if neighbor $i$ belongs to class $c$, and $I_i=0$ otherwise. The class fraction is the average of these indicators. If, within this neighborhood, each label independently has the same probability $p$ of being class $c$, then each indicator has expected value $p$. By linearity of expectation, the expected average is also $p$.

\hat{p}_c=\frac{1}{k}\sum_{i=1}^{k} I_i, \qquad E[\hat{p}_c]=\frac{1}{k}\sum_{i=1}^{k}E[I_i] =\frac{1}{k}\sum_{i=1}^{k}p=p.

Under those same independent, equal-probability assumptions, the variance of the fraction is

\operatorname{Var}(\hat{p}_c) =\frac{1}{k^2}\sum_{i=1}^{k}\operatorname{Var}(I_i) =\frac{1}{k^2}k p(1-p) =\frac{p(1-p)}{k}.

This idealized calculation helps explain why a fraction based on more observations can be less variable when the observations share approximately the same class probability. It does not say that a larger KNN neighborhood is always preferable: increasing $k$ can include observations from less similar predictor values and change the local composition being estimated.

In a real neighborhood, the probability of class $c$ may vary from one neighbor to another. If neighbor $i$ has class probability $p_i$, the expected fraction, under independent labels, is the average $\frac{1}{k}\sum_{i=1}^{k}p_i$, not necessarily the probability at the query point itself. The local-probability interpretation is most direct when the chosen neighbors are sufficiently similar that their class probabilities are reasonably close to the query point’s probability. Neighbor fractions are estimates from observed labels; they are not automatically calibrated probabilities or certainty statements.

Worked Example: Interpreting a Binary-Class Neighborhood

In a hypothetical classification task, a model assigns each application to class “review” or “routine.” For a query application, the six nearest training applications, in order, have labels review, routine, review, review, routine, review. Calculate both local class-probability estimates and identify what the majority vote indicates.

Method. Count the neighbors in each class, then divide each count by $k=6$.

There are four review neighbors and two routine neighbors, so

\hat{P}(\text{review}\mid X\text{ near }x_0) =\frac{4}{6}=0.6667, \qquad \hat{P}(\text{routine}\mid X\text{ near }x_0) =\frac{2}{6}=0.3333.

The estimates sum to $0.6667+0.3333=1.0000$. The majority-vote class is review, but the neighborhood is not unanimous: one-third of its labels are routine. Thus, “review” is the class favored by these neighbors, while the fractions communicate the balance of local evidence.

Common mistake avoided. Reporting only “review” discards the local class composition. The hard prediction identifies the largest fraction; it does not replace the two probability estimates.

Neighborhood Size and Class Composition

The estimate is determined by both the number of neighbors and their labels. With a fixed query point, increasing $k$ adds observations to the neighborhood. The new fractions depend on the labels of those added neighbors, so an estimate may rise or fall. For example, adding mostly observations from another class can reduce the original class’s fraction even though the original neighbors remain included.

The effect is not simply that larger neighborhoods “make probabilities smaller” or “make estimates more accurate.” A fraction is a count divided by the current $k$. Its value changes according to the new count and the new denominator. A larger neighborhood also covers a broader region of predictor space, so it may represent a less local mixture of class probabilities. The appropriate interpretation is always tied to the specific neighbors included.

For multiple classes, calculate one proportion for each class. Do not calculate only the proportion for the winning class if the question asks for a probability estimate: the remaining proportions show how the other classes share the neighborhood. A zero count gives an estimated proportion of zero for that neighborhood, but it does not prove that the true class probability is zero.

Worked Example: Local Probabilities With Three Classes

A hypothetical equipment-monitoring classifier assigns observations to one of three states: stable, alert, or shutdown. Among the eight nearest training observations to a new reading, three are stable, four are alert, and one is shutdown. Calculate the local probability estimate for each state and state which class has the largest estimated probability.

Method. Divide each class count by the total of eight neighbors.

\hat{P}(\text{stable}\mid X\text{ near }x_0)=\frac{3}{8}=0.3750, \qquad \hat{P}(\text{alert}\mid X\text{ near }x_0)=\frac{4}{8}=0.5000, \qquad \hat{P}(\text{shutdown}\mid X\text{ near }x_0)=\frac{1}{8}=0.1250.

The proportions sum to $0.3750+0.5000+0.1250=1.0000$. Alert has the largest estimated probability, 0.5000, so it is the class favored by the neighborhood. The estimate does not imply that alert is certain; half of the neighbors have one of the other two labels.

Common mistake avoided. Dividing the alert count by the number of other classes would give $4/3$, which is not a probability. Each class count must be divided by the number of neighbors, $k=8$, so that the class fractions share a common denominator and sum to one.

Worked Example: How Added Neighbors Change the Estimate

For a hypothetical binary classification problem, the first five nearest observations have labels A, A, B, A, A. The next six nearest observations have labels B, A, B, B, A, B. Calculate the class-probability estimates using the first five neighbors and then using all eleven. Explain how the estimates change.

Method. First count the labels among the nearest five. There are four A labels and one B label:

\hat{P}(A\mid X\text{ near }x_0,\ k=5)=\frac{4}{5}=0.8000, \qquad \hat{P}(B\mid X\text{ near }x_0,\ k=5)=\frac{1}{5}=0.2000.

The next six observations add two A labels and four B labels. Across all eleven neighbors, the totals are therefore $4+2=6$ for A and $1+4=5$ for B. Thus,

\hat{P}(A\mid X\text{ near }x_0,\ k=11)=\frac{6}{11}=0.5455, \qquad \hat{P}(B\mid X\text{ near }x_0,\ k=11)=\frac{5}{11}=0.4545.

Both estimates sum to one at each neighborhood size. The estimate for A falls from 0.8000 to 0.5455 because the added observations contain more B labels than A labels. A remains the more frequent class among all eleven neighbors, but the local composition is much less concentrated than it was among the nearest five.

Common mistake avoided. The estimate for A with eleven neighbors is not the average of the two neighborhood estimates $(0.8000$ and a fraction from the added six) without accounting for their different sizes. The correct calculation uses the combined count, $6$, divided by the combined total, $11$.

Interpretation and Exam Traps

When reporting a local class-probability estimate, state the class, the fraction, and the neighborhood size. For example, “Among the nine nearest observations, four have class B, so the estimated local probability for B is $4/9=0.4444$.” This makes clear both the calculation and the data on which it is based. Such calculations are aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.

Keep the following distinctions clear:

A neighbor fraction is empirical. It summarizes observed labels in a selected neighborhood; it is not known to equal the exact conditional probability at the query point.
Each class uses the same denominator. Divide each class count by $k$, not by the number of observations in some other class or by the number of classes.
Changing $k$ changes the neighborhood. Recalculate from the complete set of neighbors at the new $k$; do not assume the earlier fraction stays fixed.
A zero observed fraction is not proof of impossibility. It says that no selected neighbor had that label, not that the class can never occur near the query point.
Fractions preserve more information than a class name. A winning class alone does not describe how strongly its count exceeds the other counts.

A frequent calculation error is to keep the old numerator after increasing $k$, or to divide a class count by the number of added neighbors rather than by the total neighborhood size. If $k$ increases from 5 to 11, the new fraction uses the class’s total count among all 11 neighbors divided by 11. The denominator and numerator must refer to the same neighborhood.

Key takeaway: For each class, count its labels among the $k$ nearest observations and divide by $k$. The resulting fractions sum to one and describe the neighborhood’s local class composition. Interpret them as estimates based on those neighbors, not as exact probabilities or guarantees.

Check Your Understanding

For each problem, calculate the requested fraction or fractions from the stated neighborhood. Report probability estimates to four decimal places where needed.

A binary classifier’s five nearest neighbors include one red label and four blue labels. What are the two local class-probability estimates? (Answer: red $0.2000$, blue $0.8000$)
Among eight nearest observations in a three-class problem, the counts are X: 3, Y: 3, and Z: 2. Give all three local class-probability estimates and verify that they sum to one. (Answer: X $0.3750$, Y $0.3750$, Z $0.2500$; sum $1.0000$)
A neighborhood of nine observations contains six positive labels and three negative labels. Calculate both estimates and identify the class with the larger estimate. (Answer: positive $0.6667$, negative $0.3333$; positive has the larger estimate)
The first four nearest labels are C, C, D, A. The next three are D, C, D. Calculate the class estimates at $k=4$ and $k=7$, and describe the change for C. (Answer: at $k=4$, C $0.5000$, D $0.2500$, A $0.2500$; at $k=7$, C $0.4286$, D $0.4286$, A $0.1429$; C’s estimate falls by $0.0714$)
Assume that, conditional on a fixed neighborhood, each of ten neighbors independently has the same probability $p=0.4$ of belonging to class Q. What is the expected Q fraction, and what is the variance of that fraction under this assumption? (Answer: expected fraction $0.4000$; variance $0.0240$)

What You'll Learn