Why Larger Groups May Have More Outlier Flags

Why Sample Size Matters

Suppose two groups come from similar distributions, and unusually high or low observations are rare. A group with more observations has more chances to include one of those values. It may therefore have more outlier flags even if the groups are no different in their underlying tendency to produce extreme observations. The word may matters: a larger group does not have to contain more flags in every particular data set.

To compare groups sensibly, distinguish the count of flags from the rate, or proportion, flagged. A count answers how many observations were flagged; a rate answers what fraction of that group was flagged. Neither number, by itself, proves that the groups have different distributions or that a flagged value is erroneous.

Definition: Under the 1.5-IQR rule, calculate \(IQR=Q_3-Q_1\), where \(Q_1\) and \(Q_3\) are the first and third quartiles. The lower fence is \(Q_1-1.5(IQR)\), and the upper fence is \(Q_3+1.5(IQR)\). An observation below the lower fence or above the upper fence is flagged as a potential outlier. A flag is a descriptive signal, not a verdict about the observation.

When applying this rule separately to groups, the usual procedure is to calculate each group’s quartiles and fences from that group’s data. This is useful for describing each group’s values relative to its own middle half. But it means that a flag in one group is not necessarily an observation beyond the same numerical cutoff used for another group. Keep that distinction in mind when interpreting comparisons.

A Fixed-Cutoff Model for the Effect of Group Size

Start with a simplified model. Imagine that all observations are drawn independently from the same population, and that a fixed cutoff has been chosen in advance. Let \(q\) be the probability that one observation falls beyond that cutoff. For example, a reference distribution might imply that 2% of observations fall outside a specified pair of reference limits. In this model, each observation has the same chance \(q\) of being flagged.

Let \(K\) be the number of flagged observations among \(n\) observations. Since each observation either is flagged or is not, \(K\) follows a binomial model under the assumptions just stated. Its expected count is \(nq\). The probability of at least one flag is \(1-(1-q)^n\): the chance of no flags is \((1-q)^n\), so subtracting from 1 gives the chance of one or more.

$$ E(K)=nq $$

P(K\geq 1)=1-P(K=0)=1-(1-q)^n

These formulas describe a common chance of crossing a fixed cutoff. They do not say that each group must have exactly \(nq\) flags; that is an average over repeated samples. Nor do they automatically give the exact probabilities for flags based on quartiles and fences estimated separately from each sample.

Worked Example: More Opportunities for a Flag

Imagine two hypothetical independent groups drawn from the same population. Use a fixed pair of reference cutoffs that, by assumption, has a 2% chance of flagging any one observation. Group A has 25 observations and Group B has 100. What are the expected counts, and what is the chance that each group has at least one flag?

For Group A, \(n=25\) and \(q=0.02\). Its expected flagged count is

$$ E(K_A)=nq=25(0.02)=0.50 $$

The chance of at least one flag is

P(K_A\geq 1)=1-(1-0.02)^{25}=1-0.98^{25}\approx 1-0.6035=0.3965

So the chance is about 39.7%. For Group B,

$$ E(K_B)=100(0.02)=2.00 $$

P(K_B\geq 1)=1-0.98^{100}\approx 1-0.1326=0.8674

The larger group has a higher expected count and a higher chance of at least one flag, despite having the same per-observation flag probability. These are model-based expectations and probabilities, not predictions that these exact counts will occur. The calculation illustrates the effect of group size when the cutoff is fixed and the observations share the same probability of crossing it.

Why a Flagged Count Is Not a Fair Comparison by Itself

If group sizes differ, the larger group can have more flags while having a smaller fraction flagged. Dividing the number of flags by the group size gives the observed flag rate. This rate makes the denominator visible, but it does not remove sampling variability: a rate from a small group can change substantially when only one observation changes status.

Worked Example: Compare Counts and Rates

In a hypothetical data review, a consistent flagging procedure identifies 2 flagged observations among 30 in Group A and 5 among 120 in Group B. Compare the counts and the observed rates.

The raw count is larger in Group B: 5 flags compared with 2. The rates are

\text{Rate}_A=\frac{2}{30}=0.0667\approx 6.67\%

\text{Rate}_B=\frac{5}{120}=0.0417\approx 4.17\%

Group B has more flags in absolute number, but a smaller fraction of its observations is flagged. The count alone would obscure that difference in the denominators. These observed rates describe these two data sets; they do not establish that the underlying populations have different probabilities of producing flagged observations. In particular, the rates are based on only 30 and 120 observations, so they may vary from sample to sample.

If the two groups were being compared using the fixed-cutoff model with a common flag probability of 5%, the expected counts would be \(30(0.05)=1.50\) and \(120(0.05)=6.00\). Counts of 2 and 5 are both close to those expectations, illustrating how different group sizes can produce different typical counts even under a shared rate. This calculation is an illustration of the model, not evidence that the true flag probability is 5%.

Sample-Based Fences Need Extra Caution

The 1.5-IQR rule usually uses quartiles calculated from the observed group. That differs from the fixed-cutoff model in two important ways. First, the fences can differ across groups because their quartiles and IQRs differ. Second, the fences are estimated from the same observations being checked. A change in the sample can affect both which values are flagged and where the fences lie.

Consequently, it is not generally correct to take a fixed value of \(q\), such as 0.02, and claim that every observation has exactly that independent chance of being flagged by a group’s sample-based Tukey fences. In that setting, the flags are linked through the shared estimated quartiles and fences. The fixed-cutoff formulas are useful for understanding the opportunity effect, but are not an exact probability calculation for every sample-based outlier analysis.

Worked Example: Similar Flag Rates With Different Group Sizes

Consider two hypothetical groups analyzed using the 1.5-IQR rule separately. Group C has 40 observations, \(Q_1=10\), and \(Q_3=14\). Group D has 100 observations, \(Q_1=10\), and \(Q_3=18\). Suppose the data review finds 2 flags in Group C and 4 in Group D. Calculate the fences and compare the counts and rates.

For Group C,

IQR_C=14-10=4,\qquad \text{lower fence}=10-1.5(4)=4,\qquad \text{upper fence}=14+1.5(4)=20

Its rate is \(2/40=0.0500\), or 5.00%. For Group D,

IQR_D=18-10=8,\qquad \text{lower fence}=10-1.5(8)=-2,\qquad \text{upper fence}=18+1.5(8)=30

Its rate is \(4/100=0.0400\), or 4.00%. Group D has twice as many flags, but a slightly smaller flagged fraction. Its fences also differ from Group C’s: for example, a value of 25 is above Group C’s upper fence of 20 but not above Group D’s upper fence of 30. Thus, counts from separately calculated fences reflect group size and group-specific cutoffs, as well as the observations themselves. A careful comparison reports the group sizes, the rule used, and the counts or rates rather than treating the counts as directly interchangeable.

What a Cautious Comparison Can Say

A useful description begins with the denominator and the flagging rule. State how many observations were examined in each group, how many were flagged, and—when meaningful—the percentage flagged. If the fences were calculated separately for each group, say so. This makes clear that the comparison is about flags relative to each group’s own quartiles, not necessarily about crossing a common numerical threshold.

The fixed-cutoff model also gives a concise justification for why larger groups may show more flags. Let \(I_j=1\) if observation \(j\) crosses the fixed cutoff and \(I_j=0\) otherwise. Then \(K=I_1+\cdots+I_n\). Each indicator has expected value \(q\), so by linearity of expectation,

E(K)=E(I_1+\cdots+I_n)=E(I_1)+\cdots+E(I_n)=q+\cdots+q=nq

Independence is not needed for this expected-count calculation if each observation still has probability \(q\); it is needed for the formula \(P(K\geq 1)=1-(1-q)^n\). This distinction is useful: the average count can increase with group size even when calculating the exact chance of at least one flag requires stronger assumptions.

For groups using sample-based Tukey fences, the simple model is a conceptual guide rather than a complete description. The group with more observations may have more opportunities to include extremes, but a larger group is not guaranteed to have more flags or a higher flagged rate. A group’s distribution and its estimated fences matter too.

Common Mistakes

Comparing counts without denominators. “Group B has more flags” does not tell whether a larger fraction of its observations was flagged. Report counts alongside group sizes and rates.
Assuming a larger group must have more flags. A larger sample has more opportunities, not a guaranteed outcome. Random variation can produce fewer flags in the larger sample.
Applying the fixed-cutoff probability formula to sample-based fences without qualification. Sample quartiles and fences are estimated from the same data, so the fixed, independent-flag model need not apply exactly.
Treating every flag as the same kind of extremeness across groups. When each group has its own fences, those fences may be at different numerical values. State how the cutoffs were obtained.
Interpreting a flag as proof of an error or a group difference. A flag identifies a value to notice under a rule. It does not, by itself, explain why the value occurred or establish a population-level difference.

Key takeaway: More observations mean more opportunities for a flag, so larger groups may have larger flagged counts even when observations share the same underlying chance of crossing a fixed cutoff. Compare counts with group sizes and rates, and remember that separately estimated 1.5-IQR fences make the comparison more nuanced.

Check Your Understanding

Use the assumptions in each question carefully.

With a fixed cutoff and a 1% chance of crossing it per observation, what is the expected number of flags in a group of 80?
Under the same fixed-cutoff model, explain why a group of 200 has a greater chance of at least one flag than a group of 20.
Group A has 3 flags among 50 observations, and Group B has 6 among 150. Calculate both flagged rates. Which group has the larger count, and which has the larger rate?
Why is the fixed-cutoff binomial calculation not automatically an exact model for flags from separately estimated 1.5-IQR fences?
Two groups have different sample sizes and each uses its own Tukey fences. Name two pieces of information, beyond the flagged counts, that should be reported to make the comparison clearer.