Why Compare Positive-Decision Rates?
A predictive model can inform decisions such as whether to invite an applicant to an interview, refer a claim for additional review, or offer a service. If the decision is binary, each person receives either a positive decision or a non-positive decision. A fairness audit may ask whether the proportion receiving the positive decision is similar across specified groups.
Statistical parity addresses that specific question. It compares groups’ positive-decision rates without conditioning on whether people later meet some outcome standard. That makes the measure straightforward to calculate, but also limited: a comparison of decision rates alone does not show how well decisions match outcomes, how people were measured, or whether the overall process is fair.
The decision and groups must be specified before interpreting the comparison. For example, “positive” could mean “approved,” “selected for review,” or “offered an interview.” It does not automatically mean “beneficial”: additional review could be helpful in one setting and burdensome in another. Likewise, a group comparison is meaningful only when the groups and the population being audited match the question under consideration.
Calculating and Interpreting the Rates
For observed data, calculate each group’s rate as the number of people who received the positive decision divided by the number of people in that group who were in scope for the decision. If group \(g\) contains \(n_g\) people and \(k_g\) received \(D=1\), the observed rate is \(k_g/n_g\). The denominator is the group’s decision-eligible audit population—not the number of positive decisions, nor the number of people with a particular outcome recorded later.
For two groups, a signed difference can make the comparison explicit. If the audit defines the order as group A minus group B, the difference is \(\widehat{p}_A-\widehat{p}_B\). A positive value means the observed rate is higher for A; a negative value means it is lower. The size of the difference is often expressed in percentage points. For instance, a difference of \(0.08\) is an eight-percentage-point difference—not an eight-percent relative change.
Why the Empirical Rate Has This Form
The rate calculation is not just a convenient convention. For each person \(j\) in group \(g\), define an indicator \(I_j\) that is 1 if the person receives the positive decision and 0 otherwise. The group’s observed positive-decision rate is the average of those indicators. Summing the indicators counts the positive decisions, so their average is exactly the positive-decision count divided by the group size.
This gives the observed rate for the people included in the audit. If the audit data are viewed as a sample from a broader population, the rate is also used to estimate that population’s conditional probability \(P(D=1\mid G=g)\). That interpretation relies on the sample representing the target population adequately. A rate calculated from a selected subset should not be assumed to describe people who were excluded from that subset.
Worked Example: Comparing Two Approval Rates
A hypothetical training program records decisions for all 200 applicants who completed its application process. Of 120 applicants in group A, 48 are approved. Of 80 applicants in group B, 24 are approved. Calculate both positive-decision rates and their difference, defined as group A minus group B. State what the comparison establishes.
Method. Use each group’s own number of applicants as its denominator, because the question concerns the share approved within each group.
For group A, the rate is \(48/120=0.40\), or 40%. For group B, it is \(24/80=0.30\), or 30%. The signed difference is \(0.40-0.30=0.10\), which is 10 percentage points.
As a check, the approval counts sum to \(48+24=72\), and the group sizes sum to \(120+80=200\). The overall approval rate is \(72/200=0.36\), or 36%. That pooled rate is a summary of the two groups together; it is not either group’s rate and cannot replace the group-by-group comparison.
The observed rates are not equal, so these data do not show statistical parity between A and B for this approval decision and audit population. They do not establish why the rates differ or whether the process is fair. The overall rate of 36% does not resolve either question.
Common mistake avoided: Dividing the 48 approvals in A by all 72 approvals would give \(48/72=0.6667\). That is A’s share of all approvals, not A’s positive-decision rate. The requested rate is \(48/120=0.40\), using all in-scope applicants in A as the denominator.
The Decision Rule Is Part of the Comparison
A model may produce a score, while an organization turns that score into a binary decision using a threshold or another rule. Statistical parity is calculated on the resulting decisions, not on the scores by themselves. Therefore, the same scores can produce different group rates under different decision rules. An audit should document which rule was applied, when it was applied, and to whom.
The rule also defines the meaning of a positive decision. A high approval rate and a high rate of referral for extra scrutiny are not interchangeable outcomes. Before comparing rates, state what \(D=1\) means in the setting and what population was eligible for that decision.
Worked Example: A Common Threshold Produces Unequal Rates
A hypothetical organization uses a score threshold of 0.70 for a positive decision. In an audit of 100 people in group Cedar, 30 have scores at or above 0.70. In an audit of 100 people in group Maple, 18 have scores at or above 0.70. Calculate the rates and signed difference, and explain what follows if Maple’s threshold is lowered to 0.55 and 30 people then qualify.
Method. Count people who meet the stated rule, then divide by the full in-scope group size. At the common threshold, Cedar’s rate is \(30/100=0.30\), or 30%, and Maple’s rate is \(18/100=0.18\), or 18%. The Cedar-minus-Maple difference is \(0.30-0.18=0.12\), or 12 percentage points.
If Maple’s threshold changes to 0.55 and 30 of its 100 people then receive the positive decision, Maple’s rate becomes \(30/100=0.30\). Cedar remains at 30%, so the observed difference becomes \(0.30-0.30=0\). The rates are equal under these specified rules.
This calculation shows that changing a decision rule can change the parity result. It does not establish that either threshold is justified, that the groups’ scores are comparable, or that the resulting decisions are fair. The rule change and its rationale would need to be examined in context.
Common mistake avoided: Reporting the equal 30% rates without disclosing that different thresholds were used would omit a material feature of the decision process. Equal rates answer the statistical-parity question for the observed rules; they do not make those rules identical.
What Statistical Parity Does Not Establish
Equal positive-decision rates are one particular fairness criterion, not a complete evaluation. A parity result does not tell an auditor whether the decisions are accurate, whether errors are distributed evenly, or whether people who receive the same decision have similar prospects. Those questions require different evidence and analyses. Nor does a rate difference, by itself, diagnose discrimination or identify which part of a process produced the difference.
The earlier material on historical bias, measurement bias, selection bias, and proxy variables matters when interpreting an audit. For example, equal observed rates cannot repair an outcome label that systematically records some groups’ outcomes less accurately. Conversely, unequal observed rates may reflect several features of the data and decision process; the rate comparison alone cannot separate them.
A responsible report should therefore state the decision, positive-decision definition, groups, audit population, period, denominators, and observed rates. It should describe the comparison as evidence about those specified decisions and people—not as a universal verdict about an entire model or organization. If the eligible population or decision process changed, the comparison may no longer answer the same question.
Worked Example: Checking the Audit Denominator
A hypothetical service desk considers 50 people in group North and 40 in group South for priority assistance. It grants priority to 20 North applicants and 12 South applicants. A separate follow-up survey records outcomes for only 30 North applicants and 32 South applicants. Calculate the statistical-parity rates for the priority decision. Explain why the survey counts are not the appropriate denominators for this calculation.
Method. Use the number considered for the priority decision in each group. North’s rate is \(20/50=0.40\), or 40%. South’s rate is \(12/40=0.30\), or 30%. The North-minus-South difference is \(0.40-0.30=0.10\), or 10 percentage points.
The survey counts refer to people with recorded follow-up outcomes, not people considered for the priority decision. Using those counts would produce \(20/30\) and \(12/32\), but those fractions combine a decision count with a different population definition. They are not the positive-decision rates among those considered. The audit should keep the decision denominator and the follow-up-outcome denominator distinct.
The calculation shows unequal observed priority rates in the specified decision population. It does not reveal the follow-up outcomes of the people who did not answer the survey, nor does it establish why the priority rates differ.
Common Exam Traps
- Using the wrong denominator. Divide positive decisions by everyone in that group who was in scope for the decision, not by all positive decisions across groups or by people with observed follow-up outcomes.
- Confusing counts and rates. Groups of different sizes can have different positive-decision counts but equal rates. Conversely, similar counts do not imply similar rates.
- Calling percentage points “percent.” Subtract rates to get a percentage-point difference. Do not describe a difference of 0.10 automatically as a 10% relative change.
- Omitting the decision definition. “Positive” must identify the action being compared. The same label can represent a benefit, a burden, or a mixture depending on the setting.
- Overstating the conclusion. Equal rates establish parity only for the specified groups, population, time, and decision rule. They do not prove that the process is fair in every relevant sense.
Check Your Understanding
For each question, use the stated audit population and decision definition. Interpret the calculation only as far as the information supports.
- A hypothetical scholarship committee considers 90 applicants in group Lark and 60 in group Finch. It awards scholarships to 27 Lark applicants and 24 Finch applicants. Find each positive-decision rate and the Lark-minus-Finch difference in percentage points. (Answer: Lark 30%, Finch 40%; difference −10 percentage points.)
- A clinic offers a hypothetical optional consultation to 40 people in group Pine and 50 in group Reed. It offers the consultation to 12 Pine and 15 Reed participants. Is there observed statistical parity for the offer? (Answer: Yes; both rates are 30%.)
- A review team examines 75 cases in group Aster and 125 in group Birch. It refers 30 Aster cases and 40 Birch cases for additional review. Calculate the two rates and the Aster-minus-Birch difference. (Answer: Aster 40%, Birch 32%; difference 8 percentage points.)
- A hypothetical hiring screen gives positive decisions to 16 of 80 applicants in group Cobalt and 15 of 60 applicants in group Coral. An analyst says the rates are equal because the positive counts differ by only one. Calculate the rates and correct the interpretation. (Answer: Cobalt 20%, Coral 25%; rates are not equal, with a Cobalt-minus-Coral difference of −5 percentage points.)
- A benefit program considers 120 people in group Elm and 80 in group Fir. It approves 36 Elm applicants and 24 Fir applicants. The recorded follow-up outcomes are available for only 90 Elm and 70 Fir applicants. Calculate the approval rates using the correct denominators and state why the follow-up counts should not replace them. (Answer: Both approval rates are 30%, so there is observed statistical parity; the follow-up counts describe a different population and do not define the approval-rate denominators.)