Why Omitting a Protected Attribute Is Not Enough
A model may be designed without a protected attribute such as age or ethnicity among its inputs. That choice can seem to ensure that the attribute plays no role in a prediction. But other inputs may be associated with the protected attribute, and historical data may reflect unequal access, treatment, or recording. A rule that never reads the protected attribute can therefore still produce different prediction rates across groups.
This distinction matters in actuarial work because removing a column is a statement about model inputs, not a complete assessment of model consequences. A fairness review must consider how the remaining variables are distributed, how the target was generated, and what predictions the system actually produces. This tutorial is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
The distinction is between the rule and the distribution of inputs to which the rule is applied. For example, a single rule might refer to a geographic category that is strongly associated with group membership. Or a recorded outcome might depend on who had access to a service that generated the records. In either case, excluding \(A\) does not remove all ways that group-related information can affect predictions.
Group Prediction Rates Depend on Input Distributions
Let \(A\) be a group attribute, \(X\) the model inputs, and \(\hat{D}\) a binary predicted decision. For group \(A=a\), the group prediction rate is \(P(\hat{D}=1\mid A=a)\). A model that uses \(X\) alone may still have different group prediction rates because the distribution of \(X\) can differ between groups.
The integral form applies generally, including when \(X\) is continuous. If \(X\) is discrete, the same result can be written as a sum:
These expressions follow from conditioning on the inputs. For a discrete \(X\), the events \(X=x\) divide the population in group \(a\) into mutually exclusive cases. The probability of a positive prediction is the sum of the probability of a positive prediction in each case, multiplied by the probability of that case. For continuous \(X\), the corresponding aggregation is an integral over the group’s conditional distribution, not a sum over individual values.
If the model uses \(X\) alone, then \(P(\hat{D}=1\mid X,A=a)=P(\hat{D}=1\mid X)\). The prediction behavior at a fixed value of \(X\) is shared across groups. However, the distribution \(F_{X\mid A=a}\) may differ across groups, so integrating the same prediction behavior against different input distributions can give different group rates.
Worked Example: A Shared Rule With a Discrete Proxy
A hypothetical insurer uses a supplemental-review rule based only on a two-category regional indicator \(X\). The model sends an application for review when \(X=H\), and does not send it when \(X=L\). In a validation sample, group \(A=0\) has 30 applications with \(X=H\) and 70 with \(X=L\); group \(A=1\) has 70 with \(X=H\) and 30 with \(X=L\). Calculate the review-prediction rate in each group, the rate difference, and the ratio of the lower rate to the higher rate.
Method. Because \(X\) is discrete, use the law of total probability. The rule is deterministic: \(P(\hat{D}=1\mid X=H,A=a)=1\) and \(P(\hat{D}=1\mid X=L,A=a)=0\) for either group.
For group \(A=0\), the conditional input probabilities are \(P(X=H\mid A=0)=30/100=0.30\) and \(P(X=L\mid A=0)=70/100=0.70\). Therefore,
For group \(A=1\), the conditional input probabilities are \(P(X=H\mid A=1)=70/100=0.70\) and \(P(X=L\mid A=1)=30/100=0.30\). Thus,
The rate difference, group \(A=1\) minus group \(A=0\), is \(0.70-0.30=0.40\), or 40 percentage points. The ratio of the lower rate to the higher rate is \(0.30/0.70=3/7\approx0.4286\), to four decimal places. The rule never uses \(A\), but the groups have different distributions of \(X\), so their prediction rates differ.
Check. The two groups each contain 100 applications, and the rule flags exactly the applications in category \(H\). It therefore flags 30 in group \(A=0\) and 70 in group \(A=1\), matching the calculated rates. The common mistake is to infer equal group rates from the shared rule; the rates depend on how often each group has the input value that triggers the rule.
The Same Logic Applies to Continuous Inputs
The issue is not limited to categories. For a continuous input, the group prediction rate averages the model’s prediction probability over a continuous conditional distribution. Even if the model applies exactly the same threshold to every person, groups with different input distributions can fall on opposite sides of that threshold at different rates.
Worked Example: A Threshold Applied to a Continuous Input
A hypothetical outreach model uses one continuous score \(X\) and predicts outreach when \(X>0.6\). Suppose \(X\mid A=0\) has a uniform distribution on \([0,1]\), while \(X\mid A=1\) has a uniform distribution on \([0.2,1.2]\). Find each group’s prediction rate, their difference, and the lower-to-higher rate ratio.
Method. For a uniform distribution on an interval of length 1, the probability that \(X\) exceeds 0.6 is the length of the part of that interval above 0.6, divided by 1. Equivalently, integrate the indicator of \(x>0.6\) against the group’s uniform density.
For group \(A=0\), the interval above the threshold is \((0.6,1]\), of length \(1-0.6=0.4\). Its conditional density is 1 on \([0,1]\), so
For group \(A=1\), the interval above the threshold is \((0.6,1.2]\), of length \(1.2-0.6=0.6\). Its conditional density is also 1 on its length-1 support, giving
The prediction-rate difference is \(0.6-0.4=0.2\), or 20 percentage points. The lower-to-higher rate ratio is \(0.4/0.6=2/3\approx0.6667\), to four decimal places.
Check. Both supports have length 1, so the calculated probabilities are also the proportions of each support above the cutoff. The rule uses no group attribute and has the same cutoff for everyone, but the supports are shifted relative to that cutoff. The rates are therefore different. Summing probabilities over individual continuous values would not be appropriate; the integral accounts for the full conditional distribution.
Historical Data Can Carry the Same Problem
A model can also inherit group-related differences through the outcome it is trained to predict. The recorded target may reflect access to assessment, reporting, or service as well as the underlying event of interest. A model can then learn patterns in the recorded target without using \(A\), while those patterns reproduce unequal historical conditions.
This is a distinction between predicting an observed label and answering a different question about an underlying event. A model’s accuracy against its label does not establish that the label fully or equally measures that underlying event. The example below separates those quantities explicitly.
Worked Example: Unequal Recording and a Group-Related Feature
A hypothetical program wants to predict a documented service need. Let \(T=1\) indicate that a person has the underlying need, and let \(Z=1\) indicate access to a process that records the need. In each group of 1,000 people, the underlying-need prevalence is 40%, and need is independent of \(Z\). The access rates are \(P(Z=1\mid A=0)=0.90\) and \(P(Z=1\mid A=1)=0.50\). Suppose the recorded target \(Y\) equals 1 only when both \(T=1\) and \(Z=1\). A model uses \(Z\) alone, assigns score 0.40 when \(Z=1\) and 0 when \(Z=0\), and predicts a positive result at a score threshold of 0.30. Calculate each group’s prediction rate and recorded-target prevalence, then explain what the prediction rate says about underlying need.
Method. First use the stated independence and recording rule to find the recorded-target rate. Then apply the model’s threshold: a score of 0.40 exceeds 0.30, while a score of 0 does not. Thus the model predicts positive exactly when \(Z=1\).
In group \(A=0\), 90% have \(Z=1\), so the prediction rate is \(0.90\). In group \(A=1\), 50% have \(Z=1\), so the prediction rate is \(0.50\). The model uses \(Z\), not \(A\), but the prediction rates differ by \(0.90-0.50=0.40\), or 40 percentage points.
For either group, the recorded target is positive when both \(T=1\) and \(Z=1\). Since \(P(T=1)=0.40\) and \(T\) is independent of \(Z\), the group \(A=0\) recorded-target prevalence is
For group \(A=1\), the same calculation gives
In a group of 1,000, these expected rates correspond to 360 and 200 recorded positive cases, respectively. The underlying-need prevalence, however, is \(0.40\) in both groups by assumption. The prediction-rate difference reflects different access to recording, not a difference in underlying-need prevalence. In this constructed example, the score 0.40 for \(Z=1\) is also the conditional probability of the recorded target given \(Z=1\), since \(T\) has prevalence 0.40 and is independent of \(Z\). The threshold turns that score into a prediction for everyone with \(Z=1\).
Check. The model’s predicted-positive rates, 90% and 50%, match the access rates because the rule predicts positive exactly when \(Z=1\). The recorded-label rates, 36% and 20%, are the access rates multiplied by the common 40% underlying prevalence. The example does not show that every difference in predictions is unfair; it shows why a model predicting a recorded label cannot, on that basis alone, be assumed to measure underlying need equally.
Common Mistakes and Practical Interpretation
A recurring error is to treat exclusion of \(A\) as if it implied statistical independence between the prediction and \(A\). It does not. The relevant condition for equal positive-prediction rates is \(P(\hat{D}=1\mid A=a)\) being the same across the groups being compared. A shared rule based on \(X\) does not establish that condition when the distributions \(F_{X\mid A=a}\) differ.
For example, using the first worked example, an incorrect calculation would say: “The rule is the same for both groups, so \(P(\hat{D}=1\mid A=0)=P(\hat{D}=1\mid A=1)\).” This would imply rates of 30% in both groups if the first group’s rate were copied across. The correction is to calculate each rate using its own conditional input distribution: \(0.30\) for group \(A=0\) and \(0.70\) for group \(A=1\).
A rate difference is a signal to examine, not a complete conclusion about fairness. Its interpretation depends on the decision’s purpose, the quality and meaning of the target, and the consequences of errors for the people affected. The fairness measures introduced earlier—such as statistical parity and group error rates—can help describe different aspects of performance, but no one measure answers every normative or operational question.
In practice, a defensible assessment distinguishes at least three questions: Was the protected attribute used directly? Are other inputs associated with group membership or shaped by unequal historical processes? And what are the resulting prediction rates and errors for relevant groups? Keeping these questions separate prevents a narrow technical claim—“the model does not use \(A\)”—from being mistaken for evidence that its effects are fair.
Check Your Understanding
For each problem, distinguish the prediction rule from the group-specific distribution or data process that affects its results.
- A review rule flags an application when a binary location indicator equals 1. In group A, 18 of 60 applicants have indicator 1; in group B, 42 of 70 do. Find each group’s prediction rate and the percentage-point difference. (Answer: A: 30%; B: 60%; difference: 30 percentage points, with B higher.)
- A model predicts positive when \(X>0.4\). For group A, \(X\) is uniform on \([0,1]\); for group B, \(X\) is uniform on \([0.1,1.1]\). Find both prediction rates and the lower-to-higher rate ratio. (Answer: A: 0.60; B: 0.70; ratio \(0.60/0.70\approx0.8571\).)
- A model predicts positive exactly when \(Z=1\). The rate of \(Z=1\) is 0.25 in group A and 0.55 in group B. State the group prediction rates and explain whether the shared rule implies equal rates. (Answer: 0.25 and 0.55; no, because the conditional distributions of \(Z\) differ.)
- Underlying event prevalence is 30% in both groups. An event is recorded only when \(Z=1\), where access rates are 80% in group A and 40% in group B; assume the event is independent of access within each group. Find the recorded-event prevalence in each group. (Answer: A: \(0.30\times0.80=0.24\); B: \(0.30\times0.40=0.12\).)
- A continuous model has prediction probability \(g(x)=P(\hat{D}=1\mid X=x,A=a)\), the same function for either group. Write the general expression for the group prediction rate and state what can make the rates differ. (Answer: \(E[g(X)\mid A=a]=\int g(x)\,dF_{X\mid A=a}(x)\); different conditional input distributions can produce different rates.)