Why Fairness Scope Depends on Context
A model can influence access to credit, insurance, employment, education, health services, or public benefits. In each setting, a fairness discussion needs to identify which groups are relevant to the decision and why. It is not enough to list familiar demographic categories or assume that the same list applies to every model.
An attribute may be expressly protected by law in one jurisdiction or decision context, treated differently in another, or not named in a relevant statute at all. It may still matter ethically if it is connected to unequal access, a foreseeable harm, or a group’s ability to participate in the process. Conversely, an attribute’s importance in one decision does not establish that it has the same legal status or ethical significance in another.
The goal is to define a defensible scope: specify the decision, the affected population, the jurisdiction and time period, and the reasons for considering particular attributes. This is a disciplined analytical starting point, not a substitute for legal advice.
Define the Decision Context Before Naming Groups
A useful scope statement begins with the actual decision, not with a generic model description. “The model predicts risk” is incomplete. A prediction used to invite an applicant to provide more information is different from the same prediction used to deny an application. The decision determines who is affected, what is at stake, and which legal and ethical questions need review.
For a structured analysis, represent the context as \(C=(D,P,J,T,U)\), where \(D\) is the decision, \(P\) is the affected population, \(J\) is the relevant jurisdiction, \(T\) is the time period, and \(U\) is how the model output is used. This is a checklist for reasoning, not a legal test. A change in any component can change which attributes merit consideration.
Describe the action the model informs, such as setting a review priority, determining eligibility, or recommending an offer. Identify whether a person makes the final decision and what discretion that person has.
Specify the population subject to the decision, including whether the process covers applicants, existing customers, employees, students, or another group.
Record the jurisdiction or jurisdictions, relevant organizational setting, and time period. Do not assume that a rule from one location or sector automatically applies elsewhere.
Separate attributes that may have a legal basis from those that warrant ethical attention because of the decision’s purpose or possible consequences. Mark unresolved legal questions for qualified review.
State what is outside the current scope and why. Reassess if the model’s use, population, jurisdiction, or consequences change.
The distinction between legal and ethical relevance prevents two opposite errors. Treating a legal list as the complete ethical scope can miss material risks. Treating every characteristic of interest as legally protected can make an unsupported legal claim. A careful analysis labels the basis for including each attribute rather than collapsing those questions into one.
A Simple Set-Based Justification
Let \(L_C\) be the attributes that qualified review determines are legally relevant in context \(C\), and let \(E_C\) be attributes with an ethical rationale for consideration in that same context. A working set of attributes for a fairness discussion can be written as:
The union is justified because the analysis has two distinct responsibilities: it should not omit attributes that require legal attention, and it should not ignore ethically material groups solely because their legal status is uncertain or different. An attribute can belong to both sets, either set alone, or neither. This notation does not decide whether a particular law applies; it makes the reasoning visible and avoids presenting ethical relevance as a legal conclusion.
Worked Examples
Worked Example: Eligibility Scores for a Hypothetical Training Grant
A fictional regional agency uses a score to prioritize applications for a limited vocational-training grant. Applications come from two neighboring jurisdictions. The score informs which files receive expedited review; it does not automatically approve or reject an applicant. Build an initial fairness scope without assuming that one jurisdiction’s rules apply to both.
Step 1: State the decision and use. The decision is the order in which grant applications receive expedited review. The model output prioritizes files; it is not described as an automatic eligibility decision. This distinction matters because the immediate consequence is review timing, although delays could still affect access to a limited grant.
Step 2: Identify the population. The affected population is people who submit applications to this agency from either jurisdiction. It is not all residents of the region, since non-applicants are not directly subject to this application-ranking process.
Step 3: Record jurisdiction and time. There are two jurisdictions. The analyst should create separate legal-review questions for each rather than assume that a protected-attribute list is identical across them. The applicable rules should be confirmed for the period in which the process operates.
Step 4: Separate the bases for group consideration. Attributes identified by qualified legal review as relevant to grant access in either jurisdiction belong in \(L_C\) for the appropriate setting. Separately, the agency could document an ethical rationale for considering, for example, disability or language-access needs if application procedures or review delays could create barriers. Those examples are questions to assess, not conclusions that a specific law applies.
Step 5: State the scope and its limits. The initial scope includes the legally relevant attributes identified for each jurisdiction and any additional attributes for which the agency documents a credible connection to access or harm. The scope does not claim that every such attribute is legally protected everywhere. The agency should revisit it if the score later determines eligibility directly rather than only review priority.
Method and trap avoided. The method is context-first mapping, followed by separate legal and ethical rationales. A common mistake is to use one jurisdiction’s legal categories for every applicant and label the resulting list universally applicable. The correction is to record the two jurisdictions separately and verify the relevant rules in each.
Worked Example: A Hypothetical Employer’s Interview Queue
A fictional employer uses a predictive score to decide which applicants for a customer-support role receive an interview invitation first. A human recruiter can override the ordering. The role is advertised in one jurisdiction, but applicants may apply remotely from another. Explain how the scope should be framed.
Step 1: Define the decision. The model affects interview invitations and their timing. The human override does not remove the model’s influence: an applicant placed later may have less opportunity if interview slots fill.
Step 2: Define the population. The relevant population is applicants for this role who are subject to the queueing process. Current employees and people who never apply are not the direct decision population for this particular use.
Step 3: Clarify location and coverage. The advertisement’s location alone may not settle which legal requirements are relevant to remote applicants. The employer needs qualified review of the locations and organizational setting involved, rather than inferring jurisdiction solely from where the model is hosted.
Step 4: Distinguish legal and ethical questions. The legal review determines which attributes fall within applicable employment requirements. The ethical review asks whether characteristics such as age, disability, or other circumstances could be relevant to fair access to interviews in this process. Mentioning an attribute for ethical consideration does not establish its legal status or settle how it should be measured.
Step 5: Document use and change conditions. The scope statement should say that the current use orders interview invitations with recruiter override. If the employer removes that override, changes the role, or expands recruitment to additional locations, the context changes and the scope should be reviewed again.
Method and trap avoided. The method is to trace the model’s actual influence on a defined employment decision, then assess legal and ethical relevance in the locations involved. The trap is to conclude that human review makes the model irrelevant to fairness. The correction is to recognize that prioritization can affect opportunity even when a person retains discretion.
Worked Example: A Hypothetical Insurer’s Service-Triage Model
A fictional insurer uses a model to prioritize requests for help with policy administration. The score is not intended to set premiums, determine coverage, or deny claims. Customers in several jurisdictions use the service. Identify a sound initial scope and explain what should not be inferred.
Step 1: Name the decision precisely. The decision is the order in which service requests receive attention. Because the model does not set price or determine coverage in this stated use, the scope should not describe it as an underwriting or claims-decision model.
Step 2: Identify the affected people. The directly affected population is customers whose service requests enter the triage process. The potential consequence is a difference in waiting time or ease of resolving an administrative issue.
Step 3: Specify jurisdictions. Customers may be subject to different rules depending on the jurisdiction and product context. The insurer should obtain qualified review for the applicable settings rather than treating “insurance” as one universal legal category.
Step 4: Build the two-part attribute scope. Attributes identified by legal review for the relevant service decision form the legal component \(L_C\). The ethical component \(E_C\) may include groups for which service access or delay could create a meaningful barrier, provided the rationale is recorded. The analyst should not assume that an attribute is irrelevant merely because it is not an input to the score.
Step 5: Avoid overclaiming. The initial analysis can say that the model affects service prioritization and that certain groups warrant review. It cannot, from this description alone, establish that the model produces a particular disparate outcome, that a law has been violated, or that the same scope applies if the model is later used for coverage or pricing.
Method and trap avoided. The method is to anchor scope to the stated purpose and consequences, then document legal and ethical bases separately. The trap is to carry the service scope unchanged into a later, higher-stakes use. The correction is to reassess \(C\) whenever the decision or use changes.
Common Scope Errors
A useful scope statement is specific enough to guide review but cautious enough not to make claims the evidence or legal analysis does not support. Watch for the following errors:
- Assuming a universal legal list. The legal status of an attribute can depend on jurisdiction, decision type, and applicable rules. Name the relevant setting and seek qualified review.
- Equating “not legally protected” with “not important.” An attribute may still be ethically relevant because of the decision’s purpose or possible consequences.
- Equating “ethically relevant” with “legally protected.” Ethical concern is a reason for careful analysis, not proof of a legal classification or violation.
- Describing the prediction instead of the decision. “Predicts service need” does not explain whether the score prioritizes assistance, changes eligibility, or affects a different outcome.
- Freezing scope after a use change. A model moved to a new population, jurisdiction, or decision needs a fresh context assessment.
Check Your Understanding
For each scenario, identify the decision context and give a careful scope conclusion. Treat the situations as hypothetical; do not assume a legal result that has not been established.
- A city uses a model to order requests for home-repair inspections. State two parts of the context that should be specified before naming relevant groups, and explain why. (Answer: Specify the inspection-priority decision and the affected requesters; also identify the jurisdiction and how the ranking affects service, because these facts shape legal review and ethical relevance.)
- A company’s model affects interview invitations in two jurisdictions. Why is it insufficient to use one legal attribute list without review? (Answer: Applicable requirements may differ by jurisdiction and employment context, so each setting needs qualified review; the shared ethical rationale can be documented separately.)
- A group characteristic is not identified as legally protected in the analyst’s initial review, but the decision may create a significant access barrier for that group. Should it be excluded from the fairness discussion? (Answer: No. It can be included as ethically relevant with the rationale recorded, while its legal status remains a separate question for qualified review.)
- A model initially prioritizes customer-service requests, then is proposed for deciding claim eligibility. Which part of the scope analysis must be revisited? (Answer: Reassess the decision, consequences, affected population, applicable jurisdiction, and both legal and ethical attribute sets; the original service scope does not automatically carry over.)
- An analyst writes \(G_C=L_C\) and concludes that no other group characteristics need consideration. What is the conceptual error, and what expression better represents the proposed scope? (Answer: It assumes legal relevance exhausts ethical relevance. Use \(G_C=L_C\cup E_C\), while keeping legal and ethical rationales distinct.)