Frame the Investigation Before Choosing a Model
A fraud-screening model is useful only if it helps investigators decide which cases to review, and when. A vague request such as “detect fraud” does not yet say what one prediction describes, what counts as the event, or how the result will enter an investigation process. Those choices shape the data and the practical meaning of a model score.
For a well-framed task, specify four connected elements: the unit being screened, the fraud event of interest, the time at which screening happens, and the way investigators will use the resulting priorities. The aim is to create a prediction task that supports a real workflow, not to treat a model score as a finding of wrongdoing.
Four Decisions That Define the Task
Start by naming the unit of prediction: the transaction, claim, policy, account, or other case to which one score will apply. The unit should make sense in the investigation workflow. If an investigator opens and reviews one claim file at a time, a claim-level queue may be more actionable than a queue of claim line items. If each card transaction can be reviewed independently, a transaction-level queue may fit better.
Next, state the event being predicted in operational terms. For example, a hypothetical insurer might be interested in whether a claim is ultimately substantiated as involving intentional misrepresentation. That is different from predicting whether a claim looks unusual, is referred for review, or generates a high payment. Those events can be useful for other purposes, but they do not mean the same thing as fraud.
Then specify when screening occurs. A score used to route a claim before payment must be based on information available at that point. A score assigned after an investigation has concluded would not serve that same early-screening task. State the decision point plainly, such as “after the initial claim submission and before payment authorization.”
Finally, explain how the score guides review. Investigators may have capacity to examine only a limited number of cases in a period. A practical use is therefore often to rank eligible cases and direct available review capacity to the highest-ranked cases. This is prioritization, not a rule that every unreviewed case is legitimate or that every reviewed case is fraudulent.
State exactly what receives one prediction, such as one submitted claim or one card transaction.
Describe the fraud outcome the organization wants to identify, rather than substituting a related but different event such as a referral or an unusual amount.
Identify when the score is produced in the workflow and what decision it is intended to inform.
Say whether cases are ranked, how often the queue is created, and how investigation capacity affects the number selected.
Worked Example: Prioritizing Submitted Claims Before Payment
A hypothetical property insurer receives 240 new claims in a week. The investigation team can open 18 new claim files during that week. The intended screening point is after initial submission but before payment authorization. Frame the task and calculate the share of weekly claims that can be prioritized for review.
Step 1: Unit. One prediction is assigned to each submitted claim, because the team opens and investigates claim files. The 240 claims therefore correspond to 240 candidate cases.
Step 2: Event. The target event is whether a claim is later substantiated, through the insurer’s investigation process, as involving intentional fraud. A claim being referred for investigation is not itself the target event; it is part of the review process.
Step 3: Timing. Produce the score after the initial claim submission and before payment authorization. The intended task is to prioritize review at this point, not to make a final coverage or fraud determination.
Step 4: Use and capacity calculation. Rank the 240 claims by score and send the top 18 to the team, subject to the stated weekly capacity. The review share is
Thus, 7.5% of the weekly claim volume can be selected under this simple capacity plan. The remaining claims are not thereby declared fraud-free; they are not selected for this round of review.
Method and mistake avoided. The method is to match one queue entry to one investigable claim file, then compare capacity with the number of eligible claims. A common mistake is to describe the task as “identify all fraudulent claims.” That ignores the limited review capacity and incorrectly turns a prioritization tool into a guarantee of detection.
Why Capacity Implies Ranking
When investigators have fewer review slots than eligible cases, a prioritized queue is one direct way to use the score. Suppose there are \(N\) cases and capacity to review \(K\), where \(0 \leq K \leq N\). If cases are ordered from highest to lowest score, selecting the first \(K\) assigns exactly the available number of slots. The fraction selected is \(K/N\), provided \(N>0\).
This follows from counting: each selected case uses one slot, so selecting \(K\) cases uses \(K\) slots; dividing that number by the \(N\) eligible cases gives the selected share. This calculation describes workload, not the fraction of cases that are fraudulent and not the model’s accuracy.
The ranking rule does not require a fixed score cutoff. If case volumes or investigator capacity change from one period to another, the number selected can change while the capacity-based rule remains clear. The score is an input to case ordering; the investigation remains the process for assessing the underlying evidence.
Worked Example: Screening Card Transactions Each Day
A hypothetical payment service processes 3,600 card transactions on a particular day. Its fraud team can review 27 transactions that day. Screening is intended to occur before a transaction is released for settlement. The team wants to prioritize transactions later found to involve unauthorized use of a payment credential.
Step 1: Unit and event. One prediction corresponds to one transaction. The event is whether that transaction is later established to involve unauthorized use. A transaction that triggers a routine security check is not automatically a confirmed instance of that event.
Step 2: Timing and action. Score each transaction before release for settlement, using information available at that point. Rank the 3,600 transactions and route the top 27 to the available review process. The task is to prioritize reviews, not to cancel every high-scoring transaction automatically.
Step 3: Calculate the review share. With \(K=27\) and \(N=3{,}600\),
The team can review 0.75% of that day’s transactions under this plan. This is a workload calculation only; it does not tell us how many of the 27 selected transactions will be fraudulent.
Step 4: State the task in one sentence. “Before settlement, assign each card transaction a score for later-confirmed unauthorized use and rank transactions so the team can review up to 27 per day.” This sentence identifies the unit, event, timing, and operational use.
Method and mistake avoided. The method is transaction-level scoring followed by a daily capacity-based ranking. A common mistake is to say the team reviews 27 “accounts” when the queue actually contains transactions. That wording changes what one review slot represents and makes the task description inconsistent with the workflow.
Use the Score as a Review Input
A fraud score may help allocate scarce attention, but it should not be described as a final investigation outcome. A high score means the case is prioritized under the chosen system; it does not prove fraud. A low score means the case is lower in the ranking; it does not prove the absence of fraud.
This distinction matters because the investigation can produce consequences beyond routing a file: a payment may be delayed, a claim may be denied, or a customer may face additional scrutiny. Unless the task explicitly calls for an automated action and the organization has justified that use, keep the prediction task separate from the final decision. In a review workflow, investigators assess the case using the relevant evidence and procedures.
The wording of the task should also make clear whether review is immediate or scheduled, and whether capacity is stated per day, week, or another period. “Review the top cases” is incomplete if it does not identify the relevant queue and the operational limit. State the unit of capacity consistently: for example, 18 claim files per week or 27 transactions per day.
Worked Example: Choosing Between Claim-Level and Line-Level Review
A hypothetical health insurer receives 84 claims in a weekly batch. Each claim contains an average of 5 service lines, for 420 service lines in total. Investigators can open 12 claim files during the week. The proposed workflow is to examine the claim file as a whole. Frame a consistent screening task and identify a tempting but incorrect capacity calculation.
Step 1: Match the unit to the review. Since one investigation opens one claim file, the unit should be one claim, not one service line. The queue contains 84 claims.
Step 2: State the event and timing. The event is whether a claim is later substantiated as involving intentional fraud. The score is produced after receipt of the claim and before the weekly investigation queue is assigned.
Step 3: Apply capacity at the claim level. The 12 available file reviews cover
Therefore, the team can review approximately 14.3% of the claims. To express the percentage to one decimal place, \(0.142857 \times 100=14.2857\%\), which rounds to 14.3%.
Common mistake and correction. A tempting calculation is \(12/420\approx 2.9\%\), treating the 12 file reviews as if they were 12 service-line reviews. That denominator does not match the unit of review. The correct calculation is \(12/84\), because each available slot opens one claim file. If the intended task instead ranked and reviewed individual lines, the unit, action, and capacity would all need to be stated at that level.
Check. The calculation gives a claim-level review share, not a service-line-level share or a fraud prevalence estimate. Its interpretation is consistent with the specified workflow.
Common Framing Mistakes
- Using a vague event. “Suspicious” or “high risk” is not a precise outcome. State the particular fraud event the task is meant to predict.
- Confusing review with fraud. Referral, investigation, and substantiation are different events. A referral is an action taken by the organization, not automatically proof of fraud.
- Leaving out the screening point. “Score claims” does not explain whether the score is used before payment, after payment, or at another stage.
- Ignoring available review capacity. If the task is to support a queue, say how many cases can be reviewed and over what period.
- Calling a score a verdict. A risk ranking can guide attention. It does not establish wrongdoing or replace the investigation.
A concise task statement can be written as: “At [screening time], score each [unit] for [specified fraud event] and rank eligible cases so investigators can review [capacity] per [period].” Filling in each bracket forces the task to connect the prediction with the operational decision it is intended to support. This framing is aligned with the SOA Exam PA (Predictive Analytics) syllabus.
Check Your Understanding
For each scenario, identify the prediction unit, event, screening point, and proposed queue use. Check any capacity arithmetic as well.
- A hypothetical lender receives 150 loan applications each week and can investigate 10. Scores are produced after application submission but before approval. The target event is later-substantiated identity fraud. What is the unit, event, timing, queue rule, and review share? (Answer: One application; later-substantiated identity fraud; after submission and before approval; rank applications and review up to 10 weekly; \(10/150=6.7\%\) to one decimal place.)
- A hypothetical delivery platform processes 2,400 orders per day. Its team can review 36 orders before dispatch, prioritizing orders later confirmed to involve payment fraud. State the unit and calculate the daily review share. (Answer: One order; rank orders before dispatch and review up to 36 daily; \(36/2{,}400=1.5\%\).)
- A hypothetical insurer receives 96 claims in a monthly batch. It can investigate 8 claim files, and each file has multiple invoices. The score is assigned after receipt and before payment. What unit should the task use if each investigation opens one claim file, and what percentage of claims can be selected? (Answer: One claim file; \(8/96=8.333\ldots\%\), or 8.3% to one decimal place.)
- A hypothetical bank can review 20 of 500 wire transfers each day. A manager proposes ranking accounts instead, although investigators open individual transfer records. What unit best matches the stated action, and what is the daily review share if the 500 transfers form the eligible queue? (Answer: One wire transfer; \(20/500=4\%\). Ranking accounts would not match the stated transfer-level review action.)
- A hypothetical insurer scores 180 newly submitted claims for later-substantiated intentional misrepresentation. Screening occurs before payment authorization, and 15 claims can be reviewed each week. Write a concise task statement and calculate the selected share. (Answer: “Before payment authorization, score each newly submitted claim for later-substantiated intentional misrepresentation and rank claims so investigators can review up to 15 weekly.” The selected share is \(15/180=8.3\%\) to one decimal place.)