Using Past Events to Ask Better Forward-Looking Questions
Incident and near-miss reviews provide concrete evidence about how a process can fail, how a control can be bypassed, and what might happen if a similar sequence occurs again. They are useful for identifying possible future risks, but they are not a forecast by themselves. A review is based on events that were detected, recorded, and selected for examination; other pathways may remain unseen, and historical pathways may change after operations or controls change.
The purpose is therefore not simply to count past events. It is to move carefully from a description of what happened to a hypothesis about an uncertain event that could recur, under stated conditions, and to record what evidence supports that hypothesis. This uses the earlier material on cause-event-effect risk statements without turning a review into a full risk assessment.
A near miss is not the same thing as a risk. It is evidence that some part of a possible pathway occurred, while the harmful consequence did not occur in that instance. For example, a payment file may contain a duplicate instruction that an automated control blocks. The duplicate instruction and its attempted processing are observed; a future payment escaping the control is still uncertain.
The review should distinguish what the record establishes from what the reviewer infers. “A second reviewer was unavailable” may be a recorded fact. “Staffing pressure could make the second review unavailable again” is a possible recurrence condition that needs support or further investigation. Treating the inference as proven can overstate the evidence; ignoring it can miss a useful risk hypothesis.
A Structured Review of Incident Evidence
A disciplined review moves from the record to a forward-looking pathway in small steps. The point is to preserve important differences among causes, events, consequences, and current issues. A service disruption that has already occurred is an issue to manage; the uncertain risk question is whether a similar disruption could occur again and through what pathway.
Specify the process, period, event types, and records being examined. A narrow review can still produce useful insights, but its conclusions apply only within the stated boundary.
Record the sequence, controls involved, actual outcome, detection method, and relevant conditions. Separate documented facts from interpretations and missing information.
Identify whether there was an actual consequence or a near miss. Do not infer that a severe consequence was likely merely because it was possible.
Ask what uncertain event could recur, what conditions might enable it, and what could happen if a relevant control does not prevent or detect it.
Check for duplicate reports, selection effects, missing exposure information, changed procedures, and alternative explanations. State what remains uncertain rather than filling gaps with assumptions.
A recurrence pathway should be specific enough to support a meaningful question, but not more certain than the evidence permits. A useful form is: “If [condition or cause] recurs, [uncertain event] could occur, potentially leading to [consequence].” “Could” matters: a reviewed event demonstrates that a sequence occurred once; it does not establish that the sequence will recur or that every future occurrence will have the same outcome.
Worked Example: From a Blocked Duplicate Payment to a Recurrence Pathway
A hypothetical insurer reviews three reports from a monthly payment process. In report A, an operator resubmitted a payment file after a delayed confirmation; a duplicate instruction was blocked by the payment system. In report B, a team member found a duplicate entry during a manual reconciliation before payment. In report C, one duplicate payment was made after a file was resubmitted, and the error was corrected the next day. Identify what is observed and propose a defensible recurrence pathway.
Step 1: Separate records from event types. There are three reports, but they do not establish three distinct underlying risks. Reports A and C both involve a file resubmission; report B involves a duplicate entry detected during reconciliation. Keep the records distinct while comparing their mechanisms.
Step 2: Classify outcomes. Report A is a near miss: a duplicate instruction existed, but a system control blocked payment. Report B is also a near miss if the duplicate could otherwise have been paid; the record says it was detected before payment. Report C is an incident because a duplicate payment occurred, even though it was corrected the next day.
Step 3: State observed facts, not presumed causes. The records establish that duplicate instructions or entries appeared and that a resubmission occurred in two cases. They do not establish why the confirmation was delayed, whether staff routinely resubmit files, or whether the same control weakness caused every report.
Step 4: Form a recurrence hypothesis. A defensible pathway is: if a delayed or ambiguous file confirmation prompts a resubmission and duplicate detection does not stop the second instruction, a duplicate payment could occur. The reports support the possibility of the sequence; they do not prove that the confirmation delay caused the duplicate in report C.
Step 5: Identify evidence to verify. Review timestamps, resubmission logs, system-control results, and reconciliation procedures. These could clarify whether the reports share a pathway or whether they represent different mechanisms. The common mistake is to count “three reports” as “three separate risks” or to assume that all three had the same cause. The correction is to preserve the record-level detail while grouping only where the event mechanism is supported.
No interest or mortality assumptions are relevant to this process review; no probability model is needed to classify these records or state the pathway.
When Counts Help—and What They Do Not Establish
Counts can help describe observed experience when the records are sufficiently consistent and the amount of exposure is known. The denominator matters: ten events in a small number of transactions may indicate a different pattern from ten events across a much larger number. A count per unit of exposure is a descriptive rate, not automatically the probability of a loss.
For a simple model, suppose recorded events follow a homogeneous Poisson process over exposure \(E\), with constant event rate \(\lambda\) per exposure unit. Then the event count \(N\) has distribution \(N\sim\operatorname{Poisson}(\lambda E)\). Under this model, the maximum-likelihood estimate of the rate is \(N/E\). This model assumes a stable rate over the reviewed exposure and a suitable counting process; it does not correct for events that were missed, changes in reporting, or changing operating conditions.
The estimate follows by maximizing the Poisson likelihood. For \(N>0\), the log-likelihood, omitting terms that do not depend on \(\lambda\), is \(N\log(\lambda E)-\lambda E\). Its derivative with respect to \(\lambda\) is \(N/\lambda-E\). Setting that derivative equal to zero gives \(\widehat{\lambda}=N/E\); the second derivative is \(-N/\lambda^2<0\), confirming a maximum. If \(N=0\), the likelihood is maximized at the boundary estimate \(\widehat{\lambda}=0\), which does not prove that the true rate is zero.
Worked Example: Interpreting a Near-Miss Rate
A hypothetical claims administrator records 12 near misses during 80,000 claim validations over six months. Assume, solely for this calculation, that recorded near misses follow a homogeneous Poisson process with a constant rate per validation and that each validation is counted once. Estimate the recorded near-miss rate per 10,000 validations. Under the same model, calculate the probability of at least one recorded near miss in a future block of 20,000 validations.
Method and calculation. The observed count is \(N=12\), and exposure is \(E=80{,}000\) validations. The estimated rate per validation is \(\widehat{\lambda}=12/80{,}000=0.00015\). For 10,000 validations, the estimated expected count is \(0.00015(10{,}000)=1.5\), so the recorded near-miss rate is 1.5 per 10,000 validations.
For the future block, the Poisson mean is \(\widehat{\lambda}E=0.00015(20{,}000)=3\). Thus the probability of at least one recorded near miss is \(1-P(N=0)=1-e^{-3}=1-0.049787\ldots=0.9502\), to four decimal places.
Check. The expected count scales with exposure: 20,000 is one quarter of 80,000, and the estimated mean count is \(12/4=3\). The probability is below 1 and is high because a Poisson mean of 3 gives a zero-count probability of about 0.0498.
Common mistake and correction. It would be wrong to describe \(12/80{,}000=0.00015\) as the probability of a claim loss per validation: the 12 observations are recorded near misses, not losses. Under the stated model, it is an estimated rate of recorded near misses per validation. The Poisson calculation estimates the chance of at least one such recorded event in a specified exposure block, not the chance of a financial loss. In practice, the constant-rate and complete-recording assumptions also require scrutiny.
The model is useful for illustrating the role of exposure, not for turning a review into a precise forecast. If procedures changed during the period, reporting improved, or the events are clustered by a shared outage, the constant-rate assumption may be unsuitable. Also, a count of recorded incidents alone says nothing about unreported events or pathways that have not yet occurred.
Comparing Records Without Overstating Their Similarity
Repeated reports may indicate a shared pathway, but superficially similar outcomes can have different causes. Conversely, different consequences can arise from the same event sequence. Compare the mechanism and conditions, not only labels such as “payment error” or “system issue.” Preserve distinctions where evidence is incomplete, and express a common pathway as a hypothesis to examine rather than a confirmed causal conclusion.
For example, several records of delayed customer communications might arise from a shared data-feed interruption, from separate staff workflow errors, or from both. The records should be compared for timing, process steps, control operation, and detection. A common consequence does not prove a common event pathway, and a common pathway does not guarantee the same consequence each time.
Worked Example: Finding a Shared Mechanism Across Different Outcomes
A hypothetical underwriting team reviews two events. In the first, a stale reference table led a quote system to display an outdated premium factor; an analyst noticed the discrepancy before the quote was sent. In the second, a similar outdated factor appeared in a renewal file and was sent to a customer, after which the file was corrected. Determine what can be combined into a recurrence hypothesis and what should remain distinct.
Step 1: Identify the observed event. In both records, an outdated reference factor was used in a pricing workflow. The first record ended in detection before communication; the second resulted in an incorrect customer communication that was later corrected.
Step 2: Separate outcome from mechanism. The outcomes differ: one was a near miss and one an incident. The apparent shared mechanism is use of an outdated reference table, but the records do not by themselves establish why the table was stale or whether the same update failure affected both workflows.
Step 3: State the possible pathway precisely. If a reference table is not updated or its version is not checked, a pricing workflow could use an outdated factor. If review controls fail to detect the discrepancy before communication, an incorrect quote or renewal communication could reach a customer. The first part is supported by both records; the control-failure condition is directly relevant to the second outcome and remains a future possibility in the first.
Step 4: Identify what further evidence would distinguish explanations. Compare table version histories, update responsibilities, system logs, review steps, and the dates of both events. Until that evidence is reviewed, do not state that one specific update failure caused both events.
The common mistake is to treat different outcomes as proof of unrelated risks, or to treat a similar technical symptom as proof of one common cause. The correction is to compare the event mechanism separately from the consequence and to mark uncertain causal links as hypotheses.
Limits of Incident and Near-Miss Evidence
A review describes an observed set, not every plausible future pathway. That limitation arises for several reasons:
- Reporting and detection. Minor events and near misses may be less likely to be recorded than visible losses. Changes in reporting practice can alter counts even when underlying processes do not change.
- Selection. Reviews may focus on severe or unusual events, making the selected records unrepresentative of routine operations.
- Exposure. A raw count without the number or type of transactions, operating hours, or other relevant exposure can be difficult to interpret.
- Changing conditions. A historical pathway may no longer be plausible after a process, system, staffing model, or control changes. Conversely, new conditions may create pathways not present in the reviewed records.
- Incomplete descriptions. A short report may omit a crucial condition or alternative explanation. Absence of a detail from a record is not evidence that the condition was absent.
These limitations do not make reviews unhelpful. They guide how conclusions are worded. Say “the records suggest a possible pathway” when evidence is incomplete; say “the records document” only for facts actually established. Use reviews to generate and refine questions about recurrence, then compare those questions with other identification methods and current operating conditions. Do not claim that the reviewed observations exhaust future possibilities. This approach is aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.
Check Your Understanding
For each problem, distinguish what the record shows from what can reasonably be inferred about recurrence.
- A hypothetical service center records four cases in which a customer form was entered twice. In three cases, a duplicate account was caught before activation; in one case, the duplicate account was activated. State how many incidents and near misses are described, and give one cautious recurrence hypothesis. (Answer: 1 incident and 3 near misses; duplicate entry could recur and lead to duplicate account activation if detection controls fail.)
- A hypothetical logistics team finds 6 near misses in 30,000 package scans. Under a constant-rate Poisson model, calculate the estimated near-miss rate per 10,000 scans. (Answer: \(6/30{,}000=0.0002\) per scan, or 2 per 10,000 scans.)
- During 50,000 hypothetical account updates, 5 recorded near misses occur. Assuming a homogeneous Poisson process with rate 0.0001 per update, calculate the probability of at least one recorded near miss in the next 10,000 updates. Give the probability to four decimal places. (Answer: \(1-e^{-0.0001(10{,}000)}=1-e^{-1}=0.6321\).)
- Two hypothetical incidents both cause a delayed customer notice. One followed a failed data transfer; the other followed an unapproved manual hold. Should the reports automatically be treated as one recurrence pathway? State the reason. (Answer: No; the consequence is shared, but the observed mechanisms differ. Compare evidence before proposing a common pathway.)
- A review of 15 hypothetical incidents finds no cases of a particular system outage. Can the reviewer conclude that an outage pathway is not plausible? (Answer: No; the reviewed records do not establish that a pathway is impossible or that the review is a complete inventory.)