Why KRI Selection Begins With the Exposure
A dashboard can contain many numbers and still provide little useful information about risk. A measure may be easy to obtain yet have a weak connection to the risk of concern; another may be relevant but too inconsistent to support a sound decision. Selecting key risk indicators (KRIs) therefore begins with the exposure being monitored, not with the data already available.
The exposure should be stated clearly enough to identify the business objective, the process or dependency involved, the way the risk could arise, and the consequences that matter. A KRI is then judged by how well it helps decision-makers observe a meaningful condition of that exposure. The approach is aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus.
A KRI differs from a performance measure in its purpose. A measure of completed transactions, for example, may describe workload or productivity. It becomes a plausible KRI only if its relationship to a specified risk is clear. A process-volume measure might provide essential context for another indicator, but that does not automatically make it a KRI on its own.
Four Tests for Candidate Indicators
A practical review compares candidates on four dimensions. These dimensions are related but not interchangeable: strong relevance cannot compensate for unusable data, and excellent data quality cannot make an irrelevant measure useful.
| Dimension | Selection question | Warning sign |
|---|---|---|
| Relevance | Does the measure have a credible connection to the defined exposure or a condition affecting it? | The measure is convenient but the risk mechanism is unclear. |
| Sensitivity | Would the measure change when a meaningful change in the exposure or its controls occurs? | The measure mostly reflects unrelated activity or changes only after substantial delay. |
| Reliability | Can it be calculated consistently from sufficiently complete and dependable data? | Definitions, coverage, or recording practices vary across units or periods. |
| Actionability | Can a decision-maker use the information to investigate, decide, or respond? | The measure signals a condition but provides no practical route to understanding or response. |
These tests require judgment in context. Sensitivity does not mean that every small movement is meaningful: a highly variable measure can react to random fluctuations as well as to changes in exposure. Reliability is not simply the existence of a data source; consistent scope, numerator, denominator, timing, and calculation matter. Actionability concerns whether the information can support a decision, not whether a particular response is already specified.
A Structured Selection Method
A disciplined selection process makes the reasoning visible. First define the exposure. Then identify plausible candidates and specify what each measures. Review the candidates against the four dimensions, check how much of the exposure each captures, and remove measures that add little distinct information. A scoring table can organize the review, but it does not replace the underlying explanation.
Describe the objective at risk, the relevant process or dependency, and the important ways the exposure could affect that objective.
For each candidate, state exactly what is counted or measured, its scope, and any denominator needed to interpret it.
Assess relevance, sensitivity, reliability, and actionability. Record important limitations rather than hiding them in a single score.
Ask which risk mechanisms are represented and whether two candidates mainly repeat the same information.
Retain a manageable group that jointly informs the exposure. Keep a lower-scoring candidate when it adds important, distinct information and explain why.
If scoring is used, define the scale before comparing candidates. For example, a team could rate each dimension from 1 (weak) to 5 (strong), with equal weights for an initial screen. The resulting average is a summary of structured judgments, not a probability of loss or an objective measure of risk. Equal weighting is a modeling choice; a different exposure may require different weights. Scores should be accompanied by evidence and limitations.
When a Rate Is More Informative Than a Count
A candidate count can be misleading if the amount of activity that creates opportunities for the event varies. Suppose the expected number of events is proportional to the number of comparable opportunities. Under that assumption, let \(N\) be the event count and \(E\) the number of opportunities, and suppose \(E[N\mid E]=\lambda E\). Then:
The rate \(N/E\) removes the expected effect of volume under this proportionality assumption, which can make comparisons more informative when exposure volume changes. This reasoning does not apply automatically. It can fail if opportunities differ materially in risk, if the relationship is not proportional, or if the numerator and denominator have inconsistent scope. A rate should not be selected merely because it looks more sophisticated than a count.
Worked Example: Selecting KRIs for Payment Integrity
Method. Define the risk, score plausible candidates on a common scale, and select measures with distinct connections to the exposure. A hypothetical insurer is concerned that errors or unauthorized changes in its claim-payment workflow could result in incorrect or misdirected payments. The selection team uses a 1-to-5 scale, where 1 is weak and 5 is strong, and gives the four dimensions equal weight. Scores are informed judgments for this scenario, not universal benchmarks.
| Candidate | Relevance | Sensitivity | Reliability | Actionability |
|---|---|---|---|---|
| Payment-detail changes lacking independent verification, per 1,000 changes | 5 | 4 | 4 | 5 |
| Payment corrections, per 1,000 payments | 4 | 4 | 5 | 4 |
| Total number of claims paid | 2 | 1 | 5 | 2 |
| Late-payment rate | 3 | 3 | 5 | 4 |
For the first candidate, the dimension scores sum to \(5+4+4+5=18\); the average is \(18/4=4.50\). For payment corrections, the sum is \(4+4+5+4=17\), so the average is \(17/4=4.25\). Total claims paid scores \(2+1+5+2=10\), giving \(10/4=2.50\). The late-payment rate scores \(3+3+5+4=15\), giving \(15/4=3.75\).
The team selects the first two candidates. The first reflects a control condition around changes to payment details; the second reflects correction activity within payment processing. They address related but distinct ways payment integrity can be affected. Total claims paid is dependable to count, but it primarily describes volume and is weakly connected to this defined exposure. Late-payment rate is useful for a different service concern, but it does not directly add enough information about incorrect or misdirected payments to be retained in this compact set.
Check. The arithmetic gives averages of 4.50, 4.25, 2.50, and 3.75, respectively. The selection does not simply retain every candidate above an arbitrary score: it also checks whether a measure informs the particular exposure and adds distinct information. The common mistake avoided is treating a high score—or an easily obtained number—as sufficient justification for selection.
Worked Example: Choosing a Denominator for Reconciliation Exceptions
Method. Compare a raw count with a rate when the amount of activity differs. A hypothetical insurer monitors incomplete daily cash reconciliations at several service units. In one reporting period, 9 of 300 scheduled reconciliations are incomplete; in another, 9 of 150 are incomplete. Assume for this comparison that scheduled reconciliations are reasonably comparable opportunities and that the numerator and denominator cover the same units and period.
The first-period rate is \(9/300=0.03\), or \(3\%\). The second-period rate is \(9/150=0.06\), or \(6\%\). The rate has doubled even though the count remains 9. This normalized indicator better reflects the difference in incomplete reconciliations per scheduled opportunity under the stated assumption.
Common mistake—using only the count. A reader might calculate \(9-9=0\) and conclude that the measured condition is unchanged. That calculation ignores the change in scheduled activity. The correction is to compare like denominators: \(9/300=3\%\) versus \(9/150=6\%\). The rate is higher in the second period.
The count may still be useful for estimating the workload involved in resolving exceptions, so normalization does not make counts worthless. The selection decision depends on the question the indicator is meant to answer. Here, the risk-monitoring question concerns incomplete reconciliations relative to the opportunities to complete them. If scheduled work were a poor proxy for actual reconciliation exposure, the proposed rate would need further review rather than automatic acceptance.
Check. Both numerators equal 9, but the denominators differ by a factor of two. The resulting rates are \(3\%\) and \(6\%\), so the rate communicates a difference that the count alone conceals.
Worked Example: Selecting a Compact Set for Vendor Dependency
Method. Use dimension scores as a screen, then consider whether candidates collectively represent distinct parts of the exposure. A hypothetical insurer depends on a vendor’s policy-administration system; an interruption could disrupt important policy transactions. The team has capacity to retain two indicators and uses the same equal-weight, 1-to-5 scale as in the previous example.
| Candidate | Relevance | Sensitivity | Reliability | Actionability | Score sum | Average |
|---|---|---|---|---|---|---|
| Critical workflows that pass a recovery test, as a percentage of those tested | 5 | 4 | 4 | 5 | 18 | 4.50 |
| Unresolved material vendor-control findings | 4 | 4 | 3 | 3 | 14 | 3.50 |
| Unplanned interruption hours for the critical system during the month | 4 | 4 | 5 | 4 | 17 | 4.25 |
| Number of meetings held with vendor staff | 1 | 1 | 4 | 1 | 7 | 1.75 |
For the recovery-test measure, \(5+4+4+5=18\), and \(18/4=4.50\). For unresolved findings, \(4+4+3+3=14\), and \(14/4=3.50\). For interruption hours, \(4+4+5+4=17\), and \(17/4=4.25\). For meetings, \(1+1+4+1=7\), and \(7/4=1.75\).
The team selects recovery-test results and interruption hours. They describe different aspects of the dependency: the first informs whether critical workflows can be recovered under the tested conditions; the second records interruption experienced in operation. In this hypothetical review, evidence shows that many unresolved findings concern recovery-test weaknesses, so retaining both the findings count and test results would add less distinct information than retaining interruption hours. Meetings are easy to count but do not establish that risk-reducing work is effective.
Check. The selected pair has average scores of 4.50 and 4.25 and covers both tested recovery capability and observed interruptions. The method avoids selecting two measures simply because they refer to the same vendor, or selecting a convenient activity measure without a clear connection to the exposure.
Common Selection Traps
- Scoring without defining the risk. A score has no stable meaning until the exposure and decision purpose are clear.
- Confusing availability with reliability. A data field can exist while being incomplete, inconsistently defined, or mismatched to the risk scope.
- Assuming a rate is always superior to a count. A denominator must represent comparable opportunities, and the rate may answer a different question from the count.
- Keeping many near-duplicates. Several similar measures can create the appearance of broad coverage without adding much information.
- Choosing only the highest average scores. A compact set should jointly represent important parts of the exposure; a somewhat lower-scoring measure may add distinct, valuable coverage.
- Treating an indicator as a verdict. A KRI supplies information for review and decision-making. Its movement alone does not prove why a risk changed or establish that a loss will occur.
Check Your Understanding
For each problem, identify the selection reasoning and check the final result.
- A hypothetical claims unit records 12 payment corrections among 600 payments in one period and 15 among 1,000 in the next. Calculate both correction rates and state which period has the higher rate. (Answer: 2.00% and 1.50%, respectively; the first period has the higher rate.)
- A candidate KRI receives relevance 4, sensitivity 3, reliability 5, and actionability 4 on an equal-weight 1-to-5 scale. Calculate its average and identify one limitation of using that average alone. (Answer: 4.00; the average can conceal a weaker dimension and does not establish that the candidate adds distinct exposure coverage.)
- A hypothetical service process has 6 unresolved exceptions among 120 scheduled reviews in one unit and 8 among 200 in another. Calculate each exception rate and identify the higher-rate unit. (Answer: 5.00% and 4.00%; the first unit has the higher rate.)
- A risk team has capacity for two measures. Recovery-test pass rate scores 18 across the four equally weighted dimensions; unplanned interruption hours score 16; meeting count scores 12. The first two measures cover tested recovery capability and experienced interruption, while meeting count has no demonstrated connection to risk reduction. Which pair should the team retain, and why? (Answer: Retain recovery-test pass rate and interruption hours; they have the two highest sums and provide complementary information.)
- A hypothetical payment process reports 20 unchecked bank-detail changes among 400 total changes in one period and 18 among 300 in another. Calculate both rates and explain why comparing counts alone could mislead the selection discussion. (Answer: 5.00% and 6.00%; the second period has the higher rate, while raw counts suggest fewer unchecked changes despite the higher proportion.)