Why Effective Challenge Matters
A risk committee adds value when it tests important assumptions, decisions, and control claims rather than merely receiving them. Effective challenge is not measured by how many questions members ask or how forcefully they ask them. It is measured by whether the questions expose a material uncertainty, use relevant evidence, are sufficiently independent of the proposal being reviewed, and lead to a clear next step.
A question such as “Are we comfortable with this?” may invite discussion but does not identify what needs testing. A stronger question names the assumption or evidence at issue, explains why it matters, and asks for a response that could change the committee’s understanding or decision. Challenge should be respectful and focused: its purpose is to improve the quality of judgment, not to assign blame.
These four characteristics are related but distinct. A specific question can still rely on weak evidence; an independent reviewer can identify an issue without ensuring it is resolved. Committee members should assess the whole chain from question to conclusion. This is consistent with the governance responsibilities of risk owners, control functions, and internal audit: challenge does not transfer ownership of the risk or replace independent assurance.
A Practical Test for Committee Challenge
For each material matter, members can ask four questions. The test is a practical evaluation tool, not a numerical score. A weakness in one dimension should not be concealed by strength in another.
| Dimension | What effective challenge does | Warning sign |
|---|---|---|
| Specificity | Identifies a defined assumption, exposure, control, or decision and the concern to be examined. | “Any concerns?” without naming the issue. |
| Evidence | Requests relevant data, a comparison, validation, or a reasoned explanation. | Accepts an assertion such as “the control is working” without support. |
| Independence | Tests the presenter’s view using a suitably separate source or perspective. | Relies only on a self-assessment produced by the team whose proposal is under review. |
| Follow-through | Establishes who will respond, by when, and what conclusion or decision depends on the response. | The question is discussed but neither answered nor assigned. |
A useful way to formulate a question is to state the claim, identify the evidence that would test it, and specify the consequence of an unresolved gap. For example: “Management attributes the increase in reported losses to one event. Can the risk owner reconcile the remaining increase to the forecast, and will the committee defer approval if that reconciliation is not supported?” This is more useful than “Why are losses up?”
State what the committee is being asked to accept, approve, or note.
Identify the assumption, variance, evidence gap, or control dependency that could alter the conclusion.
Ask what data or independent perspective would confirm or challenge the claim.
Record the responsible person, deadline, required response, and any decision or escalation that depends on it.
Using Benchmarks to Make Questions Testable
Evidence-based challenge often starts with a comparison: an actual result against a forecast, an observed control exception against an agreed standard, or a proposed exposure against the current portfolio. The comparison makes the concern concrete. It does not, by itself, establish the cause or prove that a proposal is unacceptable.
For a positive benchmark, the absolute variance is the observed amount minus the benchmark. The relative variance expresses that difference as a proportion of the benchmark. This is useful because the same dollar difference may be material in one context and small in another.
The relative variance formula requires a nonzero benchmark and should be interpreted in context. It does not explain the variance. To reach an explanation, the committee needs an appropriate decomposition, supporting data, and, where warranted, a perspective independent of the original estimate. A favorable overall result can also conceal an adverse component, so members should consider whether the total is being driven by offsetting effects.
Worked Example: Testing a Claims Variance
A hypothetical insurer’s committee receives a quarterly update showing claims paid of $12.4 million against a forecast of $10.0 million. Management says that a single severe event explains the difference; the event accounts for $1.5 million of paid claims. Evaluate the quality of the evidence and frame effective follow-up.
Method. Compare the result with the stated forecast, remove the identified event amount to see what remains, and require evidence that reconciles the residual. The comparison identifies a question; it does not determine the cause.
Step 1: Calculate the total variance. The absolute variance is $12.4 million minus $10.0 million, or $2.4 million. The relative variance is $2.4 million divided by $10.0 million, or 24%.
Step 2: Test the explanation offered. If the event’s $1.5 million is included in actual claims and is the only adjustment, the remaining amount is $12.4 million minus $1.5 million, or $10.9 million. Compared with the $10.0 million forecast, this leaves an unexplained $0.9 million adverse difference. That residual is 9% of the forecast: $0.9 million divided by $10.0 million.
Step 3: Formulate the challenge. “The event explains $1.5 million of the $2.4 million variance. What claims categories or assumptions explain the remaining $0.9 million, and can the risk owner reconcile that amount to the forecast using the quarter’s underlying claims data?”
Step 4: Secure follow-through. The committee should assign the response to the accountable risk owner, set a due date before the next relevant decision, and specify the evidence required—for example, a reconciliation by claims category and a comparison of actual experience with the forecast assumptions. If the reconciliation is incomplete, the committee should record that the cause remains unresolved rather than treating the event explanation as complete.
Check. The identified event plus the residual is $1.5 million + $0.9 million = $2.4 million, the total variance. The common mistake avoided is accepting a plausible partial explanation as if it reconciled the full difference.
Independence and Constructive Scrutiny
Independence in committee challenge does not mean that every issue requires an external reviewer. It means that the evidence is not accepted solely because it comes from the person or team whose performance, proposal, or control is being assessed. The appropriate source depends on the issue. A risk owner may explain operational causes; a control function may provide monitoring evidence; internal audit may provide independent assurance within its role. The committee should not confuse these perspectives or ask one function to assume another’s responsibility.
Challenge is strongest when it is proportionate. For a routine, low-impact matter, a concise explanation supported by existing monitoring may be enough. For a material control weakness or a decision with significant uncertainty, members may require corroborating evidence, a separate review, or a staged decision. The committee should be clear about what evidence would resolve the concern; otherwise, a request for “more assurance” can become indefinite and unhelpful.
Worked Example: Evaluating a Vendor Security Claim
A hypothetical insurer is considering whether to continue using a claims-processing vendor. The business owner reports that a recent security test found 15 control findings and that 12 have been remediated. The owner recommends continuation, describing the remaining findings as “minor.” Determine what the committee should test before relying on that conclusion.
Method. Separate the factual status from the assessment of severity, test the status using an appropriately independent source, and define an action and decision point.
Step 1: State what the reported numbers establish. Twelve of 15 findings are reported as remediated, so three remain open. The reported remediation proportion is 12 divided by 15, or 80%. This percentage does not establish that the remaining findings are minor or that the vendor’s service is acceptably controlled.
Step 2: Identify the evidence gap. The business owner’s summary does not provide the severity of each open finding, the evidence used to verify closure of the 12 resolved findings, or the potential impact of the three open items on claims processing. The committee should not infer that all 15 findings are equally important.
Step 3: Ask a specific, independent question. “For each of the three open findings, what system or data could be affected, what is the severity assessment and rationale, and who independently verified closure of the other 12? Please provide the test report and evidence of remediation, not only the vendor’s summary.”
Step 4: Assign a decision-linked action. The committee can ask the risk owner to obtain the underlying report and a control-function review by a specified date. It should identify who will assess the residual risk and whether continuation is within the relevant delegated authority. If the evidence does not support the “minor” assessment, the matter should return for a decision or be escalated through the established governance route.
Check. The arithmetic is internally consistent: 12 remediated plus 3 open equals 15 findings. The common mistake avoided is treating an 80% closure rate as proof of adequate control, without considering the severity and independence of the evidence.
Challenge That Changes the Decision
Effective challenge need not result in rejecting a proposal. It may confirm the proposal, change its conditions, delay a decision, or reveal that the matter belongs with another decision-maker. The committee’s role is to make the uncertainty and its implications visible, then ensure that the proper person acts within the relevant authority.
A useful test is whether the committee’s question could lead to a different conclusion if the answer were adverse. If the answer cannot affect the decision, the question may be ceremonial. Conversely, if the decision depends on information that has not been provided, the committee should not mistake discussion for resolution.
Worked Example: Challenging Growth in a Concentrated Portfolio
A hypothetical insurer has 50,000 policies, of which 10,000 are in a coastal region. Management proposes adding 10,000 policies next year, including 6,000 coastal policies. The proposal’s summary describes the increase as 20% portfolio growth and does not separately discuss the geographic mix. Evaluate the missing challenge and specify what follow-up would make the decision better informed.
Method. Recalculate the exposure mix before and after the proposal, compare the changes in total and concentrated exposure, and request analysis of the risk implications rather than assuming a particular outcome.
Step 1: Calculate the current coastal share. The current share is 10,000 divided by 50,000, or 20%.
Step 2: Calculate the proposed portfolio totals. If all 10,000 proposed policies are added, the resulting total is 50,000 + 10,000 = 60,000. The coastal count becomes 10,000 + 6,000 = 16,000.
Step 3: Calculate the post-proposal coastal share and changes. The resulting share is 16,000 divided by 60,000, or approximately 26.67%. This is an increase of approximately 6.67 percentage points from the current 20%. Coastal policies increase by 6,000 divided by 10,000, or 60%, while total policies increase by 10,000 divided by 50,000, or 20%.
Step 4: Frame the challenge. “The proposal grows the total portfolio by 20%, but the coastal count grows by 60% and its share rises from 20% to about 26.67%. Has the exposure analysis assessed the geographic concentration and its effect on the relevant risk scenarios? Please provide the analysis and identify whether the proposal is within the applicable approval authority.”
Step 5: Close the loop. The committee should assign the analysis to the accountable risk owner, request the relevant scenario or exposure evidence, and set a deadline before approval. If the decision is outside the committee’s delegated authority, it should make a recommendation or escalate the matter to the authorized decision-maker rather than imply that the committee itself has approved it.
Check. The proposed mix reconciles: 16,000 coastal policies plus 44,000 non-coastal policies equals 60,000 total. A common calculation error is to call 6,000 divided by 60,000, or 10%, the increase in coastal concentration. That figure is the new coastal policies as a share of the final portfolio; the increase in the coastal share is about 6.67 percentage points.
Common Traps and a Closing Test
- Counting questions instead of testing quality. Several broad questions can produce less useful scrutiny than one question tied to a material assumption and a clear evidence request.
- Confusing confidence with evidence. A presenter’s confidence, or a committee’s consensus, does not reconcile a variance or validate a control claim.
- Accepting an incomplete explanation. A cause that explains part of an adverse result does not automatically explain the remainder.
- Treating self-assessment as independent assurance. A risk owner’s account is important evidence of ownership, but it may not independently test the owner’s own conclusion.
- Leaving actions open-ended. “Provide more detail” is not a complete action unless the committee identifies the detail required, who will provide it, and when.
- Overstepping authority. Challenge can identify a concern and prompt escalation; it does not itself grant a committee a decision right it does not hold.
The central discipline is to connect the question to evidence and the evidence to a conclusion. For candidates aligned with the CAS Exam 9 (Risk Management for Actuaries) syllabus, this means distinguishing substantive scrutiny from formal committee activity and recognizing that accountability remains with the appropriate risk owner and decision-maker.
Check Your Understanding
For each hypothetical case, identify the central weakness in the challenge and the next step that would make it effective.
- A committee is told that quarterly operating losses were $3.6 million against a $3.0 million forecast. A single incident accounts for $0.2 million. What is the remaining variance, what is it as a percentage of forecast, and what evidence should the committee request? (Answer: $0.4 million remains; it is 13.33% of the $3.0 million forecast. Request a reconciliation of the residual by loss category or cause, supported by underlying data and assigned to an owner with a due date.)
- A project sponsor says a control has passed all tests but provides no test period, population, or reviewer. Name two specific evidence questions and one independence check. (Answer: Ask what period and population were tested and what exceptions were found; ask for the test results and closure evidence. Confirm whether testing was performed or validated by a function sufficiently separate from the project sponsor.)
- A portfolio has 8,000 policies, including 1,600 in one region. A proposal adds 2,000 policies, of which 800 are in that region. Calculate the regional share before and after, and name the relevant challenge. (Answer: Before, 1,600/8,000 = 20%; after, 2,400/10,000 = 24%. Ask for evidence assessing the regional concentration and its implications before the authorized decision-maker acts.)
- A committee asks a risk owner to “look into” an unresolved control exception. The minutes give no deadline or decision point. What follow-through is missing? (Answer: Identify the required evidence or analysis, name the accountable owner, set a due date, and state what conclusion, decision, or escalation will follow from the response.)
- A vendor’s 20 controls were reviewed; 16 are reported as satisfactory and four have exceptions. The presenter says the overall result is acceptable because 80% passed. What is wrong with that conclusion, and what should be tested? (Answer: The 80% pass rate does not show whether the four exceptions are material or whether the review was reliable. Request the severity, impact, remediation status, and supporting test evidence for each exception, with suitable independent validation and a decision-linked follow-up.)