Why Alert Design Is a Dosing-Safety Decision
A model-informed dosing service can produce a sound recommendation and still fail to support safe care if its alert arrives too late, interrupts for a low-value issue, or obscures a more urgent warning. Alert design is therefore part of the clinical decision-support logic: it determines which concern is surfaced, when it is surfaced, and what response the system asks a clinician to consider.
The central design task is to balance two risks. A warning that is too quiet or delayed may not prompt action before harm occurs. A warning that interrupts too often, or for concerns that do not require immediate attention, can become easy to dismiss. This tutorial treats severity, timing, and interruptiveness as separate dimensions that must be considered together. It does not prescribe universal clinical thresholds; those must be established for the medication, model, setting, and local clinical policy.
A severe potential outcome does not automatically mean every related alert should interrupt every user. The concern must also be sufficiently credible, relevant to the current decision, and actionable. Conversely, a moderate concern may warrant prompt attention when the opportunity to act is about to pass. For a deployed service, the design should be based on validated model behavior, reliable inputs, and a clearly defined clinical response.
A Practical Classification Rule
A local alert policy can define categories for each dimension. The categories below are examples of a workable framework, not universal medical standards. A team should specify the thresholds and actions for its own use case and document the evidence and clinical review supporting them.
| Dimension | Example categories | Design question |
|---|---|---|
| Severity | High: plausible serious harm; moderate: clinically meaningful but less serious harm; low: limited consequence or optimization opportunity | What could happen if the concern is not addressed? |
| Timing | Immediate: before the next dose or action; soon: within a defined short review window; routine: at the next appropriate review | How long can review safely wait? |
| Interruptiveness | Passive display; non-blocking notification; interruptive alert requiring acknowledgement or a response | How much attention should the system demand now? |
The classification should produce a clear next step. An alert that says only “model concern detected” transfers the interpretation burden to the clinician. A useful warning states the relevant concern, the time frame, the recommended action or review, and the basis for the warning at a level that can be understood at the point of care. The recommendation itself should remain distinguishable from the alert’s urgency.
The earlier material on communicating model uncertainty is relevant when the model’s output is uncertain. A high-severity label should not hide uncertainty in the underlying prediction. If uncertainty is material, the alert should convey it or direct the user to an appropriate review rather than presenting a conditional prediction as a certain event.
From Risk to Interruptiveness
One useful design check is to compare the expected harm that an alert may prevent with the burden it imposes. For a specified decision and time window, define \(p\) as the estimated probability of the concerning event without the alert, \(L\) as the loss associated with that event in a common decision unit, \(r\) as the probability that the alert leads to an effective response, and \(C\) as the burden of the alert in the same unit. A simplified expected net benefit is:
This expression is a structured way to expose assumptions, not a validated scoring rule. The terms must refer to the same population, decision, and time horizon; the response probability must reflect what the alert can realistically change; and the harm and burden estimates must be expressed in a comparable unit. If those assumptions cannot be defended, the calculation should not be used to claim that an alert is beneficial. Clinical judgment, policy, and evaluation data remain essential.
The expression also explains why severity alone is not enough. Even when \(L\) is large, an alert may have little value if the event is very unlikely, the warning is not actionable, or no effective response is available. On the other hand, an alert may be worthwhile when the possible harm is serious and a timely action can meaningfully reduce it. Interruptiveness should be the least forceful level that is likely to achieve the needed response within the available time.
Worked Example: Choosing a Channel for a High-Severity Concern
Suppose a hypothetical dosing service flags a patient-specific exposure concern that a locally reviewed policy classifies as potentially serious. A dose-related decision is due within 20 minutes. For a design exercise, the team estimates a 0.04 probability of the concerning event without an alert, a loss of 100 decision units if it occurs, a 0.60 probability that a timely alert leads to an effective response, and a burden of 0.50 units for an interruptive alert. These are illustrative assumptions, not clinical estimates.
The expected harm reduction is:
The illustrative net benefit is \(2.40 - 0.50 = 1.90\) units. Under the stated assumptions, an interruptive alert is a plausible design choice because the concern is serious, the decision is soon, and the alert may enable an effective response. The arithmetic does not prove that the alert is safe or effective in practice; the inputs and resulting behavior still require clinical review and evaluation.
Worked Cases: Timing and Relevance Change the Design
Worked Example: A Concern That Can Wait for Routine Review
In a hypothetical service, a model estimates a modest departure from a target exposure range. Local review classifies the potential consequence as low severity, and no dosing action is due for four days. Assume, solely for this design exercise, \(p=0.01\), \(L=10\) units, \(r=0.50\), and an interruptive-alert burden of \(C=0.40\) units.
The expected harm reduction is \(0.01 \times 10 \times 0.50 = 0.05\) units, giving an illustrative net benefit of \(0.05 - 0.40 = -0.35\) units for the interruptive option. The result argues against interrupting the clinician for this scenario under these assumptions. A passive display or a non-blocking reminder at the next appropriate review may be more proportionate, provided the concern remains visible and can be acted on in time.
This is not a rule that low-severity concerns should never be shown. It shows why the timing and available response matter: a concern that does not need action now may be communicated without demanding immediate attention.
Worked Example: A Serious Concern With a Delayed Action Window
Consider a different hypothetical case. A model identifies a possible high-severity concern, but the next relevant dose decision is expected in 36 hours, and the clinical policy allows review by the next scheduled team assessment. The concern is credible and actionable, but it does not require an immediate interruption during the current encounter.
The classification is high severity, non-immediate timing, and therefore not automatically “interruptive now.” A non-blocking notification or a clearly prioritized task for the defined review window may be appropriate if the system can reliably deliver it and the workflow ensures it will be seen before the decision. The design must also specify what happens if the review window is missed. If the dose becomes due sooner than expected, the urgency classification changes and the alert may need to escalate.
The key check is temporal: the notification pathway must allow enough time for review and action. A nominally non-interruptive alert is not an adequate substitute if it can remain unseen until after the dosing decision.
Justifying a Proportionate Alert
The classification can be applied as a sequence of explicit checks. This helps prevent a high score on one dimension from concealing a failure on another.
Check that required inputs are available, current, and internally consistent, and that the model output is suitable for this patient and decision. If a critical input is unreliable, a definitive dosing warning may mislead; direct the user to the appropriate data check or review instead.
Specify the consequence if no effective action is taken. Do not use the word “severe” merely because a model output crossed a numerical boundary; connect the boundary to a reviewed clinical consequence.
State when the decision must be reviewed and what event would make the concern more urgent. The timing category should be tied to the actual opportunity to intervene.
Use an interruption when timely attention is necessary and a useful response is available. Otherwise, choose a less disruptive channel that still makes the concern visible before action is needed.
Specify whether repeated alerts are suppressed, updated, or escalated, and identify the conditions that justify a new interruption. A changed patient state or a nearer deadline may justify escalation; an unchanged warning usually does not.
The sequence is a design justification, not a substitute for clinical governance. Its purpose is to make the assumptions reviewable and the alert’s behavior predictable.
Measuring Whether the Design Works
Alert evaluation should count both the warnings that appear and the warnings that do not. A large number of acknowledgements does not establish that alerts were useful, and a low alert rate does not establish that important warnings were preserved. Evaluation should stratify results by severity, timing, interruptiveness, and clinical context, then examine whether the intended response occurred in the required window.
Useful measures include the number of interruptive alerts per 100 relevant dosing decisions, the proportion judged actionable on review, the proportion of urgent concerns that were surfaced before the decision deadline, and the frequency of duplicate alerts for an unchanged condition. Review missed alerts as well as displayed alerts: a quiet design that fails to surface urgent cases can appear efficient if only interruption counts are measured.
Worked Example: Interpreting an Alert-Burden Review
A hypothetical review examines 120 dosing decisions and finds 18 interruptive alerts. Of these, a clinical panel judges 12 actionable and 6 not actionable for the decision being made. The panel also identifies 4 urgent concerns that were not surfaced before their action deadline.
The interruptive-alert rate is \(18/120=0.15\), or 15 alerts per 100 decisions. The actionable proportion among displayed alerts is \(12/18=0.667\), or 66.7%. The missed-urgent count is 4; it must be reported separately rather than hidden by the display rate or actionable proportion.
These summaries identify different design problems. The six non-actionable alerts may indicate overly broad triggers, poor timing, or inadequate context. The four missed urgent concerns require investigation of trigger coverage, data availability, and timing logic. The counts alone do not show why either problem occurred or whether a particular alert caused a clinical outcome, so case review is needed before changing thresholds.
Common Failure Modes and Alternatives
Several failure modes recur when severity, timing, and interruption are collapsed into a single “alert level.”
- Interrupting whenever a prediction is unusual. An unusual estimate is not necessarily harmful or actionable. Establish the clinical consequence and response pathway before assigning urgency.
- Using severity as a proxy for timing. A serious concern can have a longer review window; a less serious concern can still require prompt action if a decision is imminent.
- Repeating an unchanged warning. Repeated interruptions can add burden without adding information. Prefer a persistent, visible status or a changed alert only when the state, action window, or recommended response changes.
- Suppressing alerts after a dismissal without context. A prior acknowledgement does not prove that the issue is resolved. The earlier material on clinician overrides and rationale capture is relevant when recording how a dismissal or override is interpreted.
- Alerting on unreliable inputs as if they were certain. If data quality or timing undermines the recommendation, disclose the limitation or route the case for review rather than presenting false precision.
Alternatives to an interruptive alert include a passive display, a non-blocking notification, a prioritized review task, or a recommendation presented when the relevant decision is being made. These are not interchangeable: their suitability depends on whether the channel can reach the responsible user before the action window closes. Interruptive alerts are most defensible when potential harm is important, the need is time-sensitive, and an effective response is available. Their limitations include interruption burden, false positives, habituation, and the possibility that a warning is technically displayed but not meaningfully understood.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A hypothetical warning concerns a potentially serious outcome, but review is not needed until tomorrow and the current dose decision is complete. Which dimensions should guide whether to interrupt now?
- A low-severity estimate appears slightly outside a model target, but the input sample time is uncertain. Should the system present a definitive interruptive dosing warning? Explain.
- In a design exercise, \(p=0.02\), \(L=50\), \(r=0.40\), and \(C=0.30\) units. Calculate the simplified expected net benefit and state one limitation.
- A review finds 10 actionable alerts among 25 displayed alerts and 3 urgent concerns that were missed. Calculate the actionable proportion and explain why it is not a complete measure of performance.
- Why may an unchanged warning be better handled as a persistent status than as a repeated interruption?
Solutions. (1) Separate severity from timing: determine the actual action deadline and whether a reliable non-interruptive route will ensure review tomorrow. Seriousness alone does not establish a need to interrupt now. (2) No, not as a definitive dosing warning. Uncertain timing may undermine the trigger; the system should communicate the limitation or route the case for appropriate review under local policy. (3) \(p L r=0.02 \times 50 \times 0.40=0.40\) units, so net benefit is \(0.40-0.30=0.10\) units under the stated assumptions. This simplified result depends on uncertain estimates and does not establish real-world benefit. (4) \(10/25=0.40\), or 40%. This describes only displayed alerts judged actionable; it does not account for the 3 missed urgent concerns, consequences, or whether users could act in time. (5) Repeating an unchanged warning can increase interruption burden without adding information. A persistent status can preserve visibility, while a new interruption can be reserved for a meaningful change in risk, timing, or required action.