Why Site Readiness Is a Separate Decision
A dosing service can have a well-performing model and a carefully designed interface yet still be unsuitable for a particular site. Staff may not be available when recommendations are needed, essential laboratory results may arrive late, or the local workflow may not provide a clear person responsible for reviewing a recommendation. Site readiness asks whether the people, systems, information, workflows, and oversight needed for safe use are in place at the intended location.
This is an operational assessment, not a conclusion about the model’s predictive performance. It also differs from deciding whether a site is ready to evaluate the service in a limited test. The question here is whether the site has resolved the prerequisites required for its intended deployment scope, or has a controlled plan to resolve any noncritical gaps before that scope begins.
Readiness is specific to a site and a use case. A service intended for weekday inpatient dosing may require different coverage and data pathways from one intended for continuous use in several care settings. Begin by stating the population, care setting, users, operating hours, and decisions in scope. Without that boundary, an item such as “staffing is adequate” has no operational meaning.
Assess Five Readiness Domains
Use a consistent set of domains so that an assessment does not focus only on software installation or training. For each requirement, record the evidence reviewed, the gap or uncertainty, the risk if it remains unresolved, and the person responsible for the assessment. Evidence should be local and current: a signed process, a tested account, an observed workflow, or an approved policy is stronger than an informal assurance that something “should work.”
| Domain | Questions to answer | Examples of useful evidence |
|---|---|---|
| Staffing and capability | Who reviews recommendations, who acts on them, and who covers absences and off-hours? Are responsibilities understood? | Named role coverage, approved schedules, competency records, escalation contacts |
| Infrastructure | Can authorized users access the service in the intended environment? Are devices, interfaces, connectivity, support, and downtime arrangements available? | Access checks, interface ownership, service-desk route, documented downtime procedure |
| Data availability | Are the required patient, medication, laboratory, and timing data available at the point of decision in usable form? | Local data-field inventory, source ownership, timestamp and unit checks, exception handling |
| Local workflow | Where does the recommendation appear, who reviews it, and how are acceptance, modification, or non-use handled? | Observed workflow, role map, escalation route, documented review and follow-up steps |
| Governance | Who authorizes use, defines scope, manages access and incidents, and approves changes to local processes? | Approval record, named accountable lead, incident route, access and change-control responsibilities |
Assess each domain against the intended scope, not against an idealized universal standard. For example, a service that is not intended for overnight use still needs a clear procedure for a patient whose dose decision arises overnight. That procedure might direct the clinician to an established alternative process; it must not leave responsibility ambiguous.
Use Evidence, Gates, and a Transparent Rubric
A practical assessment can label each requirement met, partially met, not met, or unknown. Define these labels before reviewing the site. “Met” means the requirement is supported by current, site-specific evidence. “Partially met” means some conditions are in place but a stated limitation remains. “Not met” means the requirement is absent or contradicted by evidence. “Unknown” means the assessor has not obtained enough evidence to decide; it is not a favorable status.
Some requirements are readiness gates: if unmet, the intended use cannot proceed safely. Examples include no accountable clinician for reviewing a recommendation, no reliable route to required input data, or no authorized process for handling a system outage. Other gaps may be manageable before a defined future date, provided that they do not expose users or patients to the unsupported use. Classify gates explicitly and state the reason for each classification.
A numerical rubric may help compare domains or track progress. If used, state its scale and calculation, and label it as a local management aid rather than a validated safety measure. For instance, a site might assign 0 for not met, 1 for partially met, and 2 for met, then divide the points earned by the maximum possible points. That percentage describes rubric completion only. It does not establish that remaining gaps are harmless or that the model is clinically valid at the site.
The reason is straightforward: an average permits compensation. Strong infrastructure and training scores could numerically offset the absence of anyone authorized to review a dose recommendation. But those strengths do not supply the missing clinical responsibility. Therefore, calculate a summary only after separately identifying gates; readiness requires both an acceptable summary under the local policy and completion of every gate.
Worked Example: A High Summary Score With a Critical Gap
Hypothetical site. A hospital assesses five domains using 0, 1, or 2 points, where 0 means not met, 1 partially met, and 2 met. Its results are staffing 2, infrastructure 2, data 2, workflow 1, and governance 1. The local policy treats named clinical responsibility for reviewing recommendations as a critical gate. The assessment finds that no role has been formally assigned this responsibility.
Calculate the rubric summary. The points earned are \(2+2+2+1+1=8\). The maximum is \(5\times2=10\), so the completion percentage is \(8/10\times100=80\%\).
Interpret the evidence. The 80% figure indicates the rubric’s level of completion; it is not an 80% probability of safe use. Because no reviewer role is assigned, the critical gate is unmet. The site is therefore not ready for the defined use, regardless of the summary score.
Action required. The site must assign an accountable role, define backup coverage and escalation, and document that users understand the responsibility. The assessor should verify the approved role assignment and the operational coverage before marking the gate met.
Turn Gaps Into an Action Plan
An assessment is useful only if it leads to decisions and verifiable work. For each gap, specify the requirement, the evidence showing the gap, its consequence, the action, one accountable owner, dependencies, a due date, and objective closure evidence. “Improve data quality” is not an actionable item. “The interface owner will confirm that the current weight and its collection time are available to the dosing workflow; closure requires an approved field mapping and a documented test of missing or stale values” is more precise.
Prioritize first by whether the gap blocks a gate, then by the potential consequence and likelihood of encountering it, and then by dependencies and time to resolve. A gap that affects every recommendation usually deserves earlier attention than a rare cosmetic issue. If several actions depend on an interface change, do not schedule downstream sign-off as if that dependency had already been met. Record interim controls only when they are explicit, feasible, and authorized.
State the intended users, patients, setting, operating hours, and decision. Phrase each readiness item so that it can be checked.
Review records and system arrangements, and confirm important workflow claims with the people who perform the work. Mark unavailable evidence as unknown.
Decide whether the item is a critical gate, a noncritical gap, or met. Explain why the classification fits the defined scope.
Name an accountable owner, dependencies, a completion date, and any approved interim control. Escalate actions that lack an owner or feasible route to completion.
Check the specified evidence, update the assessment, and document whether every gate is met for the stated scope. Reassess when a material assumption or local process changes.
Worked Example: Resolving a Data-Availability Gap
Hypothetical assessment finding. A service requires the latest serum concentration and the time it was collected. At a site, the concentration is visible in the EHR, but its collection time is not passed to the dosing workflow. Staff sometimes enter a time manually, and the assessment finds no standard instruction for this step.
Why the gap matters. The concentration value alone may be insufficient for the intended decision: the clinical meaning depends on when the sample was collected relative to dosing. The current arrangement therefore does not establish that the required data are reliably available at the decision point.
Action plan. The site data lead owns confirmation of the source field and interface mapping; the clinical informatics lead confirms the local display and exception workflow. The dependency is agreement on the authoritative timestamp field. Closure evidence is an approved mapping plus documented checks showing the value and collection time arrive together, and that absent or unusable times are clearly identified rather than silently treated as valid.
Readiness decision. Until this requirement is met, the site should not represent the relevant data pathway as ready. Whether the gap blocks all intended use or only a narrower subset depends on the stated scope and an authorized clinical process; that decision must be explicit, not inferred from the fact that staff can sometimes enter a time manually.
Check Whether the Assessment Is Credible
Readiness findings can be misleading when they rely on optimistic assumptions or omit the people who do the work. Look for contradictions between written procedures and actual practice, unclear ownership, evidence that applies to a different care setting, or “met” ratings based only on plans for future work. A requirement with no evidence should remain unknown. It should not be converted to met because an assessor expects that a problem will be rare.
Check whether the assessment covers routine and exception conditions: absence of the usual reviewer, delayed or missing inputs, access problems, and service downtime. This does not require re-evaluating the model. It tests whether the local people and processes know what to do when ordinary assumptions fail. If an exception has no clear owner or response, record it as a readiness gap.
The strongest assessment combines document review with targeted conversations and observation. A policy may name a role, while staff may describe a different actual process. That discrepancy is evidence to resolve, not a reason to choose whichever account is more convenient. Record who supplied the evidence, when it was checked, and what scope it supports so the decision can be revisited when local arrangements change.
Comparison, Limitations, and Common Pitfalls
A domain-by-domain readiness assessment is more informative than a single checklist completion rate because it identifies where a weakness lies and who must address it. A weighted score can help managers summarize progress, but weights introduce judgment and can conceal a critical deficiency. A binary go/no-go decision is clear but can hide important distinctions among noncritical gaps. In practice, use a gate decision for essential prerequisites and a detailed action register for the remaining work; use an aggregate score only as a secondary tracking aid.
Readiness is also time- and scope-dependent. A change in operating hours, staffing, interface, patient population, or local governance may invalidate part of an earlier assessment. A completed assessment does not prove the model is accurate, establish clinical effectiveness, or guarantee that every future encounter will follow the intended process. Those are separate questions. It establishes only that the site’s defined operational prerequisites have been assessed against evidence at a stated point in time.
- Pitfall: treating a high score as permission to proceed. A score can average away a critical gap. Identify gates separately and do not allow unrelated strengths to compensate for them.
- Pitfall: marking planned work as complete. A requested interface change is not evidence that the required data are available. Close the item only when the stated evidence has been checked.
- Pitfall: treating “unknown” as “probably met.” Missing evidence is uncertainty. Assign an evidence-gathering action or treat the issue as unresolved.
- Pitfall: assigning an action to a group rather than an owner. If responsibility is collective, accountability can disappear. Name one accountable owner and identify contributors separately.
- Pitfall: assessing a generic site rather than a defined use. Requirements depend on users, setting, hours, and decisions. State those boundaries before rating readiness.
Worked Example: Prioritizing Staffing Actions
Hypothetical site. A clinic plans to use dosing recommendations during weekday clinic hours. One pharmacist is trained, but that pharmacist is absent on some clinic days. There is no named backup, and the local process does not say who handles a recommendation when the pharmacist is unavailable. The same assessment finds that a nonurgent staff reference sheet needs updating.
Classify the findings. The lack of backup and an unambiguous coverage process is a critical staffing and workflow gap if recommendations may be generated during those absences. The reference-sheet update is a separate gap; it may be noncritical if required information is already available through an approved source and users can follow the current procedure.
Prioritize and specify closure. First, the service lead and clinic manager assign a primary and backup reviewer for every intended operating period, including an escalation route when neither is available. Closure requires an approved coverage schedule and confirmation from the named staff that they know the process. Next, the education lead updates the reference sheet by a stated date and verifies that it matches the approved local procedure.
Decision. The clinic is not ready for use during uncovered periods until the coverage gate is resolved. It may assess a narrower intended scope only if that scope and its operating limits are formally defined and consistently enforceable; an informal expectation that users will “find someone” is not evidence of coverage.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- A site has evidence for all technical requirements, but no role is assigned to review recommendations during evening coverage. Why is an overall readiness score insufficient to justify use?
- A required laboratory value is present in the EHR, but its result time is not available to the dosing workflow. What should the assessment record, and what would count as useful closure evidence?
- A readiness item is marked “unknown” because the assessor has not confirmed who handles downtime. Should it be counted as met, partially met, or unresolved? What is a suitable next action?
- Why should an action plan name one accountable owner, a dependency, and closure evidence rather than only a due date?
- A site is assessed as ready for weekday inpatient use. What could make that conclusion no longer apply?
Solutions.
1. The absence of an evening reviewer is a potential critical staffing and workflow gap if the intended use includes evening decisions. Technical readiness cannot supply clinical review responsibility, and a summary score may hide that deficiency. The site should remain not ready for that scope until the role and coverage are documented and verified, or until a narrower scope with enforceable limits is formally defined.
2. Record that the required value and its result time are not both available at the decision point, and classify the data requirement as unresolved against the stated scope. Useful closure evidence includes an approved source-field mapping and documented checks that the value and time arrive together, including checks for missing or unusable times.
3. It is unresolved, not met. The next action is to identify the downtime process owner, obtain the approved procedure, and confirm with relevant staff how it is used. Closure requires evidence that names the responsible roles and explains what users do when the service is unavailable.
4. A due date alone does not establish who will do the work, what must happen first, or how completion will be judged. A named owner creates accountability, a dependency makes sequencing visible, and closure evidence allows the assessor to verify that the requirement—not merely the task—has been met.
5. A change in setting, hours, staffing coverage, data pathway, workflow, or authorized scope could invalidate the original evidence. The site should reassess affected requirements when a material local assumption changes; weekday inpatient readiness does not automatically establish readiness for evenings or another care setting.