Agentic AI in a Hospital: Where an Agent May Act, and Where It May Only Ask

The vocabulary changed some time in the last eighteen months. Hospital software proposals that used to promise AI-assisted this and AI-powered that now promise agents: software that does not merely suggest, but plans a sequence of steps, calls systems, and completes work without a person in the loop for each step.

The distinction is real and it is not marketing. A model that drafts a discharge summary for a doctor to sign is a tool. Software that reads a rejected claim, decides which document is missing, retrieves it, re-files the claim and updates the receivables ledger is something else, and it needs a different set of questions asked of it. Most hospitals are being sold the first thing under the second thing's name, which is confusing but harmless. The hospitals that are genuinely being sold the second thing are the ones who need to think carefully.

What makes something an agent rather than a feature?

An agent decides on a sequence of actions and executes them against real systems; a feature produces an output that a person then acts on. The dividing line is whether anything changes in the hospital's records without a human deciding that it should.

By that test, ambient documentation is not an agent — it drafts, a clinician signs. An alert is not an agent. A model that predicts discharge readiness is not an agent. But software that autonomously re-orders stock when a reorder point is breached is an agent, and hospitals have had those for years without calling them that. So has anything that auto-posts a payment, auto-assigns a bed, or auto-cancels an unconfirmed appointment.

Recognising that is useful because it deflates the novelty and sharpens the actual question. Hospitals already delegate actions to software. What is new in 2026 is the breadth of what can be delegated and the fact that the decision procedure is now a language model whose reasoning is neither fixed nor fully inspectable. The governance question is not "should we allow agents", which was answered years ago by every auto-reorder rule in your pharmacy. It is "which actions, under what authority, reversible how".

Which agent use cases are actually working?

The ones where the action is reversible, the domain is administrative rather than clinical, and there is a natural verification step downstream.

Where deployments are demonstrably delivering value:

• Documentation drafting, which is the most widely adopted healthcare AI use case anywhere and remains a draft-and-sign workflow
• Claim status chasing and denial triage, where the agent classifies a rejection and assembles the response pack for a human to submit
• Prior authorisation preparation, gathering the clinical evidence a payer will ask for before anyone asks
• Coding suggestion against the clinical record, reviewed by a coder
• Patient communication — appointment confirmation, preparation instructions, follow-up reminders — within scripted bounds
• Scheduling optimisation, proposing a theatre list or a roster that a human approves

Notice the shape. In almost every entry, the agent does the assembly and a person does the commitment. That is not timidity; it is where the economics currently are. The assembly is the expensive, tedious, error-prone part, and it is the part where being wrong costs a re-run rather than a patient.

Our broader AI article covers what is real across the wider category, and hospital AI governance covers the control framework.

Where must an agent not act autonomously?

Anywhere the action is clinical, irreversible, or financially final without a person's name attached to it.

The list is short and should be written down before any pilot, not after one. No autonomous change to a medication order, a dose, a diagnosis or an allergy record. No autonomous discharge decision or triage disposition. No autonomous release of a diagnostic report. No autonomous refund, write-off, credit note or price override. No autonomous disclosure of a patient record to any external party — that one is a consent question under the DPDP Act before it is an AI question. And no autonomous deletion of anything, ever.

Two design rules make the rest safe. Every agent action must be attributable — the audit trail records that this agent, on this version, acting under this delegated authority, did this thing, and a named human owns that delegation. And every agent action must be reversible by a person who did not need to understand the agent to reverse it. If reversing a mistake requires a support ticket to the vendor, the action should not have been delegated.

How should a hospital pilot one?

On one workflow with a measurable baseline, with the agent's output compared against human output for a defined period before anything is switched off.

The failure pattern is recognisable. A pilot starts without a baseline, so at the end nobody can say whether the agent helped. It runs on a workflow chosen because it demos well rather than because it hurts. Success is reported as "staff liked it", which is a real signal but not a business case. And the agent is left running with permissions granted for the pilot that nobody revisits.

A better shape: pick a workflow where you already measure cycle time — claim first-pass rate, denial turnaround, documentation lag — because those numbers are the only honest scoreboard. Run the agent in shadow mode first, producing output that nobody acts on, and compare. Then run it with human confirmation on every action. Only then consider narrowing the confirmation to exceptions, and only for the action classes on the permitted list.

What should be in the contract and the audit trail?

Model disclosure, data boundaries, action logs, and the right to turn it off without losing the workflow.

Ask where inference happens and whether patient data leaves your jurisdiction — the answer determines whether this is a procurement decision or a data residency decision. Ask whether your data trains anyone's model, and get the answer in the agreement rather than in an email. Require that agent actions are separable in the audit trail, so you can produce every action an agent took in a period without filtering by hand. Require a kill switch that degrades to the manual workflow rather than to nothing. And require notice before a model version changes, because a system whose behaviour changes silently cannot be validated.

Where Kōami fits

Kōami's position is that an agent is only as good as the record it acts on, and that the hospital rather than the vendor should hold the delegation. Structured clinical data, an audit trail that records the actor and the authority for every write, and role-based permissions that apply to automated actors the same way they apply to people — those are the prerequisites, and they are worth having whether or not you ever switch an agent on.

Where we are deliberately conservative is the permitted-action list above. We would rather ship assembly-and-approve workflows that a hospital can audit than autonomous clinical or financial actions that demo impressively and fail quietly. If you are evaluating agentic claims from anyone, bring the question that settles it to a demo: show me the audit trail for one agent action, and show me how a ward sister reverses it.