Human in the loop: an open source approval bound to one call

The definition, and the stop it asks for inside an agent

Human in the loop (HITL) is an operating model in which a person decides before an automated action runs. Published definitions of the term agree on that much. Elementum calls HITL "an AI operating model in which a human must review, approve, or act on AI output before any automated system proceeds with execution" (Elementum). Torch.AI compresses it to "a person to approve each action before the system acts" (Torch.AI). n8n names the timing: human in the loop is "a synchronous control pattern. The workflow stops at a decision gate until a human provides a required signal" (n8n).

Every one of those definitions borrows a control-loop picture: a process that runs, measures what it produced and corrects. The person sits inside that loop. That picture explains why HITL is easy to define and hard to build. A loop corrects only when the correction relates to what the loop just did.

The distinction is decided per action, not per system. One agent can hold a refund for approval and let a read run unattended. Most of the argument between in-the-loop and on-the-loop oversight settles once the action is named.

For an agent, human in the loop means the run stops and a person decides before anything leaves the system. The agent does not act first and wait for the review. Flowable describes the stop: "a person must make a decision before the workflow can continue" (Flowable). Decagon describes the unit: "a person approves each decision before it takes effect" (Decagon).

The shape holds five steps in every implementation. The agent proposes an action, the workflow pauses, a person sees the proposal, the person decides, and the agent resumes on that decision. Steps three and four are where implementations part company. Definitions say a person approves. They rarely say what that person is shown, or what one approval covers.

Kortix answers both in the record of a held call. A held call returns 202 with status: "pending_approval", an approval_url, an approval_summary and an execution_id, and nothing has run at that point. The approver opens the call and sees the arguments the provider will receive. One approval covers one exact request: the same connector, action, arguments and account. Change an argument and the call needs a new approval. After an approval, the identical call runs again without a second decision if it repeats within 15 minutes. A call nobody decides expires, the session does not resume, and the agent asks again with a new call.

The delay is real, and the fatigue is the expensive failure

An approval queue slows a workflow because every held call waits for a person allowed to decide it. That delay is real and predictable. The expensive failure is the other one, and Decagon names it. "Human-in-the-loop fails through fatigue", where reviewers who approve a long run of correct proposals stop reading closely (Decagon).

Fatigue is an outcome of what the prompt shows. A prompt reading "the agent wants to use Gmail" teaches a reviewer to click. A prompt naming the recipient, the subject and the account keeps the reviewer reading, because the thing under judgment is on the screen. An approval bound to one exact call can be refused. A blanket approval that covers a class of future calls has decided them in advance. A queue of vague approvals costs time and buys nothing.

Kortix binds the decision at the point where the call is composed. A rule can test the call's arguments, with arg as a dot path into them (message.channel) and match for the pattern, and negate: true inverts a condition. A rule holds up to 10 conditions and all must match. A missing argument never matches the pattern. A malformed condition fails closed, so a typo narrows the gate instead of opening it. There is no session-wide approval either. An action that must run unattended needs an Allow rule, and the decision moves to whoever writes that rule.

The cost is reviewer throughput

The honest cost of in-the-loop oversight is reviewer throughput. Each held call consumes a decision from someone allowed to make it. The ceiling on the workflow is then the number of decisions those people can take, not the number of calls the agent can make. That is the reason to reserve the gate rather than fit it everywhere.

In Kortix the decider is a project manager or the person who started the session. A call made in a session started with an API token is decided by project managers only. Those people form one queue, and a busy connector makes it a long one.

Reserve approval for the calls where a wrong yes is expensive: money leaving an account, a message to someone outside the company, a delete. Let reads run. Argument conditions settle most of the routine volume without a reviewer. That is what keeps the queue short enough for people to keep reading it. The approval workflow page takes the policy apart line by line, and the guide to choosing covers which action goes behind which setting.

The approver, the queue, and the order rules are read in

Actions are always_run, require_approval, block, shown as Allow, Ask, Block. Approval holds the call; the agent's turn pauses and resumes.

Kortix evaluates each call in a fixed order, and the first match wins. Project-wide policies: in kortix.yaml come first, then the connector's own policies:, then sensitive: true on the connector, then policy.default_mode. Setting sensitive: true holds every remaining action on that connector, reads included, which suits a connector where any call is consequential.

Approval gates you set. Off until you set them. policy.default_mode falls back to allow_all, which runs every action no rule matches, while risk runs reads and holds writes and destructive calls.

A person decides a held call in the web app under Review → Needs you, at the approval_url, or on the approval card in a Slack or Teams thread. A call nobody decides expires after 24 hours, or after the project's approvals.ttl. The approval_url expires at the same time.

Code takes the same shape. Session work reaches main through a change request. Merge is default-deny for agents. A change to the policy itself therefore meets a person before it governs anyone.