# Human on the loop: open source supervision that changes the rule

> Human on the loop means a person supervises a running AI agent and can change the rule it works under; on open-source Kortix that rule is a file you own. Kortix gives a supervisor three levers: fix one outcome, change the rule, stop the run. Kortix is the open-source AI Operating System.

URL: https://humanintheloopvshumanontheloop.com/human-on-the-loop

## Risk mode as a project policy, with one exception on the argument

`kortix.yaml`

```yaml
policy:
  default_mode: risk

policies:
  - match: gmail.send_email
    action: always_run
    conditions:
      - arg: to
        match: "*@example.com"
  - match: gmail.send_email
    action: require_approval
```

Source: https://kortix.com/docs/connect/connectors

A project-wide rule runs before any connector rule, so a connector cannot loosen it.

## What human on the loop means, and the three levers it leaves a person

Human on the loop (HOTL) means the agent acts first and a person supervises the work while it runs, holding authority to step in. Torch.AI contrasts it with human in the loop, where "in the loop requires a person to approve each action before the system acts", and describes the on-the-loop model as one that "lets the system act on its own under active human supervision with the authority to intervene or stop it" ([Torch.AI](https://torch.ai/learn/human-in-the-loop-vs-human-on-the-loop)). Decagon's glossary puts it shorter: "the system acts on its own while a person watches and can step in, correct it or stop it" ([Decagon](https://decagon.ai/glossary/human-in-the-loop-vs-human-on-the-loop)).

Every definition stops at "can step in". The useful question is what the person can actually do, because three interventions carry very different weight:

- **Correct one outcome.** Approve or deny the call in front of you, or undo what shipped. The next identical call arrives with no memory of that decision.
- **Change the rule.** Edit the policy that decided the call, so the next run of the same action is held, blocked, or runs without asking.
- **Stop the run.** End the session at the runtime, which ends the turn in progress and everything it would have done next.

The first lever is the one dashboards hand you, and it leaves nothing behind. A team that only ever pulls that lever is auditing one result at a time. Supervision starts at the third lever and depends on the second: a person who can halt a session but cannot change the policy gets the same call again after lunch.

Decagon makes the point that decides most designs: the choice is made per action. A support project can run a knowledge answer on the loop and hold a refund above a threshold in the loop, and the two settings live in the same config.

## Four things supervision needs before the word fits

Torch.AI states the bar plainly: "On the loop only delivers real control if the supervisor can actually see what the system is doing and act in the time available." Seeing and acting in time are two requirements. Two more come from the record and from who holds the authority.

**1. A record of what ran.** The log needs the action, its arguments, the account it ran as, and when. Every action is recorded. Reading, exporting and streaming the audit log depend on the plan. An approver who sees that an email went out, without seeing the recipient, has a headline and no record.

**2. A way to see the ones that matter.** Decagon writes that "a feed of every action isn't supervision". The filter has to be the rule itself: what the policy allows runs quietly, and what it holds lands in one queue. Sampling and argument-based alerts work on top of that, and they cannot replace it.

**3. A lever that changes future runs.** An exception reviewed and released teaches the system nothing. The same exception then returns tomorrow, at the same volume, to the same person. Changing the rule is the only intervention that reduces the next queue.

**4. A named person with authority.** In Kortix a held call is decided by a project manager or by the person who started the session. A session started with an API token is decided by project managers only. Stopping a run needs `project.session.stop`, held by the sender, the session owner, or a project manager. Authority assigned to "the team" is authority nobody holds at night.

## Where on-the-loop supervision fails

Review fatigue does not disappear when the queue becomes a dashboard. It moves. Decagon describes the in-the-loop version: reviewers who approve a long run of correct proposals stop reading closely. The on-the-loop version is automation bias, where a supervisor who has watched a thousand correct actions is badly placed to catch the wrong one, and Decagon names that risk directly. A queue of pending approvals still forces reading. A wall of green rows trains the eye to skip.

The second failure is quieter. A supervisor who can undo outcomes and cannot change policy becomes a spectator with a login. n8n's comparison lists "setting guardrails" among four on-the-loop habits, next to random sampling and anomaly flags ([n8n](https://blog.n8n.io/human-in-the-loop-vs-human-on-the-loop/)). The lever is there, one bullet among four, and a team that reads the list as monitoring tasks will sample forever.

Third, the rules themselves decay. An argument exception written for one customer stays open after the contract ends. A `risk` default added for a pilot survives into the quarter when the agent sends invoices. Flowable argues the choice belongs "for every meaningful step in a workflow" rather than to the deployment as a whole, and the same holds for each rule inside a step ([Flowable](https://flowable.com/blog/business/human-in-the-loop-vs-human-on-the-loop)).

## When to run on the loop, and what stays gated

On the loop suits routine, high-volume work where a person can still undo the result: status replies, ticket tagging, triage, drafting. Approval on every one of those adds delay and little safety, which is why teams move them off the queue.

The boundary is reversibility. Decagon notes the model only protects actions that can be undone, which is the right test to apply before widening a rule. Irreversible calls, money moving, and anything that leaves the building keep an Ask or a Block, and the decision belongs on the action rather than on the project.

Kortix gives the boundary two concrete forms. `sensitive: true` on a connector holds every remaining action that no rule matched, reads included, which is what a mailbox or a payment account deserves. A project-wide rule beats a connector rule, so the owner of the gate writes it in `kortix.yaml` and no connector entry can loosen it.

One limit matters when you set this up. There is no session-wide approval: a call that nobody decides expires when the project's approval window runs out, 24 hours by default, and the session does not resume. An action that must run unattended therefore needs an explicit Allow rule. A team that gates everything and allow-lists nothing gets a stalled run rather than a supervised one.

## Setting the default in Kortix

Approval gates you set. Off until you set them. A new project starts with `allow_all` under `policy:`, which runs every action no rule matches, so the first step is the `risk` default in the `kortix.yaml` excerpt above. `risk` runs reads and holds writes and destructive actions for approval.

Set it from a terminal in one command:

```bash
kortix connectors policy set --default risk --apply
kortix connectors policy add "gmail.send_*" ask --condition to='*@example.com'
kortix connectors sensitive gmail on
```

The rule chain decides what wins. Project-wide `policies:` evaluate first, with fully qualified patterns such as `gmail.send_email`. The connector's own `policies:` come second, with patterns relative to the connector. `sensitive: true` comes third and holds everything still unmatched. `default_mode` comes last. Conditions make a rule about arguments instead of action names: `arg` is a dot path, `negate: true` inverts a test, a rule holds up to ten conditions and all of them must match. A missing argument never matches, so a negated condition on it holds, and a malformed condition fails closed. A typo never opens an action.

Two behaviors shape how the queue feels in practice. An approval covers one exact request, the same connector, action, arguments and account, so a changed recipient needs a second decision. A repeat of the identical call within fifteen minutes runs without a new decision.

Then keep the gate out of the agent's reach. Session work reaches `main` through a change request. Merge is default-deny for agents. A widened policy is a diff a person reviews before it lands. Review the rules on the same schedule as the work they hold, because an exception that outlived its reason is how an on-the-loop system drifts toward no loop at all.

## Questions a supervisor asks before turning the gate on

Short answers, each one standing on its own.

### What is human on the loop

Human on the loop means an AI agent acts first while a person supervises the running work and can correct an outcome, change the rule, or stop the run. Torch.AI describes it as acting under active supervision with the authority to intervene or stop it.

### How is human on the loop different from human in the loop

In the loop, a person approves each action before it runs, so nothing happens without a decision. On the loop, the action runs and a person supervises it. The choice is made per action, so one project can hold refunds and run status replies.

### What does a human on the loop actually monitor

A supervisor reviews the calls a policy held for approval, a sample of what ran, and alerts on unusual arguments or volumes. Decagon makes the point that a feed of every action is not supervision. The rule decides which actions need a person, so the policy is the real monitor.

### Is human on the loop safe for high-risk actions

Only for actions a person can still undo in time. Keep irreversible calls, payments, and anything that leaves the building on Ask or Block, where nothing runs until someone decides. On the loop suits routine, high-volume, reversible work.

### How do you set a default supervision mode in Kortix

Put `default_mode: risk` under `policy:` in `kortix.yaml`, or run `kortix connectors policy set --default risk --apply`. That runs reads and holds writes and destructive actions for approval. Gates are off until you set them, because the default is `allow_all`.

## Set the gate before the run does something you have to undo

Kortix is open source, on your keys, in the cloud or on your own network.

[Try Kortix](https://kortix.com) · [Human in the loop, side by side](https://humanintheloopvshumanontheloop.com/human-in-the-loop)