HUMAN-IN-THE-LOOP AGENT DESIGN: APPROVAL QUEUES THAT TEACH THE SYSTEM INSTEAD OF BOTTLENECKING IT
How to design HITL for AI agents: thresholds, queues, override trails, and using disagreements as evaluation signal - the pattern behind Agent Cloud approvals.
Human-in-the-loop is how you calibrate agent autonomy with evidence: consequential actions route to approval, humans decide, and the disagreement log becomes your best evaluation dataset. Done poorly, HITL is a ticket pile. Done well, it is how security says yes - see /products/agent-cloud.
Thresholds, not vibes
Start tight (amount limits, supplier allowlists, write scopes). Review the approval log monthly: widen where humans always agree, tighten where they don't. Version the policy so auditors can name what decided a March payment.
Queue design
- →Show evidence, not just the proposed action.
- →One-click approve/deny with required reason on deny.
- →SLA and escalation so queues don't silently rot.
Overrides are features
Every override should teach: store the reason, feed it into eval sets, and consider a policy patch. Agents without override trails cannot be trusted in regulated workflows.