Skip to content
Leone Intelligence Systems

Insights

The Production AI Brief.

Practical notes from production-agent work: evaluation, GraphRAG, approvals, and the handover problems prototypes hide.

Controls

Design approval controls before tools leave the sandbox

External actions need a visible review point, bounded permissions, a stop path, and a record of who approved the action.

5 min

Approval is a product surface: operators need enough context to decide, enough control to stop, and a record that survives handover.

Classify the actions

Separate read-only suggestions from actions that change records, contact people, move money, or alter a live system. The control should match the consequence.

  • List every tool action and its external consequence
  • Set the permission and approval level for each action
  • Keep high-consequence actions disabled until the gate is explicit

Make review usable

A reviewer cannot approve safely from a button alone. Show the proposed action, source context, policy result, affected target, and edit or reject options in one place.

  • Show the proposed change and its target
  • Expose sources, policy checks, and missing information
  • Offer approve, edit, reject, and escalate outcomes

Recover and learn

The control path must include what happens after a rejection, timeout, partial action, or incident. Keep the trace useful for the next operator and the next release.

  • Define stop, retry, rollback, and incident ownership
  • Record the reviewer decision and system state
  • Feed recurring failures into the improvement backlog