Every time someone deploys an AI agent, the conversation follows the same pattern. Define the role. Write the instructions. Add examples. Then comes the rules: what the agent can and cannot do.
Here’s the problem: those rules are not actually rules. They are suggestions. If your governance lives inside a prompt, you don’t have governance. You have a perhaps.
The Behavioral Trap
When we think about governing an AI agent, most people fall into what I call the behavioral trap. You tell your agent what to do and what not to do, but this doesn’t always work. Why? Simple. The system is probabilistic. Same prompt, different result. Every time.
What’s the alternative? Simple: limit what the agent can and cannot do. If the option doesn’t exist, the system can’t execute it. Even if it wants to.
To understand this, let me give you an example. Imagine you hire a new employee. You give them the company handbook. One of the pages says: never share confidential documents with external parties.
Two weeks later, an email goes out with confidential information. An honest mistake. But if your system simply doesn’t allow sending emails to external addresses, the problem never happens.
The problem is not the employee. Everyone can make errors. The problem is the system.
What This Looks Like in Practice
Imagine a customer service chatbot that receives hundreds of conversations a day. In the prompt, one simple rule: never offer refunds beyond company policy without human intervention.
One day, a customer pushes hard. And the agent gives a full refund anyway.
If the refund action is connected to a system that validates the policy before executing, this never happens.
Where do you start?
First, create a map of every action your agent can take. Not what it says. What it actually does: send emails, access databases, modify data.
For each action, ask one question: what happens if this goes wrong? If the answer is “Houston, we have a problem”, that action needs executable governance.
You don’t need to rebuild your system. But start now with the riskiest action.
One Last Thing
Remember, agents are non-deterministic systems. We can never be fully sure about their output. That’s why you need to be really careful about the actions they can take.
Limit them by system. And sleep at night.
Related reading: multi-agent or skill, which is better?