The safety mindset: layers, not hope

Reliability is designed, not hoped for

An agent that works in a demo can still fail in the field: an input you did not imagine, a web page that changed, a message with hidden instructions, a model that drifted into an odd answer. The goal of safe operation is not to prevent every mistake. It is to make sure that mistakes are small, visible and reversible.

Defence in depth

No single control is enough, so you stack them. A mistake has to slip past all the layers to cause harm.

1. Scope. The agent has one narrow job and only the tools that job needs.
2. Access. The keys it uses reach only what is necessary.
3. Guardrails. Hard limits on spending, volume and destinations that the agent cannot argue its way past.
4. Approvals. A person confirms any action that is hard to undo.
5. Budgets. Cost has a ceiling per run and per month.
6. Visibility. Every step is recorded, so you can see and investigate.
7. Testing and versions. You check behaviour before it matters, and can go back when a change goes wrong.

Instructions are not controls

Telling an agent "never send email without asking" in its instructions is useful guidance, but it is not a control. A model can be tricked or can simply err. A control is something enforced outside the model: a tool that is not ticked, an approval the run cannot skip, a limit the platform applies. Whenever something matters, build a control, and use instructions as a second layer.

How this looks in PGB Agents

Each layer has a home on the agent page and in the top bar: Tools and PGB Business access (scope and access), Guardrails and Secrets (hard limits), Approvals (human gates), Budget (cost), Runs and Reports (visibility), and the Tests and Versions tabs (testing and rollback). The rest of this course goes through them.

Try it yourself

Pick an agent you built in the previous course. On paper, list every layer above and write what that agent has for each. Circle the weakest one. You will strengthen it as you go.

Recap

• Aim for mistakes that are small, visible and reversible.
• Stack layers: scope, access, guardrails, approvals, budgets, visibility, testing and versions.
• Instructions guide the model; controls enforced outside the model protect you.