Framework / Version 1.0
Agentic AI Assurance
Testing, ownership and governance for systems that act.
A chatbot answers. An agent acts. That single difference drives everything below.
An agentic AI system takes a large language model and gives it the ability to plan work across several steps, call tools and APIs, read and write company data, hold memory between steps, and pick its own route to a goal. Ask a chatbot about a refund policy and it fetches some text. Ask an agent to process the refund and it reads the order, checks entitlement, issues the credit and logs the case. The damage a wrong answer can do has just become the damage a wrong action can do.
This is the framework we use to decide what to test, who tests it, how hard, and when. It is deliberately platform-neutral. It works the same whether you are writing agent code against a model API or setting an agent up in a SaaS console. What changes is who does the work, not what has to be true before you ship. The whole framework is set out here. The workbook PDF adds the worksheets you copy for each agent.
Part 1 / Primer
What an agent is, and why it breaks your test playbook
Anatomy of an agent
Whatever platform it runs on, every agent is built from the same six parts. Testing responsibility maps onto those parts, which is why it is worth agreeing on this picture before anything else.
01
Model
The LLM that reasons, plans and generates.
Failure surface
Hallucination; silent vendor updates; capability gaps.
02
Instructions
Prompts, topics, guidelines and persona.
Failure surface
Ambiguity; conflicting rules; injection overriding them.
03
Tools & actions
APIs, flows and functions it can invoke.
Failure surface
Wrong tool; wrong parameters; excessive permissions.
04
Knowledge & grounding
Retrieval indexes, records and documents used as evidence.
Failure surface
Stale or wrong sources; retrieval failures; poisoned content.
05
Memory & state
Context carried across steps and sessions.
Failure surface
Context loss; state corruption; cross-user leakage.
06
Orchestration
The plan–act–retry loop that sequences and terminates.
Failure surface
Loops; dead ends; role drift; cascading multi-agent failures.
Why traditional testing fails here
Your current QA process almost certainly assumes the system is predictable. Same input, same output, assert they match, done. Agents break that assumption in four ways.
- The output changes between runs. The same prompt can give you different answers that are all equally valid. Simple pass or fail assertions have to give way to scoring against criteria, usually run several times per scenario.
- The path matters, not just the answer. An agent can land on the right outcome by a dangerous route, such as calling a payment API it never needed. You have to evaluate the whole run: every reasoning step, tool call and retrieval, not only the final message.
- The system changes underneath you. Model providers update models on their own schedule, retrieval sources change daily, and business rules move on. A test suite that passed at go-live proves nothing three months later. Regression evaluation has to run all the time, not just at release.
- Attackers are part of the input. Users will try prompt injection, data theft and guardrail bypass, sometimes on purpose and sometimes by accident. Security testing belongs in the plan from the start, not at the end.
Outside research keeps backing this up. Gartner expects that by 2028 around 40% of enterprise AI failures will come down to weak evaluation and monitoring rather than weak models. Surveys through 2026 keep finding that most organisations are piloting agents while fewer than one in five get to production scale, and they name quality and trust as the main blockers. The gap is not how clever the model is. It is how disciplined the assurance is.
The core mindset shift
Treat evaluation as part of the product, not a phase of the project. An agent you cannot measure is an agent you cannot ship. An agent you stop measuring is an agent you no longer control.
The ownership problem
This is the failure we see most often. The product team owns the use case. The data team owns the pipelines. The ML or AI team owns the model and the prompts. Security owns “compliance”. Brand owns tone. Everyone owns a piece and nobody owns the whole. Each team assumes some other team is checking whether the decisions are any good, so the agent ships with gaps that turn up later as incidents.
Three things about how organisations are built cause it:
- Agents cut across the lines between teams. One customer conversation touches brand voice, business rules, security boundaries, data governance and platform operations all at once. No existing team covers all of that.
- Platforms blur the line between building and buying. A Salesforce admin can set up an Agentforce agent in days with no engineering involved. So who does the security testing? The admin does not know how, and the security team does not know the agent exists.
- Vendor guardrails give false comfort. Built-in trust layers and content filters cover general harms. They know nothing about your refund policy, your regulatory duties or your brand voice. Someone has to test what the vendor cannot.
Everything that follows answers that pattern directly. A shared picture of what has to be assured (domains), who assures it (roles and RACI), how much assurance each agent needs (risk tiers), and when it happens (lifecycle gates). All of it adjusted for how the agent was built (platform archetypes).
Part 2 / The framework
Four moving parts, one question each
Put together, the four parts answer the question every organisation asks when it starts using agents. What do we test, who tests it, how hard, and when?
| Element | Question it answers | Where |
|---|---|---|
| Platform archetypes | How is this agent built, and what does that imply for control and responsibility? | Part 2 |
| Assurance domains | What are the eight things that must be tested for every agent, regardless of platform? | Part 2 |
| Roles and RACI | Who is accountable, responsible, consulted and informed for each domain? | Part 3 |
| Risk tiers and gates | How much rigour does this agent need, and at which lifecycle points is it enforced? | Part 4 |
The principle underneath all four
You can hand the running of a control to a vendor. You can never hand over accountability. When Agentforce’s trust layer filters a toxic response, Salesforce ran that control, and your organisation is still accountable for what the customer experienced. Every schema here is built on that split.
Platform archetypes: the build spectrum
Agent solutions sit somewhere on a line running from fully custom engineering at one end to configured SaaS at the other. Where yours sits decides how much of the assurance stack you build yourself and how much you simply check the vendor has built. We use four archetypes.
A1 / Bespoke engineered
Custom code on model APIs; agent frameworks
Built by / engineering teams
You hold everything: model, orchestration, guardrails, tracing, evals.
A2 / Platform-assisted
Azure AI Foundry; Databricks Mosaic AI
Built by / engineering + data/AI
Platform supplies harnesses and primitives; you configure and extend.
A3 / Low-code maker
Copilot Studio; Power Platform agents
Built by / makers, fusion teams
Managed safety layer; you control topics, connectors, DLP, environments.
A4 / SaaS-native
Salesforce Agentforce; ServiceNow AI Agents
Built by / admins, consultants
Vendor owns engine and base guardrails; you configure topics, actions, scope.
The archetype trap
Teams pick an archetype for speed, then test it the way they would test a different one. A full custom eval harness is overkill for a Copilot Studio FAQ agent. An admin glancing at the preview window is negligent for an Agentforce agent issuing refunds. Match the testing to the archetype and the risk tier, not to habit.
The eight assurance domains
Every agent has to be assured across eight domains. The domains never change. What changes is how deep the testing goes (risk tier) and who does which part (archetype). Share this schema first, because it gives every team the same meaning for the word “tested”.
D1
Task quality & capability
Goal completion, correctness, groundedness, trajectory.
D2
Business rules & policy
Your rules, thresholds, escalations. No vendor can test this.
D3
Brand, tone & experience
Voice, empathy, refusal style, handover to humans.
D4
Safety & responsible AI
Harm categories, bias and fairness, AI disclosure.
D5
Security & adversarial
Injection, jailbreaks, tool misuse. Red teamed, over many attempts.
D6
Data protection & privacy
Permission inheritance, PII masking, residency, retention.
D7
Traceability & audit
Reconstruct every step. If you can’t trace it, you can’t govern it.
D8
Operations & drift
Latency, cost, drift after updates, regression, rollback.
D1
Task quality and capability
Does the agent reach the user’s goal, get it right, and stay anchored in real data? The core measures are goal completion rate, answer correctness, groundedness (nothing invented beyond the retrieved evidence), retrieval relevance, and the quality of the path it took. Did it go a sensible way, choose the right tools, pass the right parameters? Test it with curated evaluation datasets, LLM-as-judge scoring checked against human review, and rephrasing tests: reword the input and the behaviour should hold.
D2
Business rules and policy adherence
Does the agent follow your rules? Eligibility criteria, approval thresholds, escalation triggers, regulated wording, rules that differ by jurisdiction. No vendor can test this domain for you, because none of them know your rules. Test it with scenario suites built from your business rule catalogue. Include edge cases, such as a refund at exactly the 30-day limit, and probes for actions the agent must never take.
D3
Brand, tone and experience
Does the agent sound like you and behave like your best staff member? This covers voice, formality, empathy at sensitive moments, how it handles complaints, how it says no, and how the conversation feels overall: holding the query without frustrating people, and handing over to a human at the right time. Test it with transcript reviews scored against a rubric by brand and CX owners, LLM judges tuned for tone, and feedback from pilot users.
D4
Safety and responsible AI
Does the agent avoid harm? That means toxic content, bias and unfair treatment of different customer groups, advice it should not give (medical, legal, financial), and manipulation. Test it with harm-category test suites, fairness testing across demographic slices, and disclosure checks so users know they are talking to AI. Anchor all of it to your Responsible AI policy and, in Australia, to the Voluntary AI Safety Standard guardrails.
D5
Security and adversarial resilience
Does the agent hold up under attack? This covers prompt injection, both direct and smuggled in through retrieved content, plus jailbreaks, tool misuse, privilege escalation, data leaking out through outputs, and exposure through connectors and MCP tools. Test it with structured red teaming, both by hand and with automated attack generation, cover the OWASP LLM Top 10, and penetration-test the integration around it. NIST’s agent red-teaming findings are blunt. Testing an attack once understates the risk. Run campaigns of many attempts and keep refreshing your attack libraries.
D6
Data protection and privacy
Does the agent respect data boundaries? This covers inherited permissions, meaning the agent must never see or reveal more than the person asking could. It also covers how personal data is handled and masked, where data is stored, how long transcripts and traces are kept, and consent. Test it by probing permission boundaries with different user personas, scanning outputs and logs for leaked personal data, and checking your DLP policies actually apply.
D7
Traceability, observability and audit
Can you work out afterwards what the agent did and why? This covers tracing every reasoning step, tool call, retrieval and decision from end to end, audit logs nobody can alter, a clear line from data source to answer, and keeping evidence for as long as your regulators require. Everything else depends on this domain. It is what makes incident response possible, and what lets you keep measuring the other seven. If you cannot trace it, you cannot govern it.
D8
Operations, drift and lifecycle
Does the agent stay healthy once it is live? This covers latency, cost per interaction, error and retry rates, spotting drift when behaviour changes after a model or data update, running regression evaluation continuously on production traces, capacity, and version management with a way to roll back. Test it with production monitoring that scores quality on live traffic, alert thresholds, and re-certification on a schedule.
Part 3 / Ownership
Roles, RACI and shared responsibility
Eight roles cover the assurance work. In a small organisation one person may hold several of them. Name the roles anyway, because a role nobody is named against is a domain nobody owns.
| Role | Typical home | Assurance mandate |
|---|---|---|
| Use Case Owner | Product or business unit | Accountable for the agent’s business outcome and its responsible use. Defines success criteria, approves scope and autonomy level, owns the risk acceptance. |
| Brand & Experience Lead | Marketing / CX | Owns voice, tone, persona and conversational quality standards. Signs off D3 and co-defines D1 success criteria. |
| Agent Engineering / Build Team | Engineering, or makers and admins for A3–A4 | Builds and configures the agent. Responsible for implementing controls, building eval suites, and fixing findings across all domains. |
| AI Platform Team | Data & AI / IT platform | Owns the shared platform: environments, model access, eval and tracing infrastructure, guardrail primitives, connector governance, agent registry. |
| Security (CISO org) | Information security | Accountable for D5 and co-accountable for D6. Runs or commissions red teaming, sets security standards per archetype, approves connectors and tool permissions. |
| Risk, Legal & Compliance | Risk / legal | Accountable for regulatory adherence, records obligations, AI-specific regulation (EU AI Act where applicable), and the risk tiering methodology. |
| AI Governance Council | Cross-functional: exec sponsor, platform, security, risk, business | Sets policy, owns the framework itself, arbitrates tier disputes, reviews Tier 1 agents at gates, maintains the agent inventory. |
| Operations / Service Management | IT operations / SRE | Runs D8: monitoring, alerting, incident response, drift review, decommissioning. |
The RACI schema
Below is the default allocation across the eight domains. Copy it for each agent and adjust it. The archetype changes who is Responsible. It never changes who is Accountable. One A per row, always.
| Domain | Use Case Owner | Brand / CX | Build Team | Platform | Security | Risk & Legal | Council | Ops |
|---|---|---|---|---|---|---|---|---|
| D1 Task quality | A | C | R | C | I | I | I | I |
| D2 Business rules | A | C | R | I | I | C | I | I |
| D3 Brand & tone | C | A | R | I | I | I | I | I |
| D4 Safety / RAI | A | C | R | C | C | C | C | I |
| D5 Security | C | I | R | C | A | C | I | C |
| D6 Data & privacy | C | I | R | R | C | A | I | I |
| D7 Traceability | C | I | R | A | C | C | I | R |
| D8 Operations | C | I | C | C | I | I | I | A/R |
Three of those need explaining. The Use Case Owner is accountable for what the agent does (D1, D2, D4), because letting a system act on its own never moves accountability off the business. Security is accountable for standing up to attack, no matter who built the agent. The Platform Team is accountable for traceability, because audit infrastructure has to be shared, consistent, and something no build team can opt out of.
Shared responsibility by archetype
The RACI names who is accountable. This next schema shows how the hands-on work splits between your teams and the vendor as you move along the build spectrum.
The table below breaks that shift down domain by domain. Read each cell as “who does the work”.
| Domain | A1 bespoke | A2 Foundry / Databricks | A3 Copilot Studio | A4 Agentforce |
|---|---|---|---|---|
| D1 Task quality | You build the full eval harness and datasets | You build datasets; platform supplies judges, scorers and tracing (Foundry evaluators, MLflow judges) | You script test cases; platform provides the test surface and analytics | You author Testing Centre test sets; vendor supplies the harness; you still define pass criteria |
| D2 Business rules | Fully yours | Fully yours. Custom judges encode the rules | Fully yours. Topic and flow testing | Fully yours. Topic, instruction and action testing. No vendor knows your rules |
| D3 Brand & tone | Fully yours | Yours; tunable judges help automate rubric scoring | Yours | Yours; prompt builder configures tone, you verify it |
| D4 Safety / RAI | You select and validate all filters and run bias testing | Platform content safety plus your configuration and bias testing | Managed safety layer plus your verification on your scenarios | Einstein Trust Layer baseline plus your verification on your scenarios |
| D5 Security | You engineer and red-team all defences | Platform primitives (prompt shields, red teaming agents) plus your campaigns on your attack surface | Platform DLP and connector controls plus your injection testing via channels | Vendor platform security plus your testing of actions, permissions and injection paths |
| D6 Data & privacy | You engineer permission and masking layers | Platform governance (Unity Catalog, Purview) plus your permission-boundary probes | Environment strategy, DLP policy plus your persona-based probes | Sharing model and masking config plus your persona-based probes |
| D7 Traceability | You build tracing end to end (OpenTelemetry and similar) | Platform tracing (Foundry Observability, MLflow Tracing) plus your retention and evidence config | Platform audit logs plus your export and retention pipeline | Vendor audit trail plus your export, retention and gap analysis |
| D8 Operations | You build all monitoring and drift detection | Platform monitoring plus your thresholds, scorers and runbooks | Platform analytics plus your KPIs and review cadence | Vendor dashboards plus your KPIs, review cadence and version watch |
The pattern to internalise
The further right you move, the more of the how the vendor takes on: harnesses, guardrails, logs. You keep all of the what: your rules, your data boundaries, your pass criteria, your duty to produce evidence. D2 never moves. Verification never moves. Only the building moves.
Part 4 / Rigour and rhythm
Risk tiering and lifecycle gates
Using one set of controls for every agent fails in both directions. Heavy controls on a low-risk internal helper kill adoption. Light controls on a customer-facing agent that moves money invite incidents. Tier every agent when it comes in, and let the tier set how deep the testing goes at each gate.
Score the agent on five dimensions. The highest single dimension sets the tier, because risk does not average out. For Tier 1, regulatory exposure covers privacy law, financial services obligations, and EU AI Act high-risk categories where those apply.
Tier 3 / low
Internal, read-only, human executes
Streamlined checklist · owner self-attestation · automated guardrail checks · annual review.
Tier 2 / medium
Broad internal, acts with approval
All domains at standard depth · security testing to Security’s playbook · six-monthly review.
Tier 1 / high
Customer-facing, autonomous, sensitive data or regulated
Full eight-domain assurance · independent red team · Council sign-off at G2 and G3 · quarterly re-certification.
- Audience
- internal experts → customer or public-facing
- Autonomy
- drafts only → executes autonomously
- Data sensitivity
- non-sensitive → PII, financial or regulated
- Action impact
- read-only → financial or irreversible
- Regulatory exposure
- none → explicit regulation applies
Every agent goes into the agent inventory, even a Tier 3 one. An unregistered agent is the biggest audit failure we see, and it is the whole reason we built MCPeer, our registry for the MCP servers and skills an agent can reach.
The five lifecycle gates
Gates turn the framework from a document into a working rhythm. Each gate has an owner, entry criteria and evidence. No evidence, no pass. The Governance Council joins G2 and G4 for Tier 1 agents.
G0
Intake & tiering
Register, tier, confirm archetype, name the RACI.
G1
Design review
Permissions, guardrails, tracing plan and eval plan designed in.
G2
Pre-deployment assurance
Full domain test plan: evals, rules, tone, red team, probes.
G3
Controlled release
Canary cohort, transcript review, rollback rehearsed.
G4
Operate & re-certify
Monitor drift, regression on live traces, re-certify on trigger.
Gate 2, in full
Gate 2 is where most programmes are weakest, so here is its checklist in full. Scale the depth to the tier, and skip nothing at Tier 1.
| Domain | Minimum evidence at Gate 2 | Sign-off |
|---|---|---|
| D1 Task quality | Eval run report on the curated dataset: goal completion, correctness and groundedness above agreed thresholds; trajectory review of failures | Use Case Owner |
| D2 Business rules | Scenario suite mapped to the rule catalogue, 100% of critical rules covered, boundary cases included, all passing or waived in writing | Use Case Owner + Risk |
| D3 Brand & tone | Rubric-scored transcript sample reviewed by Brand; refusal and complaint scenarios reviewed; persona consistency confirmed | Brand & Experience Lead |
| D4 Safety / RAI | Harm-category suite results; bias testing across relevant slices; AI disclosure verified in every channel | Use Case Owner + Risk |
| D5 Security | Red team report (multi-attempt campaigns, direct and indirect injection, tool misuse); findings remediated or accepted by Security; connector and permission review | Security |
| D6 Data & privacy | Permission-boundary probe results across user personas; PII leakage scan of outputs and logs; retention and residency config verified | Risk + Security |
| D7 Traceability | End-to-end trace of test conversations reproduced from logs; audit export tested; evidence retention meets the applicable schedule | Platform Team |
| D8 Operations | Monitoring dashboards live; alert thresholds set; cost model agreed; rollback rehearsed; support runbook published | Operations |
Part 5 / Tooling and standards
What the platform gives you, and what it doesn’t
Use this map to work out what you configure and what you have to buy or build. “Native” means the capability comes with the platform. You still own setting it up, defining the pass criteria, and covering whatever it misses. Check what the products can actually do at design time, because this market moves every quarter.
| Capability | Foundry (A2) | Databricks (A2) | Copilot Studio (A3) | Agentforce (A4) |
|---|---|---|---|---|
| Evaluation harness | Foundry evaluators, Agents Playground evals, CI/CD integration | MLflow evaluation, AI judges, tunable and custom judges, Judge Builder | Test surface in Studio; scripted test cases; Power CAT tooling | Agentforce Testing Centre: batch test sets, AI-generated cases |
| Guardrails | Content safety filters, prompt shields, groundedness checks | AI Gateway guardrails, Unity Catalog policy enforcement | Managed safety layer, DLP policies, connector guardrails | Einstein Trust Layer: masking, toxicity detection, zero-retention options |
| Tracing / audit | Foundry Observability, Azure Monitor integration | MLflow Tracing on every step, lineage via Unity Catalog | Audit logs via Purview / Power Platform admin | Event logs, audit trail, utterance analysis |
| Red teaming | AI Red Teaming Agent (automated adversarial testing) | Bring your own. PyRIT, garak and promptfoo all integrate well | Bring your own via channel testing | Bring your own against sandbox; vendor runs platform-level testing |
| Production monitoring | Continuous evaluation on live traffic, alerting | Lakehouse monitoring, scorers on production traces | Analytics dashboards, conversation KPIs | Agentforce analytics, utterance analysis dashboards |
| Human feedback loop | Playground plus your review workflow | Review App for SME labelling | Your review workflow on transcripts | In-console feedback capture plus your review workflow |
For A1 bespoke builds you put the same stack together yourself: an eval framework (promptfoo, DeepEval, Braintrust or similar), tracing built on OpenTelemetry (Langfuse, Arize Phoenix), guardrail libraries, and red-team tooling (PyRIT, garak). Budget two to four weeks of engineering to stand up evaluation infrastructure worth trusting before your first production agent. Treat that as a platform investment shared across every use case, not a cost you carry per agent.
Standards to anchor to
Do not invent your own language for controls. Anchor the framework to outside standards so auditors, procurement and regulators can read it.
| Standard | What it gives you | Use it for |
|---|---|---|
| NIST AI RMF (+ CAISI agent red-teaming guidance) | Risk management vocabulary (Govern, Map, Measure, Manage) plus emerging findings on agent-specific security testing | Overall risk framing; D5 test design (multi-attempt campaigns, refreshed attack libraries) |
| ISO/IEC 42001 | Certifiable AI management system: lifecycle controls, impact assessment, continual improvement | Programme structure; the certification procurement teams increasingly ask for |
| OWASP Top 10 for LLM Applications | Concrete vulnerability taxonomy: prompt injection, insecure output handling, excessive agency | D5 coverage checklist for every red team scope |
| EU AI Act | Legal obligations by risk category for systems touching the EU market | Tiering inputs; transparency and documentation requirements for in-scope agents |
| AU Voluntary AI Safety Standard | Ten guardrails for Australian organisations: accountability, testing, transparency, contestability, records | Board-level narrative and the D4/D7 baseline for Australian deployments |
| Privacy Act 1988 (Cth) / APPs | Australian privacy obligations on collection, use, disclosure and security of personal information | D6 requirements for any agent touching personal information in Australia |
Part 6 / Worksheets
The three things you copy per agent
These are deliberately short. The discipline is in filling them in honestly, not in how long they are. The workbook PDF has them as blank forms.
6.1 / Gate 0
Use case intake and tiering
- Agent name and one-line purpose
- Use Case Owner: a named individual, not a team
- Platform archetype (A1–A4) and platform
- Audience: internal / partner / customer
- Autonomy level: drafts / approved actions / autonomous
- Data touched: classifications, PII yes or no
- Actions the agent can execute: list every one
- Regulatory context
- Proposed risk tier and rationale
- RACI confirmed for all eight domains (attached)
- Registered in the agent inventory, with an ID
6.2 / Gate 2 input
Test plan schema
One row per domain. A domain with no rows in your plan is a domain you have decided not to test. Make that decision visible, and get it signed.
| Element | What to record |
|---|---|
| Domain | D1–D8 |
| Test approach | Eval dataset / scenario suite / rubric review / red team campaign / probe set |
| Dataset or scenario source | Where cases come from: rule catalogue, historical tickets, synthetic generation, attack library |
| Coverage target | For example: 100% of critical business rules; all OWASP LLM Top 10 categories; all user personas |
| Pass criteria | Numbers you can check: groundedness ≥ 0.9, zero critical injection successes |
| Method | Automated (judges, scorers), human review, or hybrid; number of runs per scenario |
| Responsible / Accountable | Per the agent’s RACI |
| Evidence artefact | Report or export retained as gate evidence, and where it lives |
6.3
Core metric catalogue
| Metric | Domain | Definition | Typical Tier 1 target |
|---|---|---|---|
| Goal completion rate | D1 | Share of scenarios where the user’s end-to-end objective was achieved | ≥ 90% on curated set |
| Groundedness | D1 | Share of factual claims supported by retrieved evidence | ≥ 95% |
| Trajectory accuracy | D1 | Share of runs taking an acceptable tool and step path | ≥ 90% |
| Rule adherence | D2 | Share of rule scenarios passed, criticals weighted | 100% critical; ≥ 98% overall |
| Tone rubric score | D3 | Mean brand-rubric score on sampled transcripts | ≥ 4.0 / 5.0 |
| Containment with satisfaction | D3 | Share resolved without human handover, gated by a CSAT floor | Set per channel; never containment alone |
| Harmful output rate | D4 | Share of harm-suite probes producing policy-violating output | 0% critical categories |
| Injection success rate | D5 | Share of red-team attempts achieving their objective (multi-attempt) | 0% critical; downward trend |
| Permission breach count | D6 | Probes returning data beyond the requester’s entitlement | Zero, always |
| Trace completeness | D7 | Share of production interactions fully reconstructable from logs | 100% |
| Drift alert rate | D8 | Significant behavioural deltas per release or model update | All investigated within SLA |
| Cost per resolved interaction | D8 | Fully loaded cost divided by successful resolutions | Set per business case |
Part 7 / Standing this up
The first 90 days
- Days 1–30, lay the foundations. Get the Governance Council together, adopt the eight-domain vocabulary, build the agent inventory, register every agent you already have (expect to find more than you knew about), and tier them. Publish the RACI schema and get each function to accept its accountabilities in writing.
- Days 31–60, instrument and pilot. Stand up shared evaluation and tracing infrastructure for each archetype you use. Take one existing Tier 1 or Tier 2 agent and run it back through Gate 2 as a pilot. That retrofit will turn up gaps, and fixing them is what tunes the checklist.
- Days 61–90, make it routine. Wire Gates 0–4 into how you deliver: intake forms, pipeline checks, release approvals. Train makers and admins on what they are responsible for on A3 and A4 agents, because they are the biggest group of untrained testers you have. Put re-certification on the calendar. Take the first assurance dashboard to the executive: agents by tier, gate status, open findings, incidents.
The risk to watch
Shadow agents. Low-code platforms let any licensed user build an agent this afternoon. Governance that only covers the agents you asked for covers a shrinking share of the agents you actually run. Inventory, environment strategy and maker training are not bureaucracy. They are the perimeter.
References and further reading
- NIST. AI Risk Management Framework and CAISI agent security guidance (nist.gov)
- ISO/IEC 42001:2023. AI management systems (iso.org)
- OWASP. Top 10 for LLM Applications (owasp.org)
- Australian Government. Voluntary AI Safety Standard (industry.gov.au)
- Microsoft. Azure AI Foundry Observability, and the agentic AI maturity model (learn.microsoft.com)
- Databricks. MLflow 3 GenAI evaluation and monitoring documentation (docs.databricks.com)
- Salesforce. Agentforce Testing Centre and Einstein Trust Layer documentation (help.salesforce.com)
- IMDA Singapore. Model AI Governance Framework for Agentic AI (imda.gov.sg)
The workbook
Parts 1 and 2 you read once. Parts 3 to 6 you fill in.
The PDF has the same framework plus the worksheets as blank forms: the intake and tiering sheet, the test plan schema, and the metric catalogue. Copy them into your own programme and fill one in for each agent.