Table of Contents
The Enterprise Agent Reference Architecture, Annotated
The vendor diagram puts one box in the middle labeled "Agent," draws arrows at your APIs, and footnotes governance as "coming soon." That picture is fine for a demo and useless for a platform team shipping to 200 seats. It hides every layer that causes a production incident and every decision you own instead of the vendor.
Here is the same system drawn from where you sit. Five layers, each one you have to staff, budget, and defend in an audit whether or not the vendor's slide shows it.
+--------------------------------------------------+
| GOVERNANCE PLANE |
| (audit log, egress policy, PII controls) |
+--------------------------------------------------+
| EVAL HARNESS |
| (acceptance evals, CI gate, regression monitor) |
+--------------------------------------------------+
| ORCHESTRATION LAYER |
| (agent harness, tool dispatch, memory, context) |
+--------------------------------------------------+
| TOOL LAYER |
| (MCP servers, APIs, sandboxed execution envs) |
+--------------------------------------------------+
| INGESTION / CONTEXT LAYER |
| (RAG, fine-tune, context window policy, DLP) |
+--------------------------------------------------+
^ ^ ^
USER/LLM PLATFORM TEAM SECURITYThis is the capstone for the enterprise catalog. The prior 19 guides each own one slice in depth; this piece shows how the slices fit and where the seams leak. If you are earlier in the arc, start with the pillar on how enterprises ship AI agents, then come back here to see the whole board. For the six recurring shapes this architecture supports, the deployment-patterns breakdown is the companion read.
Layer 1: Ingestion and context
Three decisions live here: how agents get context (RAG, fine-tune, or raw context window), what is allowed in, and who owns the boundary. Data loss prevention sits at this layer too, the filter that stops PII from reaching the model at all. Skip that filter and the failure mode is context contamination: a retrieval query returns a document holding a customer SSN, and the agent folds it into a response nobody scoped for it. Your data platform team owns the retrieval design; your security team owns the DLP boundary. When those two owners are the same overloaded person, this is the layer that ships without a filter.
| Approach | When it wins | When it fails |
|---|---|---|
| RAG | Fresh data, large corpus, clear retrieval signal | Stale indexes, weak query routing |
| Fine-tune | Stable behavioral patterns, high inference volume | Slippage on domain drift, expensive to retrain |
| Context window | Simple, low-volume, small data set | Token cost at scale, no memory across sessions |
Where DLP sits, how you split RAG from fine-tune, and what the security team signs off are worked out in data boundaries for agents.
Layer 2: Tool layer
MCP servers, REST APIs, and sandboxed execution environments (code runners, browser-use containers) are what turns a chat model into something that acts. The platform team owns the allowlist: which tools a given skill can call, under what conditions, with what egress policy. The failure mode is tool sprawl. Forty skills accumulate, each with its own API credentials and no revocation path, and a tool approved for one pilot is quietly touching a production database two months later. That is a governance incident wearing a feature's clothes. The other failure is the sandbox nobody probed for a container escape before launch: an agent that can run arbitrary code in prod because the boundary went unverified.
Three questions define this layer: which tools are exposed, which skills can call which tools, and who approves an addition. For escape and egress design, see agent sandboxing patterns; for the governance standard around tool servers, governing MCP tool servers.
Layer 3: Orchestration layer
A harness decides how the agent gets called, which tool it invokes, how memory and context carry across turns, and how output gets post-processed before delivery. This is where harness choice sets the ownership split. The Claude Agent SDK, for example, manages the agent loop, sessions, permissions, hooks, subagents, and MCP connections as SDK surface (per Anthropic's Agent SDK docs, fetched 2026-08-13); a build-your-own harness moves all of that onto your team. That is the real content of the harness decision and the build-vs-buy call.
The failure mode is invisible state. A conversation-memory store grows with no pruning policy, or a tool-dispatch loop with no timeout runs six minutes and drains a user's session token budget. Teams that skip the memory and timeout design find it at incident review, not in the roadmap. Enforce per-skill token budgets here, at the layer that already sees every call; the modeling for that is in token-budget modeling for platform teams. The orchestration layer is also where the pillar's maturity model puts the production line, so cross-check your seat count against how enterprises ship AI agents.
Layer 4: Eval harness
The eval harness sits above orchestration because it gates changes to everything below it. Acceptance evals run in CI before a model version or skill update ships. A regression monitor runs in production, samples live traffic, and flags score drops. Without this layer you learn about regressions from support tickets.
Here is how the layer fails. The eval suite gets built once, at pilot time, on a demo use case, and it passes on the current model version. The vendor ships a model update. Nobody re-runs evals, because the gate is advisory rather than enforced. Two weeks later a skill that handles financial data starts returning wrong account numbers at a measurable rate, and the eval that would have caught it was never a blocker. The fix is a policy, not a tool: decide up front what blocks a merge and what only warns.
| Condition | Recommended action |
|---|---|
| Regression on safety/PII eval | Hard block, alert on-call |
| Regression on quality eval >5% | Block, open auto-issue |
| Regression on quality eval <5% | Warn, allow with owner approval |
| New skill with no baseline | Block until baseline is set |
CI-gate mechanics live in CI for agents and eval gating; the tooling comparison across Braintrust, LangSmith, Promptfoo, and Arize is here. Where do the evals come from? From the spec. Spec-driven agent development generates the acceptance evals as a byproduct of writing the spec, and the agent PRD approach treats those evals as the contract a skill has to hold.
Layer 5: Governance plane
The governance plane is not a box at the top; it is the cross-cutting concern that spans all four layers below. Every tool call, model invocation, and retrieval query writes to the audit log. The egress policy governs which layers reach which external endpoints. PII controls sit at the ingestion and delivery boundaries. This plane is what your security team audits and what your legal team cites in a data-processing agreement.
Governance fails when it arrives late. A pilot ships with no audit log, and adding one after the fact means touching every layer, migrating existing data, and negotiating with security over what "immutable" means for a log sitting in S3. The security team's own checklist is in agent skill security review. Two attack vectors deserve their own defenses at this plane: prompt injection at the orchestration layer and tool poisoning through MCP. And if you operate in the EU, the obligations are cited and mapped in the EU AI Act and your GPAI agent stack, where "governance coming soon" is not a compliant answer.
The seams: where the layers fail each other
Clean boxes are the vendor's story. Production is the set of joints between them, and the joints are where the incidents live. Four are worth naming.
Eval-to-orchestration. Your eval harness tests a mocked tool layer. The mock passes; the live tool has a latency spike, the agent times out and retries, and token spend doubles on a run that looked green in CI. Run real tools in CI for at least the eval subset covering your highest-risk skills, not mocks.
Governance-to-tool. The egress policy is declared at the orchestration layer, but the tool layer opens connections directly. Over time the policy list and the actual network calls diverge as tools get added outside review. Enforce egress at the network layer (a Kubernetes NetworkPolicy or equivalent), not only in application code, so the declared policy and the real one cannot drift apart.
Ingestion-to-orchestration. The context policy says "retrieve the top 5 chunks." For a rare query the top 5 come back from unrelated documents, and the agent synthesizes a confident wrong answer. A bigger context window does not fix this. A relevance threshold does: drop chunks below a cosine-similarity floor even when that leaves fewer than 5.
Eval-to-governance. Acceptance tests run in the eval harness but write nothing to the audit log, so a production incident involving eval-covered behavior cannot be replayed from the record. Instrument the eval runner with the same audit client and the same trace attributes as the production harness. The OpenTelemetry GenAI semantic conventions define the GenAI span, metric, and agent-span attributes for exactly this (per OpenTelemetry, fetched 2026-08-13); standardizing on them is what lets one query span both eval runs and live traffic. The full instrumentation walk-through is in observability for agents with OpenTelemetry.
What breaks first at scale
At 20 seats you hold the whole system in your head. At 200 you cannot, and three things break in a predictable order.
The tool allowlist. At 20 seats the platform lead knows every tool by name. Once you cross into the hundreds, a new skill deploys with a tool that was approved for a different skill, and code review misses it because reviewers rotate. Automate allowlist enforcement; do not rely on a human catching it.
The eval baseline. At pilot the suite covers the demo use case. At 200 seats you have 15 skills, each with its own baseline. If nobody extended the suite as skills landed, the CI gate covers 1 skill of 15 and passes confidently on the other 14. The rollout math for a large seat count is in rolling out AI coding assistants to 500 seats.
The token budget. At 20 seats overruns are visible and correctable. Cross into the hundreds and a single skill with a poorly scoped retrieval query can eat a large share of the monthly budget before the billing dashboard catches up. Set per-skill token budgets at the orchestration layer and alert at 80% burn before month-end, not after the invoice.
The adoption checklist
Assess your stack against the architecture, one row per layer, three maturity levels. The row that is furthest to the left is the one that breaks first. Who owns each of these rows, and how the platform team, security, and a center of excellence split the work, is laid out in pilot, platform team, and CoE org designs.
| Layer | Pilot (1-20 seats) | Production (20-200 seats) | At scale (200+) |
|---|---|---|---|
| Ingestion | Manual context, no DLP | RAG with relevance threshold | DLP at boundary, chunking policy reviewed |
| Tool layer | Allowlist in code review | Allowlist enforced in CI | Network-layer egress policy, tool registry |
| Orchestration | Single-skill, no memory | Multi-skill, session memory | Memory TTL policy, per-skill token budgets |
| Eval harness | Manual eval, no CI gate | CI gate on acceptance evals | Regression monitor on live traffic |
| Governance | Log exists | Log is immutable, exportable | SIEM integration, egress policy automated |
Print it, mark your real column per row, and the gaps are your Q4 platform roadmap. The layer sitting one column behind the others is where your next incident is already scheduled.