What Infrastructure Do You Need to Run AI Agents in Production in 2027?
A production AI agent needs more than a model and a prompt. The layers are a model endpoint, an orchestration framework that runs the agent loop, a way to connect tools (the Model Context Protocol, MCP, is the open standard), memory and a vector store, a sandbox for any code the agent runs, tracing and evals so you can see and score what it did, guardrails with permissions and human approval for risky actions, and cost controls.
You can buy or rent almost all of it. Anthropic lists current models, such as Claude Opus 5.5, Claude Sonnet 5.5 and Claude Haiku 5.5, on its own API and through cloud platforms, and the frameworks below are free libraries. What costs money is the metered parts: tokens, vector reads and writes, sandbox seconds and trace volume. Prices below are list prices read on October 31, 2026 from each vendor's page, and vendor descriptions are labelled as vendor claims. Confirm current prices before you budget. For the spending side, see the hidden costs of running AI infrastructure.
Model access and inference
Most teams start with a hosted model API. Anthropic's models page lists the current Claude lineup, and the same models are offered on Amazon Bedrock, Google Cloud and Microsoft Foundry, which matters if your company already buys cloud capacity from one of them. All four current models list a 1M token context window.
- Claude Opus 5.5 (Anthropic list price)$4 per million input tokens, $20 per million output tokens.
- Claude Sonnet 5.5 (Anthropic list price)$2 per million input tokens, $10 per million output tokens.
- Claude Haiku 5.5 (Anthropic list price)From $0.10 per million input tokens and from $0.50 per million output tokens; Anthropic positions it for classification, extraction and routing.
The practical lesson is that an agent is rarely one model. A cheap, fast model can route and extract while a larger model handles the hard steps, so design the endpoint layer to switch models by step. This page does not cover self-hosting your own inference servers, because none of the sources read here give a figure for it.
Orchestration: the agent loop
The orchestration layer runs the loop of planning, calling tools and deciding when the task is done, and it is where state lives between steps. Four frameworks are common, each described here in its own words.
- LangGraph. Describes itself as a low-level orchestration framework and runtime for long-running, stateful agents, with durable execution that resumes from where an agent left off after a failure.
- OpenAI Agents SDK. Describes a lightweight package with very few abstractions; its primitives are agents, handoffs, guardrails, sessions and built-in tracing.
- Claude Agent SDK. Gives you the tools, agent loop and context management that power Claude Code, programmable in Python and TypeScript, with hooks, subagents, MCP, permissions and sessions.
- CrewAI. Calls itself the leading open-source framework for orchestrating autonomous agents, a vendor claim; it describes Flows as event-driven workflows that manage state and Crews as teams of agents.
If you would rather not run the loop yourself, Anthropic also offers Managed Agents, described as a hosted agent harness with sessions in an Anthropic-managed cloud sandbox or a self-hosted sandbox on your own infrastructure.
Tool connectivity with MCP
Agents are only useful if they can reach your systems. MCP is described on its own site as an open-source standard for connecting AI applications to external systems: data sources such as local files and databases, tools such as search engines and calculators, and workflows such as specialized prompts. The site compares it to a USB-C port for AI applications.
Both the OpenAI Agents SDK and the Claude Agent SDK list MCP support, and the MCP site names Visual Studio Code and Cursor among supporting tools. The infrastructure implication is that each internal system gets one MCP server instead of one custom integration per framework. Treat every server as a new door into your data: the MCP site has a dedicated security best practices page, and you should read it before exposing anything that writes.
Memory and vector stores
Agents need two kinds of memory. Working memory holds the current run, and LangGraph describes short-term working memory and long-term memory across sessions. The OpenAI Agents SDK describes sessions as a persistent memory layer for working context within an agent loop. Long-term knowledge, such as documents and past tickets, usually sits in a vector database.
Vector stores are metered. Pinecone's public pricing lists storage at $0.33 per GB per month on its Standard and Enterprise plans, read units at $16 to $18 per million on Standard and $24 to $27 per million on Enterprise, and write units at $4 to $4.50 per million on Standard and $6 to $6.75 per million on Enterprise, with minimum monthly usage of $50 on Standard and $500 on Enterprise. Those are one vendor's list prices and vary by cloud and region.
Sandboxed code execution
An agent that writes and runs code or shell commands should do it away from your production systems. The OpenAI Agents SDK describes sandbox agents running specialists inside real isolated workspaces, and Anthropic's Managed Agents run in a managed cloud sandbox or a sandbox on your own infrastructure.
If you buy sandboxes as a service, E2B publishes its prices: a Hobby plan at $0 plus usage with a one-time $100 in credits, a Pro plan at $150 per month plus usage, $0.000014 per vCPU-second and $0.0000045 per GiB-second of RAM, with sessions up to 1 hour on Hobby and up to 24 hours on Pro. E2B's page says its sandboxes are CPU-only, so a GPU workload needs another route.
Observability, tracing and evals
Without traces you cannot debug an agent, because the failure is somewhere in a chain of model calls and tools. Arize Phoenix says tracing lets you see what happened during a single run step by step, accepts traces over OpenTelemetry, and supports evaluations that measure output quality. The OpenAI Agents SDK has built-in tracing, and LangGraph points to LangSmith for debugging.
- Langfuse Core (vendor list price)$29 per month with 100k units included, then $8 per 100k units; Langfuse says it is open source and can be self-hosted for free.
- LangSmith Plus (vendor list price)$39 per seat per month with 10k base traces per month, then pay as you go; base traces have a retention period of 14 days.
Note the billing unit. Langfuse defines a billable unit as any tracing data point, including traces, observations and scores, so a chatty multi-step agent produces many units per user request. Evals belong in the same layer: score a fixed set of real tasks before each release, and keep the traces that failed.
Guardrails, identity and human approval
This layer decides what the agent may do. The OpenAI Agents SDK lists guardrails that validate agent inputs and outputs and built-in mechanisms for involving humans during runs. LangGraph describes inspecting and modifying agent state at any point. The Claude Agent SDK lists permissions that control which tools run automatically and which need approval, and hooks that run custom code at key points in the agent lifecycle.
- Least privilege. Give each agent only the tools it needs, and require approval for anything that writes, sends or spends.
- Identity. Run the agent under its own service identity with scoped credentials rather than a person's login. None of the pages read here prescribe a specific identity product, so this is a design rule, not a vendor claim.
- Human in the loop. Pause at the steps where a mistake is expensive and let a person approve or edit.
Cost controls
Agents make many model calls per task, so cost control is part of the infrastructure. Anthropic's models page says batch API requests are 50% off and that prompt cache reads cost 10% of the base input price, with 5% on Claude Opus 5.5 and Claude Sonnet 5.5. The batch page says most batches finish in less than 1 hour, which suits offline evals and back-office jobs, not interactive chat.
Questions people ask
What is the minimum infrastructure to run an AI agent in production?
A model endpoint, an orchestration loop, the tools the agent calls, somewhere to keep state, tracing so you can see what happened, and approval rules for risky actions. Every layer can start as a hosted service or a library; this page lists what the vendors' own pages say each one does.
Do I need to self-host a model?
Not necessarily. Current Claude models are offered through the Claude API, Amazon Bedrock, Google Cloud and Microsoft Foundry according to Anthropic's models page, so a hosted endpoint is the common starting point. Self-hosting is a separate decision about data location and cost that none of the pages read here settles for you.
Which orchestration framework should I pick?
The pages read here do not rank them. LangGraph stresses durable execution and human oversight, the OpenAI Agents SDK stresses few abstractions, the Claude Agent SDK ships built-in tools and permissions, and CrewAI describes Flows and Crews. Pick on the capabilities you need and test with your own task.
What does MCP do in an agent stack?
It is an open-source standard for connecting AI applications to external systems such as databases, tools and workflows, so a tool is built once and used by many clients. It does not replace your own permission rules; the MCP site has a separate security best practices page.
Why do agents need sandboxes?
Because an agent that runs code or commands should do it in an isolated workspace. The OpenAI Agents SDK describes sandbox agents running in real isolated workspaces, and Anthropic's Managed Agents run in a managed or self-hosted sandbox. E2B publishes per-second sandbox prices.
How do I keep agent costs under control?
Measure per run, route simple steps to a smaller model, and use the discounts vendors publish. Anthropic lists a 50% discount on batch requests and cache reads at a fraction of the base input price. Add spend limits at your own gateway, since none of the pages read here describes one.
Sources
- Anthropic: Models overview · Current Claude models, prices, platforms, batch and cache discounts
- Anthropic: Batch processing · Message Batches API cost and timing
- Model Context Protocol: What is MCP? · Definition and ecosystem
- LangGraph overview · Durable execution, memory, human-in-the-loop
- OpenAI Agents SDK documentation · Agents, guardrails, sessions, tracing, sandbox agents, MCP
- Claude Agent SDK overview · Built-in tools, permissions, hooks, MCP, Managed Agents
- CrewAI introduction · Flows and Crews
- Pinecone pricing · Vector database storage, read and write unit list prices
- E2B pricing · Sandbox plans and per-second compute prices
- Langfuse pricing · Tracing plans, billable units, self-hosting
- LangChain pricing (LangSmith) · LangSmith plans and trace retention
- Arize Phoenix documentation · Tracing over OpenTelemetry and evaluations
Figures and descriptions were read from the linked pages on October 31, 2026, several through a fetch tool that returns summaries, so confirm each price on the vendor's own site. Framework descriptions are the vendors' own. This page is not financial or legal advice.
Top photo: Xpda / CC BY-SA 4.0 via Wikimedia Commons