Arize AX is the best AI model monitoring tool: it has a $0 plan, a $50 Pro plan and a Signal feature that surfaces issues and root causes from production traces. Fiddler AI is #2 for monitoring agents and predictive ML models together, and Datadog LLM Observability is #3 for teams that want AI traces beside their services.
Open source and free to start; run locally or self-host
Developers who want free, local LLM tracing and evals before paying for a platform
Pick by what you want
Best overall
Arize AX
Best for regulated ML
Fiddler AI
Best beside APM
Datadog LLM Observability
Best free open source
Evidently AI
Best open-source tracing
Langfuse
Best for delayed labels
NannyML
Best free local tracing
Arize Phoenix
The ranking
1
Arize AX
Answrs VerdictThe most complete managed platform here, with a $0 start and a $50 Pro plan.
Price: AX Free $0; AX Pro $50 per month; AX Enterprise is customPro plan: Pro includes 50k spans per month, 10 GB ingestion, 30 days retention and unlimited users and evalsSignal: Signal surfaces what matters, uncovers root causes and generates review-ready fixes from production tracesDeployment: Enterprise runs as SaaS or self-hostedScale: The home page cites 1 trillion spans processed
Arize AX is a managed platform for observing and evaluating AI agents. Its Signal feature surfaces what matters, uncovers root causes and generates review-ready fixes from production traces, and the home page cites 1 trillion spans processed. AX Free is $0, AX Pro is $50 per month with 50k spans, 10 GB ingestion and 30 days retention, and Enterprise is custom and can be self-hosted.
Go ifTeams running AI agents in production that want issues detected and explained automatically
Skip ifYou only need classic tabular drift reports
Why #1 over #2: It pairs a $0 start with automatic issue detection, while Fiddler AI lists no flat monthly plan.
2
Fiddler AI
Answrs VerdictThe one platform that watches predictive models and agents together, with drift and explanations.
Price: Guardrails are Free; Developer is $0.002 per trace; Enterprise is contact salesCoverage: Monitors agents and predictive ML models, including inputs, predictions and explanationsMetrics: 100+ out-of-the-box and custom metrics including hallucination, toxicity, PII/PHI, drift and performanceGuardrails: The pricing page lists a latency of <80ms for its guardrailsCompliance: SOC 2 Type II certified
Fiddler AI monitors agents and predictive ML models in one platform, covering inputs, predictions and explanations. It offers 100+ out-of-the-box and custom metrics including hallucination, toxicity, PII/PHI, drift and performance, and it is SOC 2 Type II certified. Guardrails are Free, Developer is $0.002 per trace, and Enterprise is contact sales.
Go ifRegulated enterprises that must monitor both LLM agents and classic ML models
Skip ifYou want a self-serve flat monthly plan
Why #2 over #3: It covers classic ML drift as well as agents, which Datadog’s LLM product does not describe.
3
Datadog LLM Observability
Answrs VerdictThe best fit when AI monitoring should live beside APM and infrastructure data.
Price: Free $0 per month for up to 40k LLM spans; Pro $160 per month for up to 100K LLM spansBilling: Only LLM spans are billed; tool, workflow, agent, embedding and retrieval spans are freeQuality checks: Catches hallucinations, prompt injection attempts and PII exposure as they happenCorrelation: Correlates LLM spans with APM services, infrastructure signals and RUM sessionsRetention: 15-day retention on both plans, with add-ons to 30, 60 or 90 days
Datadog LLM Observability catches hallucinations, prompt injection attempts and PII exposure as they happen, and correlates LLM spans with APM services, infrastructure signals and RUM sessions. Only LLM spans are billed: Free is $0 per month for up to 40k LLM spans and Pro is $160 per month for up to 100K LLM spans, with 15-day retention on both.
Go ifTeams already on Datadog that want AI traces next to their services and infrastructure
Skip ifYou need classic tabular drift statistics
Why #3 over #4: It ties AI behavior to services and infrastructure, which open-source Evidently AI does not.
4
Evidently AI
Answrs VerdictThe free, open-source starting point that covers drift for both ML and LLM systems.
Price: Free and open source under Apache 2.0; hosted pricing is not published on the siteScope: Evaluate, test and monitor LLMs, RAG applications, AI agents and ML models in a single frameworkMetrics: A library of 100+ in-built metrics, plus custom onesMonitoring: A live dashboard tracks evaluation results so you catch drift, regressions and emerging risks earlyCommunity: 7500+ GitHub stars and 40m+ downloads
Evidently AI is open source under Apache 2.0 and evaluates, tests and monitors LLMs, RAG applications, AI agents and ML models in a single framework. It has 100+ in-built metrics, a live dashboard for continuous monitoring, 7500+ GitHub stars and 40m+ downloads. Hosted pricing is not published on the site.
Go ifTeams with an engineer who wants to own a free monitoring stack for LLMs and tabular models
Skip ifYou want a fully managed platform with a published price
Why #4 over #5: It is free and covers tabular ML as well as LLMs, while Langfuse centers on LLM tracing.
5
Langfuse
Answrs VerdictThe popular open-source tracing platform with clear published pricing.
Price: Hobby is free; Core $29 per month; Pro $199 per month; Enterprise $2499 per monthUsage: Paid plans include 100k units per month, with additional units at $8/100kOpen source: The home page shows 36.0k GitHub stars and self-hosting guides for Docker Compose, Kubernetes (Helm) and TerraformOwnership: Langfuse joined ClickHouseTracing: Traces, evaluates and improves AI agents on one open platform
Langfuse is an open-source platform that traces, evaluates and improves AI agents; its home page shows 36.0k GitHub stars and self-hosting guides for Docker Compose, Kubernetes (Helm) and Terraform. Hobby is free, Core is $29 per month, Pro is $199 per month and Enterprise is $2499 per month, with 100k units included and additional units at $8/100k. Langfuse joined ClickHouse.
Go ifDevelopers who want open-source LLM tracing they can self-host or run as a cloud plan
Skip ifYou need tabular drift reports for classic ML models
Why #5 over #6: Its published plans start cheaper than LangSmith’s per-seat model.
6
LangSmith
Answrs VerdictThe LangChain team’s own tracing and evaluation product, with a free developer seat.
Price: Developer $0 per seat per month; Plus $39 per seat per month; Enterprise is customTrace volume: Developer includes 5k base traces per month and Plus includes 10k, then pay-as-you-goRetention: Base traces are kept for 14 days and extended traces for 180 daysSeats: Developer is limited to 1 seat; Plus adds unlimited seatsData: LangSmith says it does not use your data to train models
LangSmith is the LangChain team’s tracing and evaluation product. Developer is $0 per seat per month with 1 seat and 5k base traces, Plus is $39 per seat per month with 10k base traces, and Enterprise is custom. Base traces are kept for 14 days and extended traces for 180 days, and the vendor says it does not use your data to train models.
Go ifTeams building on LangChain or LangGraph that want traces and evals in one place
Skip ifYou want a per-seat price to stay flat as traces grow
Why #6 over #7: It has a free developer seat and a documented retention model, while New Relic prices by data ingest.
7
New Relic AI Monitoring
Answrs VerdictFull-stack observability that adds AI response, token and cost views.
Price: Free tier of 100 GB per month of data ingest, then $0.40/GB; core users start at $49 per userAI metrics: A summary of AI metrics from response time and quality to tokens, APM golden signals and infrastructureQuality: Spot and solve issues like bias, hallucination and toxicityCost control: Monitor and control costs by tracking request tokens and setting custom alertsComparison: Compare different models for cost, performance and quality in one view
New Relic AI Monitoring summarizes AI metrics from response time and quality to tokens, APM golden signals and infrastructure, helps spot bias, hallucination and toxicity, and tracks request tokens with custom alerts. The pricing page lists a free tier of 100 GB per month of data ingest, then $0.40/GB, with core users starting at $49 per user.
Go ifTeams already on New Relic that want AI response, token and cost data in the same account
Skip ifYou need classic ML drift statistics
Why #7 over #8: It adds cost and token views inside a full-stack platform, which NannyML does not attempt.
8
NannyML
Answrs VerdictThe specialist for estimating model performance when ground truth is delayed.
Price: Open source is free; Starter $399 per month; Scale $999 per month; Enterprise is a quoteStarter limits: Starter covers 2 models and 10 million predictions with a 30 days free trialMethod: Monitoring for all model types, also when ground truth is not availableAlgorithms: Flagship algorithms for concept drift, performance estimation and multivariate drift detectionOwnership: Soda announced it acquired NannyML
NannyML offers monitoring for all model types, also when ground truth is not available, with flagship algorithms for concept drift, performance estimation and multivariate drift detection. Open source is free, Starter is $399 per month for 2 models and 10 million predictions with a 30 days free trial, and Scale is $999 per month. Soda announced it acquired NannyML.
Go ifTeams whose labels arrive late and who want estimated performance instead of waiting
Skip ifYou need LLM trace capture
Why #8 over #9: It estimates performance when labels are late, which Weave’s trace-based monitors do not address.
9
Weights & Biases Weave
Answrs VerdictAgent-native tracing and monitors from the experiment-tracking vendor.
Price: Free $0 per month; Pro starts at $60 per month billed monthly; Enterprise is customProduction monitoring: Built-in and custom signals automatically capture and classify agent interactions, with alertsTracing: Sessions, turns, steps, tools and sub-agents are first-class concepts in agent tracingPro features: Pro adds Slack and email alerts and CI/CD automationsFree plan: Free includes AI application tracing and 1 GB per month of Weave data ingestion
Weights & Biases Weave monitors production agents with built-in and custom signals that capture and classify interactions, with alerts. Agent tracing treats sessions, turns, steps, tools and sub-agents as first-class concepts. Free is $0 per month with 1 GB of Weave ingestion, and Pro starts at $60 per month billed monthly. The pricing page is now branded CoreWeave Forge.
Go ifTeams that already track experiments in Weights & Biases and now run agents in production
Skip ifYou expect the old Weights & Biases pricing page; it is now branded CoreWeave Forge
Why #9 over #10: It brings agent-native tracing and monitors, while Phoenix leaves alerting to you.
10
Arize Phoenix
Answrs VerdictThe free local option that pairs with Arize AX when you scale.
Price: Open source and free to start; run locally or self-hostScope: Trace, evaluate and improve AI agents with PhoenixStandards: The Arize home page describes OpenInference as built on OpenTelemetryModel: Phoenix is the open-source observability and evals tool, separate from the managed Arize AX platform
Arize Phoenix is the open-source observability and evals tool from Arize, separate from the managed Arize AX platform: you trace, evaluate and improve AI agents, and you run it locally or self-host it. It is free to start, which makes it a low-risk way to try LLM tracing before choosing a managed platform.
Go ifDevelopers who want free, local LLM tracing and evals before paying for a platform
Skip ifYou need managed alerting and enterprise support from day one
How we ranked these
We ranked tools by how much of the model lifecycle they monitor (agents, LLMs and classic ML), then by each vendor’s published price, using only vendor pages. The old list also had WhyLabs, Superwise, Kolena, Neptune.ai, Mona and Censius. We replaced them because we could not confirm a current model monitoring product with a dashboard image from their own sites: Superwise’s pricing page now lists Sentinel and Chat plans, and the others were not verified in this pass. Facts are taken from each maker's own published information; check the official site before you buy.
Questions people ask
What is the best AI model monitoring tool?
Arize AX is our top pick: it has a $0 plan, a $50 Pro plan and a Signal feature that surfaces issues and root causes from production traces. Fiddler AI is #2 because it monitors both agents and predictive ML models.
Which AI model monitoring tools are free?
Evidently AI is free and open source under Apache 2.0, Arize Phoenix is open source and free to start, Langfuse has a free Hobby plan, and Datadog and Arize AX each have a $0 plan.
Which tool monitors models when labels arrive late?
NannyML states it offers monitoring for all model types, also when ground truth is not available, using performance estimation and drift detection.
How much does AI model monitoring cost?
Published entry prices run from Langfuse Core at $29 per month and LangSmith Plus at $39 per seat, to Arize AX Pro at $50, Weave Pro from $60, Datadog Pro at $160 and NannyML Starter at $399.
Advertised monthly price of the cheapest paid plan on each vendor’s pricing page (flat monthly plans only; per-seat and usage-based pricing excluded) at a glance
Langfuse29 USD
Arize AX50 USD
Weave60 USD
Datadog160 USD
NannyML399 USD
Advertised monthly price of the cheapest paid plan on each vendor’s pricing page (flat monthly plans only; per-seat and usage-based pricing excluded), as published by each maker. Picks without a published figure are left off.
