LLM observability and evaluation platforms: LangSmith vs Langfuse vs Arize Phoenix vs Braintrust vs MLflow 3 (August 2026)

A decision-grade comparison of five LLM and agent observability platforms, for a platform team choosing one default across multiple teams.

Rating legend: ●●● strong · ●●○ adequate · ●○○ weak · — n/a

Reading guide#

Executive summary#

For a multi-team agent platform on cloud-agnostic or self-hosted infrastructure in a regulated setting, Langfuse is the default pick at 84.8 vs 79.6 (Braintrust), 76.0 (MLflow), 71.6 (Phoenix) and 68.8 (LangSmith). It is the only contender that ships continuous LLM-as-judge evaluation on production traces in a free, MIT-licensed, fully self-hostable package with no feature gates on the core platform 7829. Every other option either paywalls production monitoring, paywalls self-hosting, or both.

Three findings should change how you act on this:

1. Tracing is commoditized; production evaluation is where every vendor draws the paywall. All five ingest OTLP and render agent span trees competently. The consistent upsell boundary is continuous evaluation of live traffic: Phoenix OSS does scheduled/batch evals and sends you to Arize AX for real-time online evals with alerting 20; open-source MLflow has no production monitoring service at all: it is a Databricks-platform integration 15; LangSmith has it but self-hosting requires an Enterprise contract 9. Langfuse is the exception, not the norm.
2. Phoenix is less vendor-neutral than its open-source status implies, and the gap is monetized. Phoenix does not natively understand OpenTelemetry gen_ai.* conventions; spans must be translated to Arize's OpenInference schema via a custom span processor, or you lose structured message indexing and span-kind rendering in the UI 16. Arize AX, the paid product, did add native GenAI-semconv support 17. The open-source tool is the one that locks you into a vendor-specific convention.
3. MLflow is a genuine answer to "do I need a new tool?" but only for half the job. Open-source MLflow 3 is fully OTel-compatible, natively supports GenAI semantic conventions for both export and ingestion, and auto-instruments 24 Python frameworks including LangGraph and Google ADK, plus TypeScript and, uniquely, Java via Spring AI and Quarkus LangChain4j 1314. What it does not have in the OSS build is continuous production monitoring, the human-feedback Review App, or production-scale trace ingestion; Databricks lists all three as platform integrations 15.

The most likely thing to change this answer: Langfuse was acquired by ClickHouse on 2026-01-16 18. The MIT license and self-hosting commitment were explicitly reaffirmed, but a 7-month-old acquisition is not yet a track record.

Recommendation: Langfuse (self-hosted) as the paved road · Confidence: medium-high on the ranking, medium on the margin over Braintrust

Decision scorecard#

Weights reflect the stated context: a regulated, multi-team, cloud-agnostic platform where self-hosting and data control are near-mandatory, production quality monitoring is the core job, and per-trace vendor pricing is a standing liability. Convenience of a managed control plane earns little here.

Reading the table: Langfuse's 5.2-point margin over Braintrust comes almost entirely from dimensions 1 and 7: licensing freedom and cost. On the product dimensions (3, 4, 5, 9) Braintrust matches or beats it. This is close to a tie in capability and a clear win on terms. If your organization is comfortable with a commercial contract and a VPC-resident data plane, Braintrust is a legitimate choice and the gap effectively vanishes.

LangSmith finishes last despite scoring 5 on three product dimensions. That is not a judgment on the product (it is the best-in-class experience for LangGraph), it is the weighting punishing a closed-source, Enterprise-gated, per-trace-priced tool in a context that explicitly prioritizes self-hosting and cost control. Re-weight dimensions 1 and 7 to near-zero (you are a LangGraph shop on LangChain Cloud with no residency constraints) and LangSmith moves to the front of the pack.

At-a-glance#

Deep dive#

1. Self-host and licensing freedom#

This is the dimension that separates the field, and the terms are genuinely different in kind, not degree.

Langfuse is MIT-licensed and the free self-hosted build carries "all core Langfuse features and APIs without any limits," with no scalability restrictions relative to paid tiers 7. Confirmed against the self-host pricing page: observability, evaluation, prompt management and datasets are all in the free tier 8. A paid LANGFUSE_EE_LICENSE_KEY unlocks a specific, published list: project-level RBAC roles, protected prompt labels, data retention policies, audit logs, server-side data masking, UI customization, organization creators, the Org Management API and SCIM, and the Instance Management API 7. Read that list carefully: it is entirely governance and multi-tenancy, not capability. For a regulated enterprise, several of those (audit logs, server-side masking, retention policies) are not optional, so budget for the Enterprise key. But you are buying governance, not unlocking the product.

MLflow is Apache-2.0 with no equivalent of an EE key. Databricks' own comparison page frames managed MLflow's advantages as platform integrations (Unity Catalog governance, managed hosting, notebook integration, the Review App, production monitoring) rather than as licensed features withheld from the OSS build 15. The page states plainly: "The core data model and APIs are completely open source. You can export and use your MLflow data anywhere." That is an unusually clean commitment and it is worth taking seriously.

Phoenix is Elastic License 2.0 1, self-hostable via Docker Compose, standalone Docker, or a Kubernetes Helm chart, with no feature gates on the OSS build. ELv2 is not OSI-approved and prohibits offering the software as a managed service to third parties. For internal enterprise use this restriction is inert, but it is the kind of thing legal review flags, and it is a real difference from Langfuse's MIT and MLflow's Apache-2.0. Rated 4 rather than 5 for that reason alone.

LangSmith offers self-hosted and hybrid deployment (SaaS control plane, self-hosted data plane, data in your VPC) only on the Enterprise plan 9. Developer and Plus are cloud-only. There is no self-serve path; pricing is custom and requires a sales engagement. This is precisely the finding flagged in the earlier Google ADK vs LangChain/LangGraph report (that LangGraph's best debugging experience is proprietary and self-hostable only under contract), and it holds up on direct inspection of the pricing page.

Braintrust is closed-source, but its hybrid model is the most interesting architecture in the field. Enterprise customers run the data plane (traces, datasets, prompts, eval results) inside their own AWS, GCP, or Azure VPC via official Terraform modules, while Braintrust manages the control plane (UI, metadata, auth). SDKs and the browser talk directly to your data plane via CORS; only metrics and status telemetry flow back to Braintrust 22.

For a regulated enterprise that wants a managed UX without shipping prompts and completions to a vendor, this is a better answer than "trust our SOC 2." The SDK can be further constrained so the data plane proxies to the control plane, "eliminating any need for outbound connections from the SDK to Braintrust's servers" 22. The hybrid-deployment page does not itself state a tier requirement; the pricing page lists on-prem/hosted deployment under Enterprise 11, which is why it scores 2 and not higher.

2. Online evaluation and production monitoring: the paywall boundary#

If you take one structural insight from this comparison, take this one. Trace collection is a solved, commoditized, free capability across all five tools. Continuous evaluation of production traffic is where four of the five draw a line and charge.

• Langfuse: LLM-as-judge evaluators run continuously on production traces via an asynchronous architecture the docs describe as processing "thousands of evaluations per minute," completing in seconds 29. Not on the EE-gated list 7, and the self-host pricing page confirms evaluators are in the free tier 8. This is the single strongest reason for the verdict.
• LangSmith: Online evals run on live production traffic without reference outputs, driven by automation rules that trigger evaluators on tracing projects: mature, well-documented, real-time 23. Available in cloud on all tiers; in your own infrastructure only under an Enterprise contract.
• Braintrust: Online scoring and monitoring on production traffic, with the caveat that the product's centre of gravity is the offline experiment loop.
• Phoenix: The OSS path is to schedule an evaluation script via cron or Airflow and feed results back into the UI for trend tracking. For continuous evals on production traffic with alerting and threshold-based triggers, Arize's own documentation directs you to AX Online Evals 20. Batch-on-a-timer is a reasonable 80% solution; it is not the same product.
• MLflow: No production monitoring service in the OSS build. Databricks provides "a production monitoring service that continuously evaluates a sample of your production traffic using LLM judges and scorers … powered by production-scale trace ingestion that includes storing traces to Unity Catalog tables", listed explicitly under "Additional capabilities on Databricks" 15. Same for the Review App human-feedback and trace-labeling UI.

One nuance worth stating precisely, because vendor marketing blurs it: open-source MLflow does have LLM judges and scorers (mlflow.genai.scorers), multimodal judges that can read images in traces as of 3.15.0, and a pytest integration for quality assertions in CI as of 3.14.0 5. What it lacks is the continuous, sampled, always-on evaluation of live traffic and the surrounding monitoring/alerting surface. Some third-party summaries flatten this into "OSS MLflow lacks high-quality LLM judges," which overstates the gap in Databricks' favour. The accurate statement is narrower and still decisive: MLflow OSS is an excellent offline evaluation harness and trace recorder, and not a production monitoring system.

3. Vendor neutrality and the OpenTelemetry GenAI conventions#

All five accept OTLP. Ingestion neutrality is universal and is not a differentiator. The differentiator is which semantic convention the backend natively understands, because that determines whether your instrumentation is portable or whether you have re-implemented vendor lock-in one layer up.

• MLflow is "fully OpenTelemetry-compatible" and "natively supports GenAI Semantic Conventions for export and ingestion" 13. Both directions. That is the strongest neutrality claim in the field.
• Braintrust implements the OTel GenAI semantic conventions; spans carrying gen_ai.* attributes map automatically to Braintrust fields at https://api.braintrust.dev/otel/v1/traces 25.
• Langfuse maps the widest set: OTel GenAI, OpenInference, MLflow's mlflow.span* attributes, and OpenLLMetry/OpenLIT, with a langfuse.* namespace taking precedence 12. Practical limitation: OTLP over HTTP only (JSON and protobuf); gRPC is not supported 12. If your collector fleet is standardized on OTLP/gRPC you will need an HTTP exporter path or a collector-side translation.
• LangSmith exposes an OTLP endpoint at https://api.smith.langchain.com/otel, but the documented expectation is the OpenLLMetry convention rather than the OTel GenAI conventions 24. Self-hosted instances use your own endpoint plus /api/v1/otel.
• Phoenix is the surprise. It "accepts traces over OTLP" and markets itself as "not tied to any specific LLM vendor or framework", but the conventions documentation states that GenAI spans must be translated to OpenInference to render properly. In TypeScript you must build a custom TraceExporter using @arizeai/openinference-genai to convert attributes before export. In Go you may emit gen_ai spans directly, but the docs warn those spans "do not carry openinference.span.kind or structured llm.input_messages.* / llm.output_messages.* indexing," so UI features that depend on OpenInference attributes are degraded 16. Meanwhile Arize AX, the commercial product, added native GenAI-semconv support with a GenAI-to-OpenInference normalization layer 17. The open-source tool requires you to adopt a vendor-specific schema; the paid tool speaks the open standard. Native gen_ai support in Phoenix is tracked as an open feature request rather than shipped behaviour 19.

A caveat on the standard itself. Multiple secondary sources report that as of July 2026 no gen_ai.* span, metric, event, or attribute is marked Stable (all carry the Development badge), and that gen_ai.* was moved out of the main semantic-conventions repository into a dedicated GenAI conventions repository in the v1.42.0 release on 2026-06-12. I confirmed the dedicated repository exists and covers "spans, metrics, and events for GenAI clients, MCP, and provider-specific conventions" 30, but I could not confirm the stability badges or the split date from a primary OpenTelemetry source in this session unverified.

Treat the specific date and stability status as directionally right and verify before citing it in a design document. The practical implication is unchanged either way: betting your instrumentation layer on a convention still under active development means accepting churn, and it argues for a backend that maps several conventions, which is Langfuse's quiet advantage.

This matters directly for the earlier ADK report's finding that ADK emits native OpenTelemetry with GenAI semantic conventions. If you run ADK, MLflow and Braintrust consume that telemetry natively; Phoenix will need a translation layer. That is a concrete, non-obvious consequence of pairing those two decisions.

4. Tracing depth for agentic flows#

Functionally close across the field; the differences are in ergonomics and in which framework is first-party.

LangSmith is the deepest for LangGraph, unsurprisingly, it is built by the same company, and node-level state transitions, retries, and interrupts surface without configuration. Phoenix shipped agent session persistence in v20.0.0 (2026-08-11) along with trace_annotations in the span-filter DSL 2, and is iterating fast: v19.21.0 through v20.1.0 landed in three days. Langfuse's v4 rewrite (March 2026) restructured around a single unified observations table specifically to make filtering and aggregation fast on large projects 28, which is the right architectural bet for high-volume multi-team use.

MLflow's coverage is the broadest by raw count: 24 Python agent frameworks with one-line auto-instrumentation, including LangChain, LangGraph, Google ADK, OpenAI Agents, DSPy, PydanticAI, CrewAI, LlamaIndex, AutoGen and the Claude Agent SDK; six TypeScript frameworks; and coding-agent integrations for Claude Code, Codex CLI, OpenCode, OpenHands and Gemini CLI 14.

Note the asymmetry this creates against the ADK report's conclusion. That report recommended LangGraph as the paved road with ADK sanctioned for JVM/Go teams and voice/multimodal work, and flagged that LangChain has no JVM answer at any price. On the observability layer, the same asymmetry recurs but resolves differently: MLflow is the only one of these five with Java auto-instrumentation (Spring AI and Quarkus LangChain4j) 14. If your JVM teams are on ADK, MLflow is the only tool here that traces them out of the box, which is a strong argument for the split-stack recommendation below.

5. Governance, security and compliance#

Braintrust leads on architecture: SOC 2 Type II, HIPAA and GDPR compliance with BAAs available, AES-256 at rest, TLS 1.2 in transit, API keys stored as one-way hashes, annual SOC 2 audit and pen-test summaries available through a Trust Center under NDA 22. The hybrid data plane (a Terraform-deployed API layer plus Postgres inside your VPC, with browser-to-data-plane CORS so customer data bypasses Braintrust's servers entirely) is the strongest data-residency story in the comparison.

Langfuse holds SOC 2 Type II, ISO 27001, and offers a dedicated HIPAA-ready region, with cloud regions in US (us-west-2), EU (eu-west-1), Japan (ap-northeast-1) and a separate HIPAA region; self-hosted supports any region or provider including air-gapped 21. Controls include server-side data masking, configurable retention policies, deletion tooling, and project-scoped tenant isolation, but note that masking, retention policies, audit logs and project-level RBAC are all EE-gated in self-host 7. A regulated deployment on the free tier will fail an audit on those four items specifically. Plan for the Enterprise key.

MLflow added workspace-level RBAC with an admin UI in 3.13.0 and trace archival to object storage in the same release 5. Self-hosted means the compliance perimeter is yours, which is either the point or the problem depending on your platform team's appetite. Deeper governance (Unity Catalog lineage, system-table audit logs, on-behalf-of-user auth for agent tool access) is Databricks-side 15.

Phoenix self-hosted puts the perimeter entirely in your hands, and v20.1.0 added JWT client-assertion support for OAuth2 2, but built-in governance surface is thinner than Langfuse EE or Databricks.

LangSmith rates a 3 not for weakness but for access: the deployment models that satisfy residency requirements exist and are well-specified (cloud / hybrid / fully self-managed in your VPC) 9 and are unavailable without an Enterprise contract.

6. Prompt management and dataset curation#

Langfuse, LangSmith and Braintrust are all strong and roughly equivalent for the core loop: version prompts, curate datasets from production traces, run experiments. Langfuse's differentiator for regulated work is protected prompt labels (EE) 7, which is a direct control on the failure mode the Ray Serve in the MDLC report identified: that user_config hot-reloads without restarting replicas, so a validated threshold can change with no image change and no deploy record. A prompt registry with protected labels plus audit logs closes exactly that governance hole for the prompt half of the config surface. Langfuse promoted Experiments to a top-level feature alongside Datasets in April 2026.

Braintrust's playground and experiment-diffing UX is the best in the field for the tight iterate-and-compare loop; it is the product's origin story. MLflow has a prompt registry with versioning, comparison and discovery in the UI, plus an MCP Registry added in 3.15.0 for cataloguing MCP servers with semantic-versioned configs and promotable aliases 5, a forward-looking feature none of the others match. Phoenix is the lightest here; prompt playground exists but prompt lifecycle management is not its focus.

Total cost of ownership#

Modelled scenario: 4 product teams, 5M traces/month, averaging 8 spans/observations per trace (40M spans/month), LLM-as-judge scoring applied to a 10% sample (500k judged traces/month), 20 engineers needing UI access, ~4KB average processed span payload (**~160GB/month**), self-hosted options on existing managed Kubernetes in one region. Retention assumed at 30–90 days, which is the honest requirement for building eval datasets from production traffic and for incident forensics.

Assumptions, stated plainly. Every figure above is a modelled estimate from published unit prices, not a quote. LangSmith and Langfuse Cloud figures are computed from verified public rates 106; Braintrust from its published Pro tier 11; infrastructure ranges are engineering estimates and are the softest numbers in the table approx.

Five things this model exposes that a sticker-price comparison hides:

1. LangSmith's retention cliff is the sharpest cost mechanic in the field. The official billing documentation prices a base trace at .05¢ per trace ($0.50 per 1,000) with 14-day retention, and an extended-retention trace at .50¢ per trace ($5.00 per 1,000) with 400-day retention 10. That is a 10× multiplier on a retention decision, and 14 days is below the useful window for building eval datasets or investigating a slow-burn quality regression. Model your real retention need, not the base rate: at 5M traces/month, all-extended retention is $25,000/mo, not $2,500.
2. Published secondary sources on LangSmith pricing are wrong. Several 2026 pricing round-ups state $2.50 per 1k base traces and $5.00 per 1k extended. The primary billing documentation says .05¢ and .50¢ per trace, or $0.50 and $5.00 per 1k 10. The extended figure agrees; the base figure is off by 5×. This is a live example of why the sticker numbers in this report are computed from primary sources only.
3. Langfuse Cloud bills "units," not traces, and units track observations/spans. At 8 spans per trace, a 5M-trace month is a 40M-unit month, 8× whatever you estimated from trace counts. Graduated rates run $8.00/100k below 1M down to $6.00/100k above 50M 6. Worked: 900k units at $8.00/100k ($72) + 9M at $7.00/100k ($630) + 30M at $6.50/100k ($1,950) = $2,652/mo usage, plus the $199 Pro platform fee. Anyone budgeting Langfuse Cloud from trace volume rather than span volume will underestimate by nearly an order of magnitude.
4. Self-hosted Langfuse is the heaviest infrastructure footprint here: Postgres, ClickHouse, Redis/Valkey, S3-compatible object storage, plus separate web and worker containers 26. Docker Compose is explicitly not production-grade (no HA, scaling, or backup); production means the Helm chart or a cloud-specific deployment. There is also a sharp operational gotcha: all components must run in UTC, or queries return incorrect or empty results 26. The free license is genuinely free; the operational cost is not zero, and it is higher than Phoenix's.
5. Judge inference tokens are a real line item and are identical across all five. Scoring 500k traces/month with a small model is $500–2,000/mo of provider spend that no platform choice avoids. At high scoring rates this can exceed the observability platform bill entirely, which means "cheapest platform" is often the wrong optimization target. Sample rates matter more than vendor choice.

Risk and strategic outlook#

Langfuse: the ClickHouse acquisition is the headline risk. ClickHouse acquired Langfuse on 2026-01-16, alongside a $400M Series D that valued ClickHouse at $15B 18. The stated commitment, quoted verbatim, is that "Langfuse remains 100% open-source under its existing MIT license for core features which allows for self-hosting at production scale," that "Langfuse Cloud continues operating as a standalone service," and that "community-first development continues with the same transparency and openness" 18 verified 2026-08-13. Note the qualifier: the commitment is scoped to core features, which is consistent with, and leaves room for, the existing EE-gated governance list.

The strategic logic is coherent rather than defensive: Langfuse already runs on ClickHouse, so this is a vertical integration of an existing dependency, not an acqui-hire. Adoption backs the position: 33.0k GitHub stars, 3,554 contributors 27, and reported 26M+ SDK installs/month. But seven months is not a track record. The realistic risk is not a license rug-pull; it is gradual gravitational pull toward ClickHouse Cloud, with the EE feature list quietly widening over time. Watch what moves into the EE-gated list at each release; that list is the canary, and it is published 7.

MLflow: the Databricks gravity well. Apache-2.0, Linux Foundation association, 27.5k stars, 6.1k forks, 60M+ monthly downloads, and daily commits 427. Genuinely the safest governance profile. The strategic risk is not abandonment but asymmetric investment: the most valuable GenAI production capabilities keep landing on the Databricks side of the line 15. If you adopt OSS MLflow for GenAI observability, assume the OSS/managed gap widens rather than narrows, and that the pitch to migrate to Databricks gets more compelling each release. That is a fine trade if you are already a Databricks shop and an uncomfortable one if you are deliberately cloud-agnostic.

Phoenix and Arize: the open-core boundary is moving in the wrong direction. Phoenix is developed at genuinely impressive velocity (v19.21.0 → v20.1.0 in three days 2) and 11.0k stars is healthy. But the two capability gaps found here (real-time online evals with alerting, and native OTel GenAI semantic convention support) both resolve to "upgrade to Arize AX" 1720. When the open standard support lands in the paid tier first, that is a signal about where the open-core line will sit in future. ELv2 rather than a true OSI license reinforces the read.

Braintrust: single-vendor, well-architected. SOC 2 Type II, HIPAA, GDPR, and the hybrid VPC data plane genuinely de-risk the data-residency question in a way no other commercial option here matches 22. The residual risks are commercial, not technical: a private company with no open-source fallback, custom Enterprise pricing, and consumption billing on processed GB and scores that scales with trace verbosity, a dimension your engineers control casually and your finance team does not. There is no escape hatch if terms change; exiting means re-instrumenting or replaying OTLP into another backend.

LangSmith: deepest product, narrowest terms. LangChain's ecosystem position is not in doubt and the LangGraph integration is best-in-class. The strategic concern for this context is structural: everything that makes it viable in a regulated, cloud-agnostic enterprise (self-hosting, hybrid deployment, custom retention) sits behind an Enterprise contract with no self-serve path and no published pricing 9. That is a procurement dependency at the observability layer, and observability is the thing you most need working during an incident.

Reversibility, ranked. Because all five ingest OTLP, exit cost is lower than it looks if you instrument through OpenTelemetry rather than a vendor SDK: you re-point an exporter. That is the single most important architectural decision in this whole comparison and it is worth more than the platform choice itself. What does not port is everything built on top of the traces: curated datasets, judge definitions, prompt version history, annotations, dashboards. Roughly ordered by exit cost, cheapest first: Phoenix ≈ MLflow (self-hosted, your data) < Langfuse (MIT, your database, full API) < Braintrust (your VPC, their schema) < LangSmith (their cloud, their schema, contract-bound).

When to pick which#

• Pick Langfuse if you want one paved road across multiple teams, need production evaluation running continuously without a per-trace meter, and can self-host on Kubernetes. Budget for the Enterprise licence key from day one if you are regulated: audit logs, server-side masking, retention policies and project-level RBAC are all EE, and you will need all four. Accept the four-service infrastructure footprint as the price of the free licence.
• Pick Braintrust if you are comfortable with a commercial contract, your evaluation loop is the centre of gravity (frequent experiments, prompt iteration, human review), and the VPC-resident data plane satisfies your residency requirements. It matches Langfuse on product and beats it on compliance architecture; you are paying for that with lock-in and consumption billing.
• Pick MLflow if you already run it for classical ML, your workloads are JVM-heavy or ADK-based, and your immediate need is trace capture plus offline evaluation in CI. It is the best answer to "do I need a new tool" for the pre-production half of the lifecycle, and it is the only option that traces Spring AI and Quarkus LangChain4j. Pair it with something else for production monitoring, or accept building that yourself.
• Pick Phoenix if you want the lightest self-hosted footprint, your team works notebook-first, and scheduled batch evaluation is sufficient. It is the cheapest and simplest to operate. Go in knowing you will instrument to OpenInference rather than the open standard, and that real-time online evals mean Arize AX.
• Pick LangSmith if you are a LangGraph shop, you can use the managed cloud, and residency is not a constraint: it is the best product for that specific situation and the scorecard's weighting is what demotes it, not its quality. If you need self-hosting, get the Enterprise quote before designing around it.
• Any of them is fine if you are running a single team, under ~500k traces/month, pre-production, and not yet regulated. At that scale every option here is adequate and free-tier-viable; the decision does not deserve the effort. Instrument through OpenTelemetry, pick the one your team finds pleasant, and revisit when volume or compliance forces the issue.

The split-stack option, and the "category error" question#

The question that prompted this report was whether an existing MLflow deployment covers enough to avoid adding a dedicated tool, and whether unifying classical ML tracking with agent tracing in one system is sound or a category error.

It is not a category error, but the seam falls in a different place than expected. Traces, runs, prompts, models and evaluation metrics genuinely do share a lineage graph, and MLflow 3's linking of them is coherent rather than forced. The evidence that this is sound design is that Databricks built a production monitoring service on top of that model rather than beside it 15. The unification is fine.

The seam is between offline and online, not between classical ML and GenAI. MLflow's design centre is the experiment: a bounded run, evaluated after the fact, recorded for lineage. Production observability is a streaming, always-on, sampled, alert-driven workload, closer to an APM system than an experiment tracker. That is why continuous production monitoring and production-scale trace ingestion sit on the Databricks side of the line 15: it is not licence gating for its own sake, it is a genuinely different system with different scaling properties, and Databricks built it where it could charge for the infrastructure.

So the honest answer for a team already running MLflow: keep it, and do not expect it to cover production. Use MLflow for what it is uniquely good at here: trace capture across the widest framework set including JVM and ADK 14, the model/prompt/trace lineage graph, offline evaluation in CI via the pytest integration, and the registry role that the Ray Serve in the MDLC report identified as unfilled by the serving layer. Add Langfuse (or Braintrust) for continuous production evaluation and monitoring. Both consume OTLP, so a single OpenTelemetry instrumentation layer can fan out to both, which is exactly the architecture the OTel-neutrality dimension was scoring, and the reason it earned 10 points of weight.

This is the same shape as the ADK report's conclusion: one paved road, a sanctioned second tool for the workloads the first genuinely cannot serve, and an open interop contract (there MCP and A2A; here OTLP) that keeps them composed rather than siloed.

Watch-outs#

• Instrument through OpenTelemetry, not a vendor SDK. This is the highest-leverage decision in this document and it costs nothing at the start. It converts platform choice from a one-way door into a configuration change, and it is what makes the split-stack option above viable.
• Langfuse does not support OTLP over gRPC: HTTP/JSON and HTTP/protobuf only 12. Check your collector configuration before committing; a gRPC-standardized fleet needs an extra hop.
• Langfuse self-hosting requires UTC on every component. Non-UTC configurations produce incorrect or empty query results 26. This is documented but easy to miss and produces a confusing failure mode.
• LangSmith's 14-day base retention is below the useful window for eval-dataset curation and slow-regression forensics, and upgrading is a 10× per-trace cost 10. Model your actual retention requirement before comparing headline rates.
• Langfuse Cloud units ≠ traces. Budget from span volume, not trace counts, or you will be off by roughly your average spans-per-trace 6.
• Phoenix's OTel neutrality is partial. Plan for a translation layer if your instrumentation emits gen_ai.* conventions, particularly relevant if you run Google ADK, which emits them natively 16.
• The OTel GenAI conventions are still moving. Agent and tool-orchestration attributes are the least settled part of the spec, which is exactly the part agent observability depends on. Prefer backends that map multiple conventions, and expect instrumentation churn over the next several quarters unverified, see §3.
• The MLflow OSS/Databricks line will move. Check it against current documentation at adoption time rather than trusting this report's snapshot; the trend is toward more capability landing on the managed side 15.
• Judge inference cost can exceed platform cost. Set sampling rates deliberately at design time; this is a bigger lever on total spend than the vendor decision.
• Shelf life: short. Phoenix shipped four releases in three days during this research 2; Langfuse rewrote both SDKs and its data model in March 2026; MLflow ships monthly. Re-verify anything version-gated or price-gated after roughly two quarters.

Methodology and sources#

Produced by live research against primary sources on 2026-08-12 and 2026-08-13: package registries for versions and licences, official pricing and documentation pages for commercial terms, vendor documentation for capability claims, and the GitHub API for adoption metrics.

The weighted scorecard is tuned to the stated context (regulated, multi-team, cloud-agnostic, self-host-preferred) and every weight is published so a reader who disagrees can recompute. Claims are tagged by confidence. Where a claim could not be confirmed against a primary source in this session it is marked [unverified] rather than smoothed over; this applies specifically to the OpenTelemetry GenAI stability status in §3.

Cost figures are modelled estimates from verified public unit prices, not quotes. Infrastructure and ops-effort ranges are engineering estimates and are the least certain numbers in the document.

One correction worth surfacing, as a caution about secondary sources on this topic: several 2026 pricing round-ups report LangSmith base traces at $2.50/1k. The primary billing documentation states .05¢ per trace, or $0.50/1k 10. All prices in this report are taken from vendor-primary pages.

1. arize-phoenix, package metadata. PyPI JSON API. https://pypi.org/pypi/arize-phoenix/json (accessed 2026-08-13)
2. Releases, Arize-ai/phoenix. GitHub. https://github.com/Arize-ai/phoenix/releases (accessed 2026-08-13)
3. langfuse, package metadata. PyPI JSON API. https://pypi.org/pypi/langfuse/json (accessed 2026-08-13)
4. mlflow, package metadata. PyPI JSON API. https://pypi.org/pypi/mlflow/json (accessed 2026-08-13)
5. MLflow Releases. MLflow. https://mlflow.org/releases/ (accessed 2026-08-13)
6. Pricing. Langfuse. https://langfuse.com/pricing (accessed 2026-08-13)
7. Self-Hosting: License Key. Langfuse. https://langfuse.com/self-hosting/license-key (accessed 2026-08-13)
8. Self-Hosted Pricing. Langfuse. https://langfuse.com/pricing-self-host (accessed 2026-08-13)
9. LangSmith Pricing. LangChain. https://www.langchain.com/pricing-langsmith (accessed 2026-08-13)
10. Usage and billing. LangChain Docs. https://docs.langchain.com/langsmith/usage-and-billing (accessed 2026-08-13)
11. Pricing. Braintrust. https://www.braintrust.dev/pricing (accessed 2026-08-13)
12. OpenTelemetry (OTEL) for LLM Observability. Langfuse. https://langfuse.com/integrations/native/opentelemetry (accessed 2026-08-13)
13. MLflow Tracing. MLflow Docs. https://mlflow.org/docs/latest/genai/tracing/ (accessed 2026-08-13)
14. Tracing Integrations. MLflow Docs. https://mlflow.org/docs/latest/genai/tracing/integrations/ (accessed 2026-08-13)
15. Open source vs. managed MLflow on Azure Databricks. Microsoft Learn and Databricks, page dated 2026-08-05. https://learn.microsoft.com/en-us/azure/databricks/mlflow3/genai/overview/oss-managed-diff (accessed 2026-08-13)
16. Translating Semantic Conventions. Arize Phoenix Docs. https://arize.com/docs/phoenix/tracing/concepts-tracing/translating-conventions (accessed 2026-08-13)
17. Arize AX adds native support for OpenTelemetry GenAI semantic conventions. Arize AI. https://arize.com/blog/arize-ax-opentelemetry-genai-semantic-conventions/ (accessed 2026-08-13)
18. ClickHouse welcomes Langfuse: The future of open-source LLM observability. ClickHouse. https://clickhouse.com/blog/clickhouse-acquires-langfuse-open-source-llm-observability (accessed 2026-08-13)
19. OpenTelemetry Gen AI Semantic Conventions Support, issue #10622. Arize-ai/phoenix. https://github.com/Arize-ai/phoenix/issues/10622 (accessed 2026-08-13)
20. Pricing. Arize AI. https://arize.com/pricing/ (accessed 2026-08-13)
21. Security. Langfuse. https://langfuse.com/security (accessed 2026-08-13)
22. Our approach to hybrid deployment, and Security. Braintrust. https://www.braintrust.dev/blog/hybrid-deployment and https://www.braintrust.dev/docs/security (accessed 2026-08-13)
23. Evaluation concepts, covering online evals and automation rules. LangChain Docs. https://docs.langchain.com/langsmith/evaluation-concepts (accessed 2026-08-13)
24. Trace with OpenTelemetry. LangChain Docs. https://docs.langchain.com/langsmith/trace-with-opentelemetry (accessed 2026-08-13)
25. OpenTelemetry (OTel). Braintrust Docs. https://www.braintrust.dev/docs/integrations/sdk-integrations/opentelemetry (accessed 2026-08-13)
26. Self-Hosting. Langfuse. https://langfuse.com/self-hosting (accessed 2026-08-13)
27. Repository metrics for Arize-ai/phoenix, mlflow/mlflow, langfuse/langfuse, langchain-ai/langsmith-sdk and braintrustdata/braintrust-sdk. GitHub REST API. https://docs.github.com/en/rest/repos/repos (accessed 2026-08-13)
28. Changelog, covering the v4 data-model rewrite of March 2026 and Experiments as a top-level feature in April 2026. Langfuse. https://langfuse.com/changelog (accessed 2026-08-13)
29. LLM-as-a-Judge, online evaluation on production traces. Langfuse Docs. https://langfuse.com/docs/evaluation/evaluation-methods/llm-as-a-judge (accessed 2026-08-13)
30. Semantic Conventions for Generative AI. OpenTelemetry. https://github.com/open-telemetry/semantic-conventions-genai (accessed 2026-08-13)