DEVELOPMENT TECHNOLOGIES

LLM Observability with Langfuse: Langfuse vs LangSmith

How Langfuse traces, evaluates, and monitors LLM and agent applications, what teams use it for, and how it compares with LangSmith and Arize Phoenix.
Langfuse logo

What Is Langfuse?

Langfuse is an open-source AI engineering platform for LLM observability: tracing, prompt management, datasets, automated evaluations, and metrics for LLM and agent applications. In a Langfuse vs LangSmith comparison, or when reviewing other LangSmith alternatives, teams look at Langfuse for its permissive license, OpenTelemetry foundation, and ClickHouse-backed analytics.

Langfuse was founded in 2023 through Y Combinator and was acquired by ClickHouse in January 2026, keeping its MIT license, self-hosting support, and public roadmap. In June 2025 the maintainers open-sourced the remaining product modules, including LLM-as-a-judge evaluations, annotation queues, prompt experiments, and the playground.

Self-hosted Langfuse is free with unlimited usage and users and includes organization-level RBAC and enterprise SSO. Only some governance features, such as SCIM, audit logs, and project-level RBAC, need an enterprise license. Capabilities are described in the official Langfuse documentation.

The managed Langfuse Cloud is SOC 2 Type II and ISO 27001 certified, offers data regions in the EU, US, and Japan plus a HIPAA region, and charges by usage units without seat fees on paid plans.

How Langfuse Works

Langfuse ingests OpenTelemetry spans asynchronously, buffers raw events in object storage and Redis, and writes observation and score records into ClickHouse for analytics, while transactional data lives in PostgreSQL. Tracing and evaluation therefore never block your application’s request path. The components are listed in the table below.

Since version 4, released in August 2026, Langfuse uses an observations-centric data model: trace attributes such as user_id and session_id are stored directly on a single wide observations table. Removing joins between trace and span tables makes UI and API queries much faster, especially on large dashboards.

On top of storage sits the evaluation and monitoring layer. It works offline against curated datasets and online against live traffic, using managed LLM-as-a-judge evaluators that also accept images, audio, video, and PDFs, deterministic Python or TypeScript code evaluators, human annotation queues, and user feedback.

Threshold alerts watch any metric, such as event count, average cost, p95 latency, or evaluation score, and notify Slack, webhooks, or GitHub Actions at warning and alert levels.

ComponentRoleNotes
SDKs and OTLP endpointOpenTelemetry-based Python and TypeScript SDKs plus an OTLP receiver.Accepts spans from 100+ frameworks, including LangChain, LlamaIndex, OpenAI SDK, and Pydantic AI.
Web containerServes the UI and ingestion APIs and writes raw events to object storage.Pushes references onto a Redis queue.
Worker containerProcesses queued events, runs evaluations, checks alerts, and writes to ClickHouse.Keeps heavy work off the ingestion path.
ClickHouseStores traces, observations, and scores for analytical queries.Columnar store built for fast aggregation.
PostgreSQLStores users, RBAC, prompt versions, datasets, and alert definitions.Transactional configuration state.
Engineer's Take

We recommend Langfuse when a team wants full ownership of trace data, no per-seat licensing, and an observability layer that does not dictate the agent framework. It works well in multi-model stacks because the SDKs sit directly on OpenTelemetry and need no proxy. Two things surprise teams: on the cloud every trace, span, and score is a billable unit, so a deep agent that emits 25 spans per run costs 25 or more units; and self-hosting means operating PostgreSQL, ClickHouse, Redis, object storage, and two container types. We keep span granularity deliberate and add PII masking in the SDK before the first production trace.

Diatom Enterprises favicon
Diatom engineering team
Langfuse at a Glance
Type

LLM observability and evaluation platform

License

MIT, with self-hosting

Built on

OpenTelemetry, ClickHouse, PostgreSQL

Strengths

Tracing, prompt management, evaluations, alerts

Compared with
Need a reliable tech partner?
20+

Years in business

80+

Senior engineers

200+

Projects delivered

What Langfuse Is Used For

Engineering teams use Langfuse to debug multi-step agents, manage prompts without redeploying, gate releases on evaluation results, audit guardrails, and track token costs per tenant.

Debugging Multi-Step Agents

When an agent built with LangGraph, CrewAI, Pydantic AI, or the Vercel AI SDK loops or hallucinates, engineers inspect the span tree to see tool arguments, retrieved chunks, latency, and tokens at every step.

Prompt Management Without Deploys

Product engineers and domain experts version and compose prompts in a shared playground. Applications fetch prompts by environment label with client-side caching, so prompts can be promoted or rolled back without an application release.

CI/CD Quality Gates

Before merging changes to an agent or retrieval pipeline, GitHub Actions run dataset experiments scored by code evaluators and LLM-as-a-judge graders, failing the build when quality drops or latency rises.

Guardrail Auditing and PII Masking

Langfuse does not block requests itself. Teams run guardrail libraries such as LLM Guard or NeMo Guardrails in the request path, record each check as a span, and mask PII in the SDK before data reaches storage.

Langfuse vs LangSmith and Arize Phoenix

Among LLM observability tools, the Langfuse vs LangSmith choice comes down to licensing, data ownership, and billing model, while Arize Phoenix offers a lighter, source-available self-hosted option. Proxy-based tools such as Helicone work in the request path instead.

Criteriontech logo langfuseLangfusetech logo langchainLangSmithtech logo arizeArize Phoenix
License & self-hostingMIT; free self-hosting with unlimited users and SSO; same codebase as the cloud.Proprietary; self-hosting only on enterprise plans.Source-available Phoenix for self-hosting; Arize AX as a commercial cloud.
Storage & ingestionClickHouse, PostgreSQL, Redis, and object storage; async OpenTelemetry ingestion.Proprietary platform built on ClickHouse, PostgreSQL, Redis, and blob storage; SDK and OTLP ingestion.SQLite or PostgreSQL for Phoenix.
Billing modelUsage units (traces, spans, scores); no seat fees on paid cloud plans.Per seat plus per trace, with extra cost for extended retention.Phoenix free to self-host; Arize AX billed by spans and data.
RetentionLong retention on paid cloud plans without per-trace fees.Short base retention; extended retention billed per trace.Plan-dependent on Arize AX.
EvaluationsLLM-as-a-judge, code evaluators, annotation queues, and alerts on all plans.Online and offline evaluators.Evaluators via OpenInference; custom code evaluators gated on Arize AX.
Framework couplingFramework-neutral; 100+ integrations through OTLP.Closely tied to the LangChain ecosystem and its deployment platform.OpenInference instrumentation across frameworks.
Swipe the table to compare →

When to Choose Langfuse

Choose Langfuse when you want an MIT-licensed platform you can self-host without feature gates, or a managed cloud with long trace retention, no seat fees, and native OpenTelemetry across any mix of LLM SDKs. It also fits teams that need deterministic code evaluators and alerts without an enterprise contract.

When Langfuse Is Not the Right Fit

If you want a lightweight single-container viewer and lack the capacity to operate PostgreSQL, ClickHouse, Redis, and object storage, self-hosted Arize Phoenix is simpler. If you want agent hosting, sandboxes, and a gateway from one vendor, or your traces routinely contain many spans so per-unit billing erodes the price advantage, LangSmith may fit your cost and operating model better.

How We Approach Langfuse Projects

We build an LLM evaluation framework and observability pipeline by instrumenting OpenTelemetry spans at the application boundary, masking sensitive data before ingestion, and wiring evaluations into CI/CD and production alerting.

Deployment Topology

Teams with residency or healthcare requirements use Langfuse Cloud in the EU, US, Japan, or HIPAA region. When data cannot leave a private network, we deploy the self-hosted stack with ClickHouse, PostgreSQL, Redis, and object storage, connected to the company’s SSO.

Span Taxonomy and LLM Guardrails

Services are instrumented asynchronously instead of routing traffic through a proxy. Guardrail libraries run in the request path to block prompt injection, each check is logged as a span, and client-side masking strips PII before records are stored.

Offline and Online Evaluation

Prompts are managed with environment labels. GitHub Actions run dataset regression suites on every pull request, and production monitors alert Slack when p95 latency, average cost, or evaluation scores cross thresholds.

Sound like the right partner?

Tell us about your project. We'll get back to you within 1 business day.

Frequently Asked Questions

Direct answers to common architectural, licensing, and cost questions when evaluating Langfuse for production AI systems.

Can I self-host Langfuse for free, and did the ClickHouse acquisition change this?

Yes. Langfuse is free to self-host under the MIT license with unlimited usage and users, organization-level RBAC, and enterprise SSO. The January 2026 acquisition by ClickHouse did not change the license, self-hosting support, or public roadmap. Only governance features such as SCIM, audit logs, project-level RBAC, and automated retention policies require an enterprise license.

How does Langfuse billing differ from LangSmith?

Langfuse Cloud bills in units, where each trace, observation (span), and evaluation score counts as one unit, with no seat fees on paid plans. LangSmith charges per seat and per trace, with extra charges for extended retention. For typical workloads Langfuse is usually cheaper, but because every span is a unit, the gap narrows for deep agent traces with many spans. Check the Langfuse pricing page for current rates.

Does Langfuse require LangChain?

No. Langfuse is framework-neutral and accepts traces from more than 100 frameworks and providers through its OpenTelemetry-based Python and TypeScript SDKs and its OTLP endpoint, including the OpenAI SDK, Vercel AI SDK, LlamaIndex, CrewAI, Pydantic AI, LiteLLM, and LangGraph.

How do you implement LLM guardrails and PII masking with Langfuse?

Langfuse traces asynchronously outside the request path, so blocking LLM guardrails run in your application with libraries such as LLM Guard, NVIDIA NeMo Guardrails, Lakera, or Azure AI Content Safety, and Langfuse records each check and runs follow-up evaluations. To keep PII out of storage, add client-side masking functions in the SDK export path, or server-side ingestion masking on self-hosted enterprise deployments.

Does Langfuse support code evaluators and production alerting?

Yes. Langfuse supports deterministic Python or TypeScript code evaluators and multimodal LLM-as-a-judge evaluators on every plan. Threshold alerts, introduced in June 2026 and included in version 4, watch any observation or score metric, such as count, average cost, p95 latency, or evaluation score, and notify Slack, webhooks, or GitHub Actions.

The People Behind

A team of skilled engineers, architects, and delivery managers, dedicated to making every engagement a success.

Let's discuss your project