Langfuse, LangSmith, Arize Phoenix, and Self-Hosted LLM Observability Compared

Which LLM observability tool keeps regulated prompts and completions inside the customer VPC? All four stacks a regulated buyer shortlists can run on infrastructure the client controls, but they differ on where traces land, how audit logs leave the box, and three-year cost.

Langfuse, LangSmith, Arize Phoenix, and a raw OpenTelemetry pipeline into Grafana or Splunk make the shortlist. An audit committee cares about five things: deployment model, data residency, SIEM export, license terms, and all-in cost. Vendor marketing collapses on contact with procurement review.

The Four Stacks in Brief

Langfuse is an open-source observability and evaluation platform built for LLM applications; it ships with a hosted tier and a self-host build the customer runs. LangSmith is the observability product from LangChain, offered as SaaS by default with an enterprise Bring Your Own Cloud mode for regulated accounts. Arize Phoenix is a source-available tracing and evaluation library, launched in April 2023, sitting next to Arize AX, the company's commercial platform.

The fourth option is no vendor at all. Instrument with the OpenTelemetry GenAI conventions, which define the gen_ai.operation.name and gen_ai.provider.name attributes and explicitly state that prompt and completion content should not be captured by default, then pipe the spans into whatever Grafana, Jaeger, Splunk, or Elastic stack the security team already runs. The trade-off is engineering burden: nothing LLM-specific comes preassembled.

Deployment Model and Data Residency

Langfuse v3 self-hosted runs entirely inside the customer's network boundary. Two application tiers sit in front of four storage services; every trace, score, and prompt record stays on the customer side of the firewall. The stack does not call back to Langfuse's cloud.

LangSmith BYOC splits the deployment. The control plane (authentication, organization configuration, billing) stays in LangChain's us-east-2 cloud; traces, prompts, datasets, and evaluators sit in an EKS data plane inside the customer's AWS account, with AWS PrivateLink between them. For many regulated buyers the split is acceptable: payloads never cross the control-plane boundary. For a HIPAA covered entity or a federal workload with sole-authority requirements, the fact that account provisioning sits in a vendor AWS account remains a review item. The LLM.co overview of SOC 2, HIPAA, and GDPR for AI walks through how reviewers typically treat that split.

Arize Phoenix installs as a pip package or Docker image and runs locally with no cloud dependency; the commercial Arize AX is a separate SaaS. A raw OpenTelemetry pipeline into Grafana or Splunk keeps the customer entirely inside their own infrastructure, at the cost of owning dashboards, retention, and evaluator code outright.

Instrumentation and the OpenTelemetry Line

The split between proprietary SDK and open spec matters more than it looks. LangSmith leans on its own SDK and span model; migration off it later means re-instrumenting every call site. Langfuse, Phoenix, and the pure OTel path converge on OpenInference, the open specification Arize published, which defines canonical span kinds (LLM, TOOL, AGENT, CHAIN, RETRIEVER, EMBEDDING, RERANKER, GUARDRAIL, EVALUATOR, PROMPT, DECISION) and attributes like input messages, output messages, and token counts. Picking an OpenInference-native stack today gives the instrumentation a better chance of surviving a vendor swap tomorrow.

The weight is not uniform. The LLM kind alone carries the bulk of the attributes defined in the spec; lighter kinds like PROMPT and GUARDRAIL carry a handful. That matters for storage budgeting and for the RAG side of the system; a walkthrough on closing the loop with Langfuse and expert feedback covers the retrieval spans most teams undercount.

1. 1Set gen_ai.operation.name (Required)
2. 2Set gen_ai.provider.name (Required)
3. 3Opt in to prompt content capture (off by default)
4. 4Opt in to completion content capture (off by default)

SIEM Export, Audit Logs, and the Compliance Trail

A procurement review asks one blunt question: can every trace and admin action be forwarded to the enterprise SIEM with the fields the security team already parses? The raw OTel stack has the cleanest answer because every span is an OTLP envelope that Splunk, Elastic, Datadog, and Chronicle already ingest without translation. The vendor stacks all claim OTel compatibility, with caveats.

Langfuse exports traces over OTLP and exposes admin-level audit events through its API; routing both to Splunk or an S3 cold store is a short pipeline job. Phoenix follows the same pattern, since OpenInference rides inside OTel attributes. LangSmith offers export at the enterprise tier, but authentication events live in the vendor-run control plane, so the full audit chain crosses the vendor boundary unless the buyer is on BYOC and accepts the PrivateLink arrangement. Content redaction policy is where enterprise LLM observability earns its keep in a regulated deployment; the LLM.co practice around enterprise LLM security narrows to the same four questions on every intake.

Total Cost Over Three Years

Sticker price is the least useful number here. The three real cost lines are seat and volume fees for SaaS, infrastructure for self-host, and engineering time to build and keep the pipeline running.

LangSmith cloud is seat-based with trace-volume overages; a 150-engineer team routinely crosses into five-figure monthly bills before a BYOC conversation starts. Langfuse self-hosted replaces that with a six-service footprint (Postgres, ClickHouse, Redis, blob storage, plus the Web and Worker containers) and roughly one platform engineer to tend it. Arize Phoenix has the lowest sticker and the narrowest feature surface; buyers who later want hosted evals or a governance workflow typically add Arize AX, moving the cost line back up. Raw OTel plus Grafana has no license fee at all but asks the team to own dashboards, retention tiering, and GenAI-specific evaluator code, which should be priced honestly rather than treated as free.

For most buyers the right exercise is modeling three years of each option against the saved license line, with a platform engineer costed in at a loaded rate. That calculation usually settles the argument before architecture does.

How to Match the Stack to the Review

A healthcare or financial services team under strict data-residency rules, with an existing SIEM and a platform team already in place: pick Langfuse self-hosted or a raw OTel pipeline, keep every byte inside the VPC, and spend the saved license budget on an internal evaluator and a dedicated redaction layer.

A product team that already lives in the LangChain SDK and can live with BYOC: LangSmith is the shortest path, provided the security review accepts the us-east-2 control plane and the PrivateLink arrangement.

A research-heavy team that wants fast local iteration on evals and does not yet need multi-tenant enterprise features: Arize Phoenix on a single VM, with the option of Arize AX later. Any team that has not yet pinned down what belongs on-prem versus in a managed data plane should start with the LLM.co LLM security audit framework before committing to a vendor.