Langfuse

Operator's take

Langfuse slots into the AI engineering stack as the observability and improvement layer — the thing you reach for when an agent pipeline is misbehaving and print() statements aren't cutting it. It records the full execution trace of an LLM call chain: which prompts fired, what tools were invoked, what context went in, what came back, how long each step took, and what it cost. For teams building on agent workflows or RAG pipelines, that trace structure is the practical difference between "something went wrong somewhere" and "the retrieval step returned the wrong chunk on queries containing modal verbs."

The prompt management piece is useful beyond debugging: version your prompts, deploy changes without a code push, and compare performance across versions with actual metrics rather than vibes. The evaluation side — including LLM-as-judge scoring — gives you a feedback loop you can run continuously in production rather than only catching regressions manually. Self-hosting is a real option for teams with data-residency requirements; Langfuse is open core (the core is MIT-licensed, with some enterprise features under a separate commercial license), so you're not locked into the cloud product if your infra team has opinions. One stewardship note worth knowing before you build on it: Langfuse has joined ClickHouse, so the product now sits under that umbrella while continuing to ship as its own platform.

What it's good at

• Nested trace capture — records the full call tree of complex LLM pipelines, with prompts, retrieved context, and model responses at each step; makes root-cause analysis tractable.
• Prompt versioning and deployment — manage prompt versions centrally and push updates without a code release; track performance impact per version.
• Automated evaluation — LLM-as-judge scoring and custom eval pipelines give you continuous quality measurement rather than one-off spot checks.
• Cost and latency analytics — token usage, latency, and spend broken down by model, feature, user, or prompt version; useful for optimization work before costs compound.
• Framework integrations — native integrations with LangChain, LlamaIndex, LiteLLM, CrewAI, OpenAI Agents SDK, Vercel AI SDK, Pydantic AI, Google ADK, and many more; OpenTelemetry support for Go, Java, and other languages.
• Self-hosting option — open core: the core is MIT-licensed (some enterprise features sit under a separate commercial license); deployable on your own infra for teams that can't send LLM inputs and outputs to a third-party cloud.

What it's not

• Not for teams that haven't shipped an LLM app yet — if you're still prototyping and haven't hit a real debugging wall, the value isn't there yet; add it when traces would actually tell you something.
• Not a no-code or low-code tool — instrumentation requires code changes; the intended user is someone comfortable with Python or TypeScript and a framework SDK.
• Not a replacement for application-level logging — Langfuse covers the LLM layer specifically; you still need your own infra observability for the surrounding system.
• Not unlimited on the free tier — the Hobby plan caps at 50k billable units/month with 30 days of data access and 2 users; production workloads will likely need Core ($29/mo) or above.