Platform · Quality
Observability: every LLM call, traced and scored
Every LLM call your VegaDūta agents make is traced to Langfuse over OpenTelemetry, tagged with the user, session, and metadata that let you reconstruct exactly what happened. On top of the traces, each agent carries a quality score with history, fed by sampled LLM-as-judge evaluations that grade real interactions and write scores back. When a call fails or behaves strangely, a dedicated LLM-call diagnosis surface helps you work out why.
To be precise about scope: this is in-product observability. Connecting your own Langfuse instance is not a supported customer feature today.
Tracing built in, not bolted on
Tracing is not an SDK you remember to add — the platform emits it for every LLM call automatically, over OpenTelemetry into Langfuse. Traces carry user, session, tags, and metadata, so "what did the agent actually send, and what came back?" is always answerable, per conversation.
Quality scores with history
Each agent has a quality score tracked over time, not a one-off grade. That history is what makes changes accountable: adjust a prompt, swap a model, or add a tool, and you can see whether quality moved — and in which direction — instead of relying on vibes.
LLM-as-judge, on real traffic
A sample of live interactions is evaluated by an LLM judge, and the resulting scores feed back into each agent's quality history. Sampling keeps evaluation continuous and affordable, and grading real traffic means the scores reflect what your users actually experience rather than a synthetic test set.
Diagnosing a failing call
When an LLM call fails, the diagnosis surface examines the call and reports what went wrong — a malformed request, a provider-side error, a truncation — so debugging starts from evidence instead of guesswork. One honest boundary: observability lives in the product, and bringing your own Langfuse instance is not currently supported.
Frequently asked questions
How does VegaDūta trace LLM calls?
Every LLM call is exported to Langfuse over OpenTelemetry automatically, with user, session, tags, and metadata attached. There is no instrumentation for you to add — tracing is part of the platform's own call path.
How does VegaDūta measure agent quality?
Each agent has a quality score with history, fed by sampled LLM-as-judge evaluations of real interactions. Because scores accumulate over time, you can see whether a prompt or model change actually improved the agent.
Can I debug why a specific LLM call failed?
Yes. The LLM-call diagnosis surface analyzes a failing call and reports the likely cause, such as a provider error or a malformed request, alongside the full trace in Langfuse. That turns debugging into reading evidence rather than reproducing blind.
Can I connect my own Langfuse instance to VegaDūta?
No — bring-your-own-Langfuse is not a supported customer feature today. Tracing and evaluation are provided in-product, and the trace data is used by the platform's own quality scoring and diagnosis tooling.
See it working in two minutes
The sandbox provisions a real tenant — describe an agent in one sentence and test it, no account, no card. Or browse ~90 industry workflow recipes to see what teams build.
