View AI activity in Langfuse

Connect Mockarty to Langfuse to inspect supported AI runs in one place. You can see what an agent did, which tools and models it used, and the token usage reported for its calls. This helps when an answer is surprising or you need to investigate an AI task. An administrator configures one Langfuse project for the whole Mockarty installation.

Both Langfuse Cloud (https://cloud.langfuse.com) and self-hosted Langfuse deployments are supported.

What gets recorded

Mockarty groups recorded activity into traces. A trace is one agent task or chat turn; it can contain:

• Tool steps (spans): tools the agent called, with their inputs, results and errors. Steps delegated to another agent appear beneath the relevant call.
• Model calls (generations): messages sent to the model, its response, model name, token usage, duration and errors.
• Session: groups related chat turns so you can follow a conversation.

For an agent task, the trace ID is its task ID. Delivery happens in the background, so a new trace may take a few seconds to appear. If the delivery buffer fills, some events are dropped; see Reliability.

Enabling the integration

Only an administrator needs to configure Langfuse. The same destination receives traces from all namespaces; each trace carries its namespace so you can filter the results.

Administrator setup

Open Admin → AI & LLM → Agent Settings → LLM Observability (Langfuse):

1. Turn on Enable Langfuse observability.
2. Enter the host and public and secret keys (pk-lf-... / sk-lf-...) from your Langfuse project’s Settings → API Keys. Leave the host empty to use Langfuse Cloud.
3. Choose Metadata only if prompts and responses must not be exported as readable text.
4. Click Save, then Test connection. The test checks the saved keys and enables links from task cards to their Langfuse traces.

While the switch is off, Mockarty does not start new Langfuse traces. There are no per-namespace Langfuse settings.

Metadata only replaces recorded inputs and outputs with a size marker. Traces still include their structure, model names, timings and token counts. Review your metadata and tool names before sending traces to an external service.

The secret key is not displayed again after saving. If your deployment requires encrypted storage for this key, configure Mockarty’s optional PII encryption before entering it.

Using the traces

• From a task card: after a successful connection test, open a recorded agent task in the tasks panel and select Open trace in Langfuse. The GET /api/v1/agent/tasks/{id} response includes the same link in langfuseTraceUrl when one is available.

• From chat: ask the assistant. It has three tools:

  • langfuse_trace_get — the full tree of one task’s spans and generations;
  • langfuse_traces_search — find recent traces by session, user, name or time window;
  • langfuse_costs_summary — per-day, per-model token usage and cost.

  For example: “Show what happened in task 3f2a… and explain why the agent used that tool” or “How many tokens did our model calls use in the last three days?”.

• REST: the same data is available at GET /api/v1/langfuse/traces, GET /api/v1/langfuse/traces/{traceId} and GET /api/v1/langfuse/costs.

curl -H "Authorization: Bearer $MOCKARTY_TOKEN" \
  "http://localhost:5770/api/v1/langfuse/traces?hours=24&limit=10"

Reliability

• Export runs in the background. If the incoming buffer fills, new events are dropped and counted. During a longer outage, the bounded retry buffer drops its oldest events.
• If Langfuse is unreachable, Mockarty pauses delivery and retries with increasing waits, from 30 seconds up to 15 minutes.
• In a cluster, each node sends its own recorded events. Turning the integration off stops new trace creation; events already queued for delivery may still be sent.