Observe your voice agent

Forgebench observes voice agents but doesn't host them. Your LiveKit Agents, Pipecat or OpenAI Agents SDK pipeline runs on your infrastructure and points its speech services at Forgebench. To govern the same calls, see Govern your voice agent.

What you get

1. Group calls into conversations and turns

Send the conversation key on every call.

from forgebench import voice_headers

# X-Forgebench-Session, X-Forgebench-Turn[, traceparent]
headers = voice_headers("call-8123", turn=3)

import { voiceHeaders } from "@seedlinglabs/forgebench-sdk";

// X-Forgebench-Session, X-Forgebench-Turn[, traceparent]
const headers = voiceHeaders("call-8123", { turn: 3 });

import ai.forgebench.Trace;
import java.util.Map;

// X-Forgebench-Session, X-Forgebench-Turn[, traceparent]
Map<String, String> headers = Trace.voiceHeaders("call-8123", 3);

# Send the same headers on each call of the turn, for example Speech Synthesis
curl -X POST https://api.forgebench.ai/v1/audio/speech \
-H "Authorization: Bearer $FORGEBENCH_API_KEY" \
-H "X-Forgebench-Session: call-8123" \
-H "X-Forgebench-Turn: 3" \
-H "Content-Type: application/json" \
-o reply.mp3 \
-d '{"model": "gpt-4o-mini-tts", "input": "How can I help?", "voice": "alloy"}'

Choose one level, best first:

1. traceparent: your OpenTelemetry trace and ours join. Python voice_headers() adds it from the current span when opentelemetry is installed; in TypeScript or Java, pass traceparent. Requires an API key.
2. Explicit turn: X-Forgebench-Session plus X-Forgebench-Turn. One trace per turn, no OpenTelemetry needed. On WebSockets, use the query parameters session, turn and traceparent, since browsers can't set headers.
3. Session only (X-Forgebench-Session, metadata.session, or chat session_id): turns are inferred. A new turn starts with a transcription call after the agent answered, or after a gap over 20 s. Two pipelines sharing a session key can be mis-grouped.
4. Nothing: one trace per call.

2. Send your pipeline's telemetry (LiveKit, Pipecat)

Point your OTLP/HTTP trace exporter at https://<control-plane>/v1/telemetry/otlp/voice/v1/traces (protobuf or JSON, gzip accepted). Use the same API key your agent uses for /v1/audio/* (chat:write scope; a telemetry:ingest key isn't accepted). The route returns 404 while voice telemetry ingest isn't enabled.

OTEL_EXPORTER_OTLP_ENDPOINT=https://<control-plane>/v1/telemetry/otlp/voice
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <gateway key>

• LiveKit Agents: build a TracerProvider with OTLPSpanExporter(endpoint=..., headers=...) and pass it to set_tracer_provider(...). Set the span attribute forgebench.session (or rely on gen_ai.conversation.id).
• Pipecat: setup_tracing(service_name=..., exporter=OTLPSpanExporter(...)) and additional_span_attributes={"forgebench.session": "<key>"} (session.id and conversation.id also work).

Forgebench reads the session key from forgebench.session, session.id, gen_ai.conversation.id or conversation.id, in that order. Spans with none still reach Langfuse (joined by trace id if you propagate traceparent) but don't appear in the console.

What we keep and what we drop

• No audio. Audio attributes, bytes values and base64 or data: blobs are dropped.
• No text unless you turn it on. voice_trace_content (Settings → Voice → Record text on traces, default off) covers both Forgebench's own text and your pipeline's. While it's off, all lk.pii.* attributes and Pipecat transcript, text, input, output and tools are dropped. When on, secrets are always stripped, and PII is handled per your guardrail policy.
• Allowlisted attributes only. Attributes naming a workspace, and langfuse.* fields, are ignored. Your spans carry no usage or cost, so nothing is counted twice.
• Limits: 8 MiB per request (64 MiB decompressed), 2000 spans per request, 64 attributes per span, 20 requests/s per key. Spans older than 24 h or more than 5 min in the future are dropped.

3. Reading the numbers

• Response gap (response_gap_ms) runs from the final transcript to the first TTS byte. It includes your LLM time and network, but not your end-of-utterance detection or playout. The timeline shows it as its own row, since it already contains STT, LLM and TTS time.
• End of utterance to first audio (e2e_first_audio_ms) is only what your pipeline reports (LiveKit lk.e2e_latency, Pipecat turn.user_bot_latency_seconds), or the sum of the parts it reports. The console labels it "customer-reported" and never merges it with response gap.
• TTS TTFB is flagged buffered while Forgebench buffers synthesis end to end; it then equals full synthesis time. The tile excludes buffered values unless you include them.
• Usage in Langfuse: STT shows audio_ms (integer milliseconds), TTS shows characters, and total is always 0. Cost is the exact ledger cost.
• Scores in Langfuse: voice.response_gap_ms, voice.tts_ttfb_ms, voice.stt_final_lag_ms and voice.e2e_first_audio_ms, at most once per turn.
• Utterance spans: one stt.utterance span per final transcript, up to 200 per connection, then one overflow span.
• Run ids that are UUIDs appear in hex form in Langfuse.

4. Retention and erasure

Console voice data is kept for 90 days by default. An admin can erase one conversation with DELETE /v1/voice/conversations/{session_key}, since a session key can identify a person.

Troubleshooting

• No turns grouped: send the session header or traceparent.
• TTFB flagged buffered: expected until Forgebench streams synthesis.
• No console rows from the pipeline: the spans need a session key (section 2).
• Verify a joined trace: the audio response header X-Forgebench-Trace-Id (chat responses use X-Trace-Id) equals the traceparent trace id you sent.

Next