Tracing reference

Everything beyond the basics in Tracing your agent: step types, marking blocks of code, common patterns, configuration, and what the SDK sends.

Steps and their types

Each step has a type, which sets how it is shown and filtered. Names are free text, such as retry 2 – fallback model or tool:send_email, and can repeat within a trace; each step has its own id.

Marking a block of code

To record part of a function as a step, use the block form. Anything recorded inside it nests under it.

with fb.start_as_current_observation("plan", as_type="chain", input={"q": question}) as plan:
  steps = make_plan(question)
  plan.update(output={"steps": steps}, metadata={"planner": "v2"})

const steps = await forgebench.startActiveObservation(
"plan",
async (plan) => {
  const steps = await makePlan(question);
  plan.update({ output: { steps }, metadata: { planner: "v2" } });
  return steps;
},
{ asType: "chain", input: { q: question } },
);

try (Observation plan = tracing.startActiveObservation(ObservationType.CHAIN, "plan")) {
  plan.input(Map.of("q", question));
  List<String> steps = makePlan(question);
  plan.output(Map.of("steps", steps)).metadata("planner", "v2");
}

curl -X POST https://api.forgebench.ai/v1/observations/batch \
-H "Authorization: Bearer $FORGEBENCH_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "observations": [
    {"id": "b7ad6b7169203331", "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
     "type": "chain", "name": "plan",
     "input": {"q": "Should I take an umbrella?"},
     "output": {"steps": ["check the weather", "answer"]},
     "metadata": {"planner": "v2"},
     "start_time": "2026-10-21T09:00:00.000Z", "end_time": "2026-10-21T09:00:00.400Z"}
  ]
}'

Patterns

Loops, retries and fallbacks

Give each attempt its own step. A failed attempt is recorded as an error with the next attempt beside it, so the trace shows how the run recovered.

for attempt in (1, 2, 3):
  try:
      with fb.start_as_current_observation(f"attempt {attempt}") as step:
          return call_model(prompt)
  except TimeoutError:
      continue

for (const attempt of [1, 2, 3]) {
try {
  return await forgebench.startActiveObservation(`attempt ${attempt}`, () => callModel(prompt));
} catch (e) {
  if (!(e instanceof Error && e.name === "TimeoutError")) throw e;
}
}

for (int attempt = 1; attempt <= 3; attempt++) {
  try {
      return tracing.observe(ObservationType.SPAN, "attempt " + attempt, () -> callModel(prompt));
  } catch (TimeoutException e) {
      // recorded as an error on this attempt's step; try the next one
  }
}

Errors

An exception inside a step marks it ERROR with the exception's type and message, then is re-raised unchanged. To flag a step without failing it, set its level, for example update(level="WARNING", status_message="fell back to the small model") (Python), update({ level: "WARNING", statusMessage: "..." }) (TypeScript) or .level(ObservationLevel.WARNING).statusMessage("...") (Java).

Users and conversations

update_current_trace (Python), updateActiveTrace (TypeScript) and updateCurrentTrace (Java) set fields on the whole trace:

Parallel work

Steps that run at the same time each keep the right parent.

• Python: asyncio tasks need nothing. For a thread pool, wrap the function: pool.map(bind_context(work), items).
• TypeScript: Promise.all and other async code need nothing on Node 20.16 or later, Node 22 and Bun. Other runtimes nest code that runs one step at a time correctly.
• Java: the current step belongs to the thread. Hand work to an executor with Tracing.wrap(task) (or Tracing.wrapSupplier for CompletableFuture.supplyAsync). The async client does this for you.

Work that starts and ends in different places

For callbacks, queue consumers or background jobs, use start_observation (Python) or startObservation (TypeScript, Java). It creates a step that doesn't become the current one; call end() when the work is done. A step that is never ended is never sent.

Continuing a trace

To add steps to a trace started elsewhere, pass its id when starting the first step. A seed derives a stable id from your own request or job id, so both sides reach the same trace without passing the id around:

trace_id = fb.create_trace_id(seed=job_id)          # same seed, same id
with fb.start_as_current_observation("resume", trace_id=trace_id):
  ...

import { createTraceId } from "@seedlinglabs/forgebench-sdk";

const traceId = await createTraceId(jobId);          // same seed, same id
await forgebench.startActiveObservation("resume", async () => {
// ...
}, { traceId });

String traceId = Tracing.createTraceId(jobId);      // same seed, same id
try (Observation resume = tracing.startActiveObservation(ObservationType.SPAN, "resume", traceId)) {
  // ...
}

# The SDKs derive the id as the first 32 hex characters of the seed's SHA-256.
TRACE_ID=$(printf %s "$JOB_ID" | shasum -a 256 | cut -c1-32)   # same seed, same id

curl -X POST https://api.forgebench.ai/v1/observations/batch \
-H "Authorization: Bearer $FORGEBENCH_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "observations": [
    {"id": "5fb397be34d26b51", "trace_id": "'"$TRACE_ID"'", "type": "span", "name": "resume",
     "start_time": "2026-10-21T09:05:00.000Z", "end_time": "2026-10-21T09:05:01.000Z"}
  ]
}'

In TypeScript await createTraceId(jobId) with startActiveObservation(name, fn, { traceId }); in Java Tracing.createTraceId(jobId) with tracing.startActiveObservation(type, name, traceId).

Model calls inside a step

A chat call made through the SDK inside a step needs nothing extra. It is recorded in the same trace, under that step, with the trace's user and session.

Two cases are not nested:

• a call given the trace id of another trace stays in that trace;
• a call from a different client (another API key or workspace) never joins this client's trace or inherits its user.

Configuration

Tracing is on by default. Settings can also come from environment variables; an invalid value is ignored with a warning.

import json

from forgebench import Forgebench, TracingConfig

def mask(data):
  # Runs on every value (a string, or JSON-shaped data) before it is sent.
  return json.loads(json.dumps(data).replace("4111-1111-1111-1111", "****"))

fb = Forgebench(tracing=TracingConfig(sample_rate=0.2, environment="production", mask=mask))

const forgebench = new Forgebench({
apiKey: process.env.FORGEBENCH_API_KEY!,
tracing: {
  sampleRate: 0.2,
  environment: "production",
  // Runs on every value (a string, or JSON-shaped data) before it is sent.
  mask: (data) => JSON.parse(JSON.stringify(data).replace("4111-1111-1111-1111", "****")),
},
});

import ai.forgebench.Forgebench;
import ai.forgebench.tracing.TracingConfig;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.TextNode;

ObjectMapper json = new ObjectMapper();

Forgebench client = Forgebench.builder()
      .tracing(TracingConfig.builder()
              .sampleRate(0.2)
              .environment("production")
              // Runs on every value (as a JsonNode) before it is sent.
              .mask(data -> {
                  try {
                      return json.readTree(data.toString().replace("4111-1111-1111-1111", "****"));
                  } catch (JsonProcessingException e) {
                      return TextNode.valueOf("[masked]");
                  }
              })
              .build())
      .build();

In TypeScript the names are camelCase (sampleRate, flushIntervalMs), and Java uses the same names on TracingConfig.builder(), where mask works on Jackson's JsonNode.

What gets sent

• Inputs and outputs are serialised to JSON. Values that can't be represented become a short description; anything over 64,000 characters is cut with a …[truncated] marker.
• To keep a function's arguments or result private, turn capture off: @observe(capture_input=False, capture_output=False) (Python), { captureInput: false, captureOutput: false } (TypeScript), or don't pass the value (Java).
• mask runs before anything leaves your process, including error messages.
• On the server, credential-shaped strings are always removed. With a PII policy on for the workspace or agent, personal data is removed too, and long text values are shortened to 4,000 characters.
• The agent, key and workspace on each step come from the API key, not from what the SDK sends, so attribution can't be forged.

Short-lived programs

Steps are sent in the background about every second. Scripts, CLIs and serverless functions should flush before exiting: fb.flush() (Python), await forgebench.flush() (TypeScript), client.flush() (Java). Closing the client also flushes. Python and the JVM flush on exit, and Node when its event loop empties, but not on process.exit(), so flush first.

One client per API key

Create one client and reuse it; each runs its own background sender. If your app uses several workspaces (one client per API key), use the client-bound form: client.observe(...) (Python, TypeScript) or client.tracing() (Java). While clients for more than one key are open, module-level observe runs the function untraced and warns once. It works again once only one key is left open.

What Forgebench does with the steps

The SDK posts finished steps to POST /v1/observations/batch, at most 100 per request. Forgebench replies immediately and stores them in the background, so tracing never slows down or fails your agent. A failed send is logged and dropped.

Any API key with chat:write, which every agent key has, can send steps; a read-only key can't. The endpoint only returns how many steps were accepted. You read traces in the console (see Who sees which traces). If the workspace's Observability module is off, steps are accepted and dropped, and the SDK logs why once.

Individual steps can be refused while the rest of the batch is kept. The SDK logs each reason once:

Next