OpenTelemetry Standardizes LLM Tracing: Implementation Guide for GenAI Semantic Conventions

OpenTelemetry just standardized LLM tracing. Here’s what it actually looks like in code.

OpenTelemetry has released experimental GenAI Semantic Conventions to standardize how LLM spans are named and attributed across different tools. This specification addresses the fragmentation where every LLM tool, such as Langfuse or Helicone, previously used incompatible custom tracing formats.

Why This Matters

The transition to standardized GenAI tracing resolves the ‘walled garden’ problem where traces were only visible in specific vendor dashboards. By adopting these conventions, engineering teams can switch between backends like Datadog, Arize Phoenix, and Jaeger without reconfiguring their entire instrumentation layer. Failure to align with these standards leads to vendor lock-in and invisible traces when metadata is stored in reserved namespaces or incorrect paths.

Key Insights

• Span naming must follow the {operation} {name} format, such as chat gpt-4o or execute_tool web_search, to be recognized by GenAI-aware backends.
• Tool attributes must be placed at the gen_ai.tool.* level rather than nested under agents, as seen in the toad-eye v1 to v2 migration.
• The OTel spec mandates that instrumentations SHOULD NOT capture prompt or completion content by default to prevent PII leaks, requiring explicit opt-in.
• A gap analysis shows that OTel covers the ‘what’ of an event, while custom namespaces like gen_ai.toad_eye.cost are still required for ‘how much’ metrics.
• The OTel NodeSDK silently disables trace export if spanProcessors is passed as an empty array, a pitfall that can lead to passing tests with zero actual observability.

Working Examples

Applying the official GenAI Semantic Conventions for an agent tool call.

// Standardized Span Naming and Attributes
span.setAttribute("gen_ai.operation.name", "chat");
span.setAttribute("gen_ai.request.model", "gpt-4o");
span.setAttribute("gen_ai.agent.name", "weather-bot");
span.setAttribute("gen_ai.tool.name", "search");
span.setAttribute("gen_ai.tool.type", "function");

Migration strategy to support both legacy and standardized attributes during a version transition.

// Dual-emit approach for backward compatibility
// New (OTel spec-compliant)
span.setAttribute("gen_ai.tool.name", toolName);
// Old (deprecated, still emitted for backward compat)
span.setAttribute("gen_ai.agent.tool.name", toolName);

Practical Applications

• System: toad-eye v2.4. Behavior: Implements dual-emission of both old and new attribute names controlled by the OTEL_SEMCONV_STABILITY_OPT_IN environment variable.
• Pitfall: Using custom span names like gen_ai.openai.gpt-4o. Consequence: The span becomes invisible to GenAI-aware backends that expect the chat {model} format.
• Use Case: Privacy-first instrumentation where prompt recording is disabled by default, only enabling JSON string capture via gen_ai.input.messages when explicitly configured.

References:

• https://dev.to/vola-trebla/opentelemetry-just-standardized-llm-tracing-heres-what-it-actually-looks-like-in-code-2e5f