Clarion Street

The Trace Is Not a Record. Agent Telemetry Has No Released Standard.

·9 min read

In September I wrote that the first half of the trust layer around agents was mostly solved, on the strength of a survey where 89% of organisations said they had observability and 37% ran online evaluations in production. Six weeks later I wrote about LangSmith Engine v2, which started acting on the traces it collected instead of only displaying them. Both posts were about tooling, and the tooling has held up.

What I did not check was the thing underneath it. A trace that your vendor can render for you is a feature. A record that outlives the vendor, survives a retention window and can be produced in an audit is an artifact with a version number, and the version number for agent telemetry does not exist yet.

In August I argued that a trace has to be complete, tamper-evident and portable before it counts as an audit trail, and I noted that the OpenTelemetry GenAI conventions were the only neutral candidate and were too young, at roughly 300 stars. The repository has 522 stars now. It still has no release.

What OpenTelemetry did in June

On 12 June 2026, OpenTelemetry published semantic conventions v1.42.0. Two things happened in that release. Every gen_ai.* attribute and span was deprecated in place and moved to a dedicated repository. In the same cut, a selection of Kubernetes and container registry resource attributes officially graduated to stable. The project stabilised resource attributes and evicted agent attributes in the same release, which reads less like a verdict on the agent work than a decision about where that work should live.

Five months on, the dedicated repository has 522 stars, 221 open issues, and a push history that runs to this morning. It has never cut a release and never tagged one. The releases page and the tags page are both empty.

Its own manifest is plainer than any commentary. It declares the schema URL gen-ai-dev/1.42.0-dev, and in the same file it depends on core schema 1.45.0. The README goes further. Under a heading that says Schema URL, the body is the word TODO.

I downloaded the model directory on 5 November and counted the stability markers myself. Fifteen YAML files carry the conventions, a sixteenth is the manifest, and across them are 278 stability markers. All 278 read development. Inside gen-ai alone there are 202, including 138 attributes in the registry.

Development status is not a verdict that the conventions are broken. They are published, they are used, and an instrumented service can follow them today. It means there is no compatibility promise: names move between commits and the person absorbing the change is whoever built on top. Agent telemetry works today. What it does not have is a version to pin, so nobody can state what their instrumented code was written against.

Core kept shipping without it

v1.45.0 on 2 November promoted RPC spans and call-duration metrics to stable, including the gRPC and Apache Dubbo conventions, reworked the gRPC and Dubbo spans as refinements of the generic ones, and moved several Kubernetes metrics to release candidate. Nothing in its release notes mentions the GenAI conventions.

A gRPC span in November 2026 has a stable definition, a version, and a compatibility promise. An agent invocation has a namespace that frameworks emit in several generations at once, and a release page that is empty.

If you are trying to read these traces for real, the churn is the first thing you hit. John Hodge keeps a status map of the namespace, last updated 3 November, and states the position without hedging: no tagged release, everything at development, and popular frameworks emitting several generations of the attributes at once. The individual changes he tracks are ordinary design calls made in public. gen_ai.system became gen_ai.provider.name. gen_ai.client.token.usage was removed in September 2026 in favour of per-call histograms. Made one at a time in review, each is sensible. Made at that rate with no release boundary, they mean two teams reading the same telemetry can be looking at different attribute sets and neither is wrong.

The products have already split the question

Langflow’s docs for 1.13.x say its OpenTelemetry output describes “the health of the Langflow service itself”, and then add: “It is not LLM tracing.” The page says Langflow “deliberately withholds prompts, completions, and other flow payloads from this export” and sends anyone who needs prompt-level tracing to a monitoring integration.

I read that as a privacy decision rather than a gap, and the documentation presents it that way. It also means the OpenTelemetry signal from a Langflow deployment tells you the flow ran and the service was healthy, and not what the agent decided.

Dify goes the other way. It ships no native tracing schema and documents per-vendor integrations instead: LangSmith, Langfuse, Opik, W&B Weave, Arize, Phoenix and Alibaba Cloud Monitor, the last of which exports through OpenTelemetry. Seven vendors, seven schemas. Whichever one you wire up becomes the shape of the record you can produce later, and changing your mind means holding two incompatible histories.

Neither product is at fault. Both are relying on something that does not exist, which is a shared definition of what a trace has to contain before anyone can treat it as evidence. The conventions were supposed to be that definition.

Two datasets measured the record

GH-AWTrace, posted on 4 November, links 25,014 verified GitHub Agentic Workflows across 5,051 repositories to 4,112,526 run records and 562,877 nonempty execution log archives. Its stated reason for existing is that studies examine workflow definitions and outputs “but provide limited evidence of what happens during execution.” The authors are careful about their own corpus: the logs preserve evidence “subject to GitHub’s log-retention limits”, and they “provide partial observations rather than complete agent trajectories.” Four million run records, and the paper’s own description of them is partial.

Soei, posted on 29 October, is smaller and more specific about the same problem. The authors first verify Agenttrail, an open-source local observability tool for coding agents, against its source, and find two event pipelines that “are never unified” and a Map service with an unauthenticated local mutation surface. Then they instrument 45 real coding-agent sessions: 102,233 records, 20,577 tool executions, 2,336 human-to-agent episodes. With a classifier blind-audited at 83.3% and 96.7% precision, they estimate that 23.4% of code-editing episodes end with no independent check after the last edit, and that 57.8% of to-dos the agent marked complete after editing had no proof in between.

The numbers I keep coming back to are about storage. 42% of sessions ran longer than 24 hours. 48.1% of transcript bytes lie past a 2 MiB replay bound, so on that tooling nearly half the record cannot be replayed. And 1,326 records across 16 sessions contain credential-like strings, which is why bounds of that kind exist in the first place.

Three pressures set the size of that record, and none of them is evidentiary need. Storage cost decides how long it is kept, secret hygiene decides what can be written down at all, and the tool’s replay configuration decides how much of the transcript can be run again. Each is defensible alone. Together they mean the artifact you keep is a sample of the run, and in most deployments nobody signed the sampling policy, because it was never anyone’s document to sign.

What I would ask instead

Set the product question aside. The one worth asking a vendor is whether the record can be reconstructed by somebody who does not have their SDK. Three tests follow, and they cost nothing to run.

Does the trace contain the decision, or only the service’s health?

Does it survive past the replay bound and the retention window?

Is it written in a schema with a version you can cite in an audit?

The first two are answerable today, and I would not expect the answers to be flattering. The third has no answer for anyone, because there is no version. A commit hash is not a schema version, and it is not what an assessor asked for.

Some of this ground is occupied. The case for treating agents like distributed systems was made here in September, and the durable-execution argument was made a few weeks earlier. I am not re-arguing either. Distributed systems earned their reliability partly from traces with stable schemas and retention guarantees, and that specific part is now checkable in a way it was not in August. On Tuesday I wrote about six papers measuring whether an agent’s account of its own work is true. This is the storage end of the same problem.

Two limits on the above. I found no study showing that adopting these conventions improves reliability or incident resolution, and no independent teardown of either product’s tracing. Soei’s authors audited a competing open-source tool and then propose their own system as the fix, so the architecture is a pitch even where the measurements are usable. GH-AWTrace is a dataset paper, not an evaluation, and none of it converts into a failure rate.

The September post said a trace is legible, which is why tracing shipped before evaluation did. I would now separate two properties inside that word. Legible to the SDK that produced it is one. Portable to somebody who did not produce it is the other, and only the first is solved. The conventions are worth using anyway, because they are the only vendor-neutral option and the alternative is one schema per vendor. What is missing is smaller and more annoying than a standard: a release, and someone who owns the retention policy. Until both exist, most teams have a trace they can open and no record they could hand over.

Sources

  1. OpenTelemetry semantic-conventions v1.42.0 release notes (OpenTelemetry, 12 June 2026)
  2. OpenTelemetry semantic-conventions v1.45.0 release notes (OpenTelemetry, 2 November 2026)
  3. open-telemetry/semantic-conventions-genai (GitHub, retrieved 5 November 2026)
  4. semantic-conventions-genai releases and tags (GitHub, retrieved 5 November 2026)
  5. semantic-conventions-genai README (retrieved 5 November 2026)
  6. semantic-conventions-genai model/manifest.yaml (retrieved 5 November 2026)
  7. semantic-conventions-genai model directory (15 convention YAML files plus the manifest, counted 5 November 2026)
  8. Langflow docs: OpenTelemetry, v1.13.x (Langflow, retrieved 5 November 2026)
  9. Dify self-host documentation index (Dify, retrieved 5 November 2026)
  10. GH-AWTrace: A Large-Scale Dataset of GitHub Agentic Workflow Execution Logs (arXiv 2611.06866v1, 4 November 2026)
  11. Soei: Structured Observability of Evidence and Intent for Multi-Agent AI Software Engineering (arXiv 2610.39468v1, 29 October 2026)
  12. OpenTelemetry GenAI semantic conventions: November 2026 status (John Hodge, updated 3 November 2026)