FR-006: OpenTelemetry Collector as SCP's Agent Telemetry Ingestion Pipeline¶

Status: Prototype built and sandbox-tested 2026-07-22 — Concrete (see "Why This Is More Concrete Than FR-005" below) Priority: High Author: Tim McCrimmon Created: 2026-07-22 Target Version: v0.4 (candidate for pull-forward into v0.3.2 if scoped narrowly — see Open Questions) Prototype: integrations/cowork-telemetry/ — see that folder's README for setup and honest test status Related: FR-005 (External AI Activity Ingestion) — this FR is the concrete design for FR-005 §5's Cowork adapter sketch, not a competing proposal. Also relevant to the already-tracked "Audit Log SIEM Export (batch)" backlog item (compliance-alignment-status.md) — same underlying technology, opposite data direction; see "Relationship to SIEM Export" below. Regulatory hook: Directly closes the single most consequential item in NexTern's Claude Enterprise Risk Assessment (Confirmed Gap #1 — "Cowork has no Anthropic-hosted audit trail, on any plan tier") — see claude-enterprise/docs/claude-enterprise-risk-assessment.md §2.

Summary¶

FR-005 proposed a pluggable ActivitySource abstraction and, in passing, sketched an "OTLP-compatible receiver" for Cowork as one candidate adapter — but left it exploratory, with no concrete technology choice. Since FR-005 was written (2026-07-05), two things have changed enough to make this buildable rather than aspirational:

The OpenTelemetry Collector (CNCF-graduated, widely deployed) is the actual, documented destination Anthropic's own Cowork OTel feature already assumes exists on the customer side — this isn't a new integration to invent, it's the standard reference implementation of the thing FR-005 already gestured at.
OpenTelemetry GenAI Semantic Conventions — a standardized gen_ai.* attribute schema covering LLM calls, agent spans, and MCP tool calling specifically, adopted across major cloud providers and observability platforms — did not exist in a mature form when FR-005 was drafted. Correction, 2026-07-22: this doesn't apply to Cowork specifically — see "Why This Is More Concrete" and Open Question #1 below. Cowork emits its own documented event schema, not gen_ai.*. The GenAI semantic conventions remain relevant as a likely landing shape for other, non-Cowork agents built on OTel-instrumented frameworks, just not as the mechanism for this particular adapter.

This FR proposes: stand up an OpenTelemetry Collector instance as SCP's concrete Activity Source receiver, ingest Cowork's OTLP logs/events stream (and, by construction, any other OTel-instrumented agent — including SCP-native custom agents built going forward — via the same receiver, normalized on its own terms), use the Collector's processor stage to close the redaction gap already flagged in the claude-enterprise compliance-monitoring doc, and feed the result into FR-005's observed_activity_log store. A working prototype implementing this now exists — see integrations/cowork-telemetry/ — sandbox-tested against synthetic data built from Anthropic's documented event schema; validation against real Cowork traffic is this week's remaining step.

Problem Statement¶

Restating FR-005's framing, narrowed to what's now concrete: Cowork produces no Anthropic-hosted compliance surface at all. The only channel is a customer-configured OpenTelemetry export, and per Anthropic's own documentation, nothing arrives until an admin explicitly points Claude Desktop's OTLP exporter at a collector endpoint — that dependency doesn't go away no matter what SCP builds. What FR-005 left unspecified is what actually receives that stream, what schema it gets normalized into, and how the "you need your own redaction/filtering, this data includes full prompt content and user emails by default" caveat gets addressed. This FR answers those three questions concretely instead of leaving them as an implementation detail.

It also answers a design question that came up independently in NexTern discovery conversations about SCP's registry scope: the registry should be one unified system aware of every agent regardless of how it entered the environment (built internally, adopted as a subscription, or shipped inside a vendor product) rather than siloed per discovery source. A collector built around a standardized schema is what makes that possible without a bespoke parser per agent framework — Cowork today, any OTel/GenAI-instrumented custom agent Ohana or a future customer builds tomorrow, all landing in the same normalized shape.

Relationship to FR-005¶

This is not a parallel or competing proposal — it's FR-005 §5's Cowork adapter, made concrete:

	FR-005 (as written)	FR-006 (this FR)
Cowork ingestion mechanism	"An OTLP-compatible receiver" (unspecified)	The actual OpenTelemetry Collector project, config-driven, not custom-built
Event schema	Draft, bespoke YAML sketch	Cowork's own documented event schema (`user_prompt`, `tool_result`, `api_request`, `api_error`, `tool_decision`) — real and confirmed, not `gen_ai.*` (corrected 2026-07-22, see Open Question #1)
Redaction/filtering	Flagged as a real caveat, not designed	Collector's processor stage — attribute processors for stripping/hashing sensitive fields before storage
Scope beyond Cowork	Chat (Compliance API poller), Propel (open question)	Same receiver extends to any OTel-instrumented custom agent, but each source's own schema still needs its own classification logic — not a free lunch from a shared convention
`ActivitySource` interface, `observed_activity_log` store	Proposed	Reused as-is — this FR is an adapter implementation against FR-005's interface, not a new storage model

FR-005's broader scope (a Compliance API poller for Chat, the open question of whether Propel exposes an audit-export API at all) is untouched by this FR and remains exploratory as FR-005 describes it.

Relationship to SIEM Export (Outbound Direction)¶

compliance-alignment-status.md tracks "Audit Log SIEM Export (batch)" as an open, warranted item — scheduled export of SCP's own audit trail to Splunk/Datadog/Elasticsearch. That's the opposite data direction from this FR (which is inbound: getting Cowork/agent telemetry into SCP), but the same underlying technology solves both — the OpenTelemetry Collector's exporter stage natively supports all three of those backends. Worth noting explicitly so a future spec for that item doesn't reinvent a pipeline this FR already stands up; not proposing to scope-expand this FR to cover it.

Why This Is More Concrete Than FR-005 (Not Exploratory)¶

FR-005's "why exploratory, not committed" caution still applies in spirit — this FR should not be pitched, scoped, or sold as "SCP closes the Cowork audit-trail gap," full stop. Anthropic's own position stands regardless of what SCP ingests: this channel is telemetry for security monitoring, not a compliance-of-record artifact, and nothing changes that framing. What's different here is narrower and more honest: the technology choice is no longer an open question. Standing up an OpenTelemetry Collector against a documented Anthropic export, using an existing standardized schema, is materially lower-risk and lower-effort than FR-005's more speculative framing implied — this is configuration and adapter-mapping work against mature, widely-adopted infrastructure, not a new system to design from scratch.

Proposed Solution¶

1. Deploy an OpenTelemetry Collector instance¶

Standard CNCF Collector, configured with: - Receiver: OTLP over HTTP only (http/json or http/protobuf) — confirmed 2026-07-22 that Cowork's exporter does not offer a gRPC option, so the prototype's collector config enables only the HTTP protocol on the otlp receiver (Claude Desktop ≥1.1.4173 required per Anthropic's documentation) - Processors: attribute processors for redaction/filtering (strip or hash prompt content, tool parameters, and user email fields per customer policy before anything is stored — this is the concrete mechanism for the caveat FR-005 and the claude-enterprise compliance doc both flag but don't resolve); batching for efficiency - Exporter: into FR-005's observed_activity_log store, either via a small custom exporter or by routing through an intermediate store (e.g., ClickHouse/Elasticsearch) that an adapter reads from to populate the normalized table

2. Normalize against Cowork's own documented schema — not `gen_ai.*` (corrected 2026-07-22)¶

Update: this section originally proposed mapping Cowork's export onto gen_ai.* GenAI semantic convention attributes. Research done while building the prototype (see Open Question #1) confirmed Cowork does not emit gen_ai.*-shaped data at all — it emits its own documented event schema (user_prompt, tool_result, api_request, api_error, tool_decision, with attributes like tool_name, cost_usd, input_tokens) via the OTel logs signal, not spans. The prototype's watcher.py normalizes this real schema directly into FR-005's observed_activity_log shape (event_type: prompt / tool_call / mcp_call / approval / model_call / error), rather than translating into gen_ai.*. A gen_ai.*-based normalization may still be worth doing later — specifically as the common landing shape for other, non-Cowork agents that do emit real GenAI semantic convention spans — but that's a distinct future adapter, not something this Cowork adapter can honestly claim today.

3. Extend `describe_coverage()` per FR-005's existing pattern¶

This adapter's coverage manifest must state plainly: nothing arrives until the customer independently configures and points Claude Desktop's OTLP exporter at this collector; redaction processors only catch what they're explicitly configured to catch; this is activity telemetry, not output-quality or compliance-of-record evidence.

4. Design for extensibility to non-Cowork sources from day one (corrected 2026-07-22)¶

The OTLP receiver and redaction/batch processor stages are generic — any future OTel-instrumented agent can send events through the same collector. What's not free is classification: each source's event schema (Cowork's proprietary one, or a future agent's real gen_ai.* spans) needs its own mapping logic, the way watcher.py::infer_event_name() and EVENT_TYPE_MAP do for Cowork specifically. The reusable part is the pipeline shape (receive → redact → normalize → store), not a single shared parser. This is still the concrete mechanism behind "one registry aware of everything" — it just means each new source is its own adapter, not a zero-effort drop-in.

What This Is Not¶

Not a claim that Cowork data becomes audit-grade by being ingested. Anthropic's own framing — telemetry for monitoring, not compliance-of-record — doesn't change because SCP is the one storing it. Same caution FR-005 states; repeating it here because it's the easiest thing for a sales conversation to accidentally overclaim.
Not a replacement for FR-005's Chat/Propel adapters. Those remain separate, exploratory work with their own open questions (notably: does Propel expose an audit-export API at all). Untouched by this FR.
Not a guarantee the customer has configured anything. The dependency on an admin pointing Claude Desktop's OTLP exporter at this collector is real and external to SCP — describe_coverage() must say so plainly, not bury it.
Not scope-expanded to the outbound SIEM export item. Related, same technology, different ticket — see "Relationship to SIEM Export" above.
Not a fix for output quality, hallucination, or safety scoring. GenAI semantic conventions standardize activity metrics (what happened, token counts, which tools) — not whether the output was correct. That stays with the human-review/guardrail process, unchanged.

Open Questions¶

~~Does Anthropic's actual Cowork OTLP export conform to (or map cleanly onto) the GenAI semantic conventions?~~ Resolved by research, 2026-07-22 — answer is no, and this changed the design. Cowork exports via the OTel logs/events signal, not traces (https://claude.com/docs/cowork/monitoring), with its own proprietary attribute schema — five event names (user_prompt, tool_result, api_request, api_error, tool_decision) and attributes like tool_name, tool_input, cost_usd, input_tokens — not literal gen_ai.* GenAI semantic convention keys and not span-based invoke_agent/execute_tool nesting. The Proposed Solution below has been updated accordingly: this adapter normalizes Cowork's own documented schema directly (event-type classification in watcher.py), rather than mapping onto gen_ai.* — the GenAI-semantic-convention framing in this FR's original summary was aspirational, not a description of what Cowork actually emits. A genuinely unresolved sub-question remains: Anthropic's docs don't state which OTLP field carries the event name itself (attribute, body, or otherwise) — the prototype's infer_event_name() classifies by which attributes are present, as a best-effort mapping pending the first real test against live Cowork traffic (this week).
Should SCP ship a documented, customer-operated Collector config as part of onboarding, or a managed/hosted Collector SCP operates on the customer's behalf? The former is lower-risk and consistent with SCP's existing "customer controls their own data" posture; the latter is more turnkey but reopens the "who operates this" question that's come up elsewhere in this engagement's governance discussions. Prototype ships the former (customer-operated, via docker-compose.yml in integrations/cowork-telemetry/) — not yet a final decision, just what was fastest to stand up this week.
Given how much of this is configuration against existing, mature infrastructure rather than net -new engineering, is this actually lightweight enough to pull into the current v0.3.2 NexTern pilot scope rather than waiting for v0.4? Worth a real effort estimate before deciding, not an assumption that "high priority" means "next version." Still open — the prototype took under a day to build and sandbox-test, which is evidence toward "lightweight," but that excludes the real -collector validation step (see Acceptance Criteria) which hasn't happened yet.
Does this FR's redaction-processor design need sign-off against NexTern's own confidentiality guardrail rules before any pilot data flows through it, given it will carry real prompt content during testing? Still open — do not point a real Nextern-tenant Claude Desktop at this collector before that sign-off happens. The prototype's blocked_key_patterns (prompt, tool_input, tool_parameters, user.email, workspace.host_paths) is a reasonable starting default, not a substitute for that review.
New, from building the prototype: the exact OTLP wire field carrying each Cowork event's name is unconfirmed (see Q1) — first real test against live Claude Desktop traffic should confirm or correct watcher.py::infer_event_name().

Acceptance Criteria (if and when this moves from proposed to committed)¶

[x] OpenTelemetry Collector config written (otel-collector-config.yaml) with an OTLP/HTTP receiver, matching Cowork's documented exporter (http/json or http/protobuf, no gRPC) — structurally validated (YAML parses, matches confirmed component config schemas); not yet run against a live Claude Desktop instance
[x] Redaction processor implemented (allow_all_keys + blocked_key_patterns + hash_function: hmac-sha256) and demonstrated end-to-end against synthetic data mimicking a live Cowork OTel stream, including the collector's own redaction.masked.keys/count audit attributes; demonstrated against synthetic data only, not yet a live Cowork OTel stream
[x] Event classification implemented for Cowork's actual documented event schema (not GenAI semantic conventions — see corrected §2 above), with unmatched events landing as event_type: unknown rather than silently dropped or force-fit
[x] Output lands in FR-005's observed_activity_log schema, correctly attributed to the claude-enterprise-cowork source_id — verified via direct SQLite inspection and the read API
[x] describe_coverage()-equivalent (/api/activity-sources/<id>/coverage) implemented and accurately reflects the OTLP-configuration dependency and the "telemetry, not compliance-of-record" caveat
[ ] Remaining: run the real collector (docker compose up) against a real, live Cowork session this week, confirm infer_event_name()'s classification holds against real wire data, and correct it if it doesn't
[ ] Remaining: sign-off on the redaction field list against NexTern's confidentiality guardrail rules before any real Nextern-tenant data flows through it (Open Question #4)
No customer-facing claim that this "closes" the Cowork audit-trail gap ships without the same caveat language FR-005 requires — this applies to demo framing too, not just documentation