FR-005: External AI Activity Ingestion (Multi-Vendor Telemetry Adapters)¶

Status: Partially Implemented (2026-07-24) — all three adapters sketched in §5 have been built and tested against real traffic. See "Implementation Status" below for what's built vs. what the original sketch got wrong. The "Why exploratory, not committed" caution below still applies in full — building these adapters does not by itself justify presenting SCP as closing NexTern's audit-trail gap or as compliance-mapped. Priority: Medium Author: Tim McCrimmon Created: 2026-07-05 Target Version: v0.5 (backlog; not yet slotted into a committed release) Related: FR-001 (Tenant-Scoped MCP Control) — complementary, not overlapping; see §"Relationship to FR-001". FR-006 (OpenTelemetry Collector as SCP's Agent Telemetry Ingestion Pipeline) — a concrete design for the Cowork adapter sketched in §5 below; that FR supersedes this one's Cowork ingestion mechanism specifically, not the rest of this FR's scope. Regulatory hook: Motivated by NexTern's audit-readiness requirement (see "Why exploratory" below) — this FR does not itself constitute compliance evidence, see "What This Is Not"

Summary¶

SCP's Agent Registry today only knows about activity that comes to it — an agent shows up in the audit trail because it called POST /api/context and asked for something. That model works for SCP-native agents, but it has no way to see activity from AI platforms that never ask SCP for anything: Claude Enterprise Chat and Claude Cowork both generate their own native activity records (the Compliance API for Chat, an OpenTelemetry event stream for Cowork) entirely independent of whether any agent involved has ever talked to SCP. The same will likely be true of Propel's own agentic/MCP features once they exist, and of whatever the next AI vendor ships after that.

This FR proposes a pluggable Activity Source abstraction: adapters that pull or receive activity records from external AI platforms' own native telemetry/audit exports, normalize them into a common event schema, and make them queryable alongside (but not merged into) SCP's existing agent audit trail. The first adapter would target Claude Enterprise (Compliance API + OTel).

Problem Statement¶

The current model:

Agent registered with SCP → Agent calls /api/context → SCP logs the request
                                                            ↓
                                            This IS the audit trail

This breaks down for activity that never touches /api/context:

Claude Enterprise Chat — activity (created/viewed/updated/deleted, sharing changes — not prompt or response text; see correction under "Implementation Status") is captured by Anthropic's Compliance API's Activity Feed. A poller now exists (§5, built 2026-07-24), so this does reach SCP — the original framing of "nothing reaches SCP" is the gap this FR closed.
Claude Cowork — has no Anthropic-hosted compliance surface at all; visibility depends entirely on whether the customer has configured an OpenTelemetry pipeline, and what that stream actually carries: prompts, tool/MCP invocations, and file access, correlated via a prompt.id and user identity, but only if OTel is set up and pointed somewhere. An adapter now exists (FR-006, §5) and has been validated against real traffic.
Propel (or any future MCP-adjacent vendor) — claims its own "enterprise trust layer" produces full audit trails, but whether that's externally queryable via an API, versus only visible inside Propel's own UI, has not been verified as of this writing. If it's the latter, FR-001's MCP Gateway is the only lever SCP has into Propel activity, not this FR.

None of this activity originates from an agent asking SCP for permission first — it's produced natively by the vendor platform. Trying to force it through the existing Agent Registry model (identity + role + allowed intents, gated by intent validation) doesn't fit: there's no intent being requested, nothing to validate against, and in many cases no SCP-registered agent identity to attach it to at all. This is structurally a different capability — closer to a passive downstream collector than an active registry of things that ask permission — and the Registry model should not be stretched to pretend otherwise.

Relationship to FR-001 (Tenant-Scoped MCP Control)¶

These are complementary, not two ways of doing the same thing, and shouldn't be collapsed into one ticket:

	FR-001 (MCP Gateway)	FR-005 (this FR)
What it sees	Every tool call that crosses an MCP boundary	Whatever a vendor's own native telemetry export emits
Vendor-agnostic?	Yes, by construction — MCP is a shared protocol	No — one adapter per vendor's export mechanism
Sees raw prompts / chat content?	No	Yes, if the vendor exports it (e.g., Compliance API)
Sees non-MCP built-in tools (web fetch/search, code exec)	No — these don't route through MCP	Yes, if the vendor's telemetry captures them (Cowork's OTel stream does)
Requires agent to be SCP-registered?	No — gateway sees the call regardless	No — this is the point
Depends on vendor having enabled/configured an export feature	No	Yes — this is a real fragility (e.g., OTel emits nothing until an admin wires an OTLP endpoint)

Put simply: FR-001 gives SCP a vendor-agnostic view of anything that flows through MCP, which covers cross-vendor scenarios (e.g., a Propel-mediated tool call) without a Propel-specific integration. FR-005 covers everything that never touches MCP at all — and for that, there's no way around building a per-vendor adapter, because there's no shared protocol to intercept.

Why Exploratory, Not Committed¶

SCP's evidentiary bar for this customer is strict, for good reason: NexTern's AI adoption is gated on a CFO condition that any audit-failure exposure or unmanaged risk means no AI at all. The customer-facing claude-enterprise docs repo holds every capability and regulatory claim to a primary-source standard because of this. If SCP work is ever positioned to NexTern as adding governance, compliance, or audit-readiness value, it inherits that same bar — see the note in this repo's top-level CLAUDE.md.

Concretely, that means this FR should not be pitched, scoped, or built as "SCP closes the Cowork audit-trail gap" or "SCP provides compliance coverage across all AI vendors." At most it ingests and re-exposes whatever the vendor already emits, subject to every limitation of that export. Whether this FR actually earns its place in the roadmap — versus being infrastructure in search of a justification — is an open question, not a foregone conclusion. Build it if and when there's a concrete need for SCP itself (not the underlying vendor telemetry) to be the query surface; don't build it to have a bullet point.

Proposed Solution (sketch as originally written — see "Implementation Status" below for¶

what actually got built and where it diverges)

1. Activity Source adapter interface¶

class ActivitySource(Protocol):
    source_id: str  # e.g. "claude-enterprise-chat", "claude-enterprise-cowork"

    async def fetch_events(self, since: datetime) -> list[NormalizedActivityEvent]:
        """Poll or receive events since the last checkpoint."""

    def describe_coverage(self) -> ActivityCoverageManifest:
        """
        Machine-readable statement of what this source does and does NOT capture,
        so coverage claims can be generated from what's actually wired up rather
        than asserted in a doc that can drift out of sync with reality.
        """

Each adapter owns its own auth, polling/subscription mechanics, and mapping into a common event shape. The describe_coverage() method matters as much as fetch_events() — it's the mechanism that stops the "what does SCP actually see" answer from silently becoming aspirational.

2. Normalized event schema (draft)¶

event_id: uuid
source_id: "claude-enterprise-chat"        # which adapter produced this
observed_at: timestamp
actor_identity: "user@nextern.com"          # not necessarily an SCP-registered agent
correlation_id: "prompt.id or session id, source-defined"
event_type: prompt | tool_call | mcp_call | file_access | approval | unknown
tenant_id: nullable                         # only if the source has tenant concept
raw_payload_ref: pointer to full record in source system (not duplicated into SCP)

3. Observed-activity store, separate from the Agent Registry¶

A new table (e.g. observed_activity_log), distinct from agent_output_logs. Entries here are not subject to intent validation, allowed-intents checks, or the circuit breaker — those concepts don't apply to activity SCP never gated in the first place. Optionally, entries correlate to a registered agent_id when the actor identity matches one (e.g., a NexTern engineer who is also behind a registered SCP agent), but correlation is best-effort, not assumed.

4. API surface (sketch)¶

POST /api/activity-sources           Register a new adapter instance + credentials
GET  /api/activity-sources           List configured sources + their coverage manifest
GET  /api/activity/observed          Query observed activity (filterable by source, actor, time)
GET  /api/activity-sources/{id}/coverage   Return describe_coverage() output for a source

5. First adapter: Claude Enterprise¶

Chat & Projects — poller against Anthropic's Compliance API (api.anthropic.com/v1/compliance/*). Correction (2026-07-24): the "no historical backfill" constraint below was wrong. That applies to Cowork's OTel export (nothing arrives until an admin configures it, and nothing before that point is recoverable) — it does not apply to the Compliance API, which Anthropic retains for 6 years with a documented backfill pattern (paginate older activity via after_id). The implemented adapter (integrations/cowork-telemetry/store/poller_compliance.py) deliberately doesn't backfill by default — it seeds its cursor from "now" for a fast, bounded first run — but that's a choice this deployment makes, not a limitation of the API. An opt-in flag (BACKFILL_ON_FIRST_RUN=1) does the full walk if ever wanted.
Cowork — an OTLP-compatible receiver that ingests whatever the customer's OTel pipeline is configured to forward (prompts, tool/MCP invocations, file access), correlated via prompt.id and user identity per Anthropic's documented event shape. This adapter's describe_coverage() must reflect that nothing arrives until the customer has independently configured and pointed an OTel pipeline at SCP — that's a real dependency, not an implementation detail to gloss over. See FR-006 for the concrete design (OpenTelemetry Collector + GenAI Semantic Conventions as the actual receiver/schema, rather than an unspecified "OTLP-compatible receiver") — written once it became clear this could be a standards-based build rather than a bespoke one.
Claude Code CLI — not in the original scope of this section (only Chat and Cowork were anticipated), but built as a third adapter using the same OTel-receiver pattern as Cowork, on a separate collector instance since Claude Code's event vocabulary, attribute names, and content-redaction defaults are all different from Cowork's — see Implementation Status.

Implementation Status (2026-07-24)¶

All three adapters anticipated or added above are built and live in integrations/cowork-telemetry/ (not the main control-plane/ — see "Build location" in FR-006):

Adapter	source_id	Mechanism	Tested against
Cowork	`claude-enterprise-cowork`	OTel collector (FR-006)	Real Cowork traffic — working
Claude Code CLI	`claude-code-cli`	Second, independent OTel collector	Real traffic (2026-07-24) — working
Chat & Projects	`claude-enterprise-chat`	Poller against the Compliance API	Real Nextern org data — working

Where the implementation matches this sketch: - observed_activity_log (§3) exists as designed, separate from agent_output_logs, with a source_id column distinguishing all three adapters in one shared table. - The normalized event schema (§2) is implemented close to as drafted — event_id, source_id, observed_at, actor_identity, correlation_id, event_type all present; raw_payload_ref became raw_attributes (the full normalized record embedded directly, not a pointer back to the source system — simpler for a demo/prototype, revisit if this ever needs to avoid storing a copy of vendor data). - describe_coverage() (§1) exists per adapter as a COVERAGE_MANIFEST dict with captures / does_not_capture / caveats, served at GET /api/activity-sources/{id}/coverage (§4) — this part of the design held up exactly as sketched.

Where the implementation diverges from this sketch — read before assuming §1/§4 are literally true of the running system: - No ActivitySource Protocol class, no POST /api/activity-sources. Each adapter is a standalone script (watcher.py, watcher_claude_code.py, poller_compliance.py) following a shared convention (a SOURCE_ID constant, a COVERAGE_MANIFEST dict, a normalize_*() function), not a formal interface, and each self-registers its own activity_sources row on startup rather than being registered externally via an API call. Fine for three built-by-the-same-person adapters; would need real interface enforcement before a fourth adapter (e.g., Propel) gets built by someone else, or before external customers configure sources themselves. - Claude Code CLI's content posture is safer than what §2's "full content" framing implies for Chat. Claude Code redacts prompt/response/tool-input content by default — nothing sensitive is even exported unless an admin opts in — whereas Cowork exports it and relies on this pipeline's redaction to mask it. The Chat/Projects adapter never had "full content" to worry about in the first place: the Activity Feed is metadata only (who did what, when, to which resource) — actual chat/response text lives behind separate content-retrieval endpoints this adapter deliberately does not call. The Problem Statement's "full content + event log" phrasing conflated these two things; corrected above.

What This Is Not¶

Not a compliance mapping tool. Ingesting Claude's telemetry does not mean SCP has verified, interpreted, or mapped that data against any regulatory framework. That's separate work, and overdue verification (e.g., against 21 CFR Part 11, EU AI Act) should not be implied by the mere existence of this ingestion pipeline.
Not a replacement for FR-001. MCP-routed activity is FR-001's job. This FR does not attempt to intercept MCP calls.
Not a guarantee of complete coverage. Each adapter's describe_coverage() output is the source of truth for what's actually captured — this FR explicitly rejects any framing that treats "we ingest vendor X's telemetry" as equivalent to "we have full visibility into vendor X."
Not scoped to Propel today. Whether Propel's audit trail is externally accessible at all is an open question (see Problem Statement) that needs to be answered — by Propel, not assumed — before a Propel adapter is designed.

Open Questions¶

Does Propel's "enterprise trust layer" expose an audit-export API, or is it UI-only? This determines whether a Propel adapter is even possible, versus FR-001's gateway being the only available lever for that vendor.
Should observed (unregistered) activity ever be promoted into a full Agent Registry entry automatically, or does that conflate "we're watching this" with "we're governing this" in a way that misrepresents SCP's actual level of control?
Is there a concrete customer need for SCP to be the query surface for this data, versus the customer querying Anthropic's Compliance API and their own OTel/SIEM tooling directly? This FR only earns its place if SCP adds something beyond duplicating what's already queryable elsewhere (e.g., cross-vendor correlation that no single vendor's own tooling can do).

Acceptance Criteria (if and when this moves from exploratory to committed)¶

[~] ActivitySource interface defined and documented, with describe_coverage() as a required method, not optional — partially: describe_coverage()'s equivalent exists and is required by convention across all three adapters; there's no formal interface/Protocol enforcing it. See "Implementation Status."
[x] At least one adapter (Claude Enterprise Chat via Compliance API, or Cowork via OTel — whichever has a concrete driving need) implemented end to end — three adapters, not just one: Cowork, Claude Code CLI, and Chat/Projects, 2026-07-24.
[x] observed_activity_log schema implemented, explicitly separate from agent_output_logs
[x] GET /api/activity-sources/{id}/coverage returns accurate, current coverage information — no hand-written doc claiming coverage that the adapter itself can't confirm
[ ] No customer-facing claim of "audit trail coverage" or "compliance mapping" ships without being independently checked against what the adapter actually ingests — ongoing discipline, not a one-time checkbox — re-verify before any customer-facing materials cite this pipeline.