Introducing Amazon CloudWatch Omni: AI-powered observability for generative AI and agentic workloads

Amazon announced CloudWatch Omni as a purpose‑built add‑on to its CloudWatch service that captures every step an AI agent takes—LLM calls, tool selections, prompt revisions—and surfaces that data inside developers’ IDEs as well as a standalone dashboard. The product ships with an extension for VS Code and the Kiro IDE, a “Cloud Login” bridge to push telemetry to AWS, and a suite of 17 built‑in evaluators that score outputs for coherence, factuality, tool‑routing and other quality dimensions. By letting engineers compare two traces side‑by‑side, run batch experiments on production traffic, and version prompts without leaving the code editor, Omni aims to replace the current patchwork of log aggregators, custom OpenTelemetry pipelines and separate AI‑monitoring dashboards.

The move reflects a broader shift where cloud vendors are extending traditional observability stacks into the generative‑AI space. Competitors such as Datadog, New Relic and Google Cloud’s Vertex AI Observability have introduced AI‑specific metrics and tracing, but Amazon’s approach is distinct in its tight IDE integration and its claim of “off‑console” operation that avoids the AWS Management Console. By supporting any model provider or runtime, Omni tries to position itself as a vendor‑agnostic layer that still leans on Amazon’s underlying data lake and OpenTelemetry ecosystem, echoing the industry’s push toward standardized telemetry for LLM‑driven applications.

If Omni gains traction, it could become a de‑facto standard for AI‑agent debugging, especially for teams already on AWS. Risks include potential data‑privacy concerns when telemetry is sent to the cloud, and the learning curve of adopting a new IDE‑centric workflow. Watch for how quickly third‑party model providers integrate with the platform, whether the 17 evaluators prove sufficient for specialized domains, and if AWS opens the data model for export to avoid vendor lock‑in.