Deploying an AI agent is easy. Keeping it reliable, safe, and accountable once it's executing real business processes is where most teams get stuck. This session covers the emerging discipline of AgentOps: the operational practices required to run agentic AI systems in production. Topics include telemetry design for agent actions, evaluation frameworks for measuring agent quality over time, debugging strategies when agents produce unexpected behavior, and identity management using Entra Agent ID and SPIFFE/SPIRE. The session draws on production experience with multi-agent systems handling client communications, task management, and document generation at scale.

You will learn:

• About design telemetry architectures that capture agent decision chains, tool calls, and outcomes
• How to build evaluation pipelines that measure agent quality and detect drift over time
• How to implement debugging workflows for diagnosing failures in multi-step agent execution