Recap Series
- Session 01: A Developer’s Roadmap to Architecting for Agents
- Session 02: Amazon Bedrock Data Automation
- Session 03: Multi-Agent on AgentCore
- Session 04: Building Agentic AI Nova Act and Strands Agents in Practice
- Session 04: Accelerating Migration Projects with Kiro using Spec-Driven Development
- Session 06: From "Matching" to "Understanding": Personalized AI Search Practice Driven by AgentCore Memory
- Session 07: Observe to Optimize – LLM Observability to AIOps Turning real-time insights into intelligent automation
- Session 08: Deploying TEAM and Building the Best Engineering Team
- Session 09: Five Hard Lessons from Five Years of So-Called Serverless Databases
- Session 14: What if AI does my job How Q Developer CLI and Kiro have changed my daily routine
- Session 16: Velocity with Vigilance: Security Essentials for Amazon Bedrock Agent Development
- Session 26: Run OSS LLMs on a Single H100 Smarter, Cheaper, Faster
- Session 28: A Modern Unified Metadata Architecture: New Approaches to Breaking Down Data Silos
- Session 29: Serverless MediaOps: Automating Video Workflows with AI on Amazon Web Services
- Session 30: Architecting for Efficiency and Reliability with Performance Testing at Scale
- Session 31: Connecting the World Through Open Source: Practical Journey of Technology, Community and Global Developer Relations
- Session 33: Building Streaming Iceberg Tables for Real-Time Logistics Analytics
- Session 34: Accelerating Large-Scale Robot Strategy Training: An Automated Closed-Loop Architecture Based on Kiro, Trainium, and EKS
- Session 35: From Vibe to Viable with spec driven development
- Session 36: Making Cloud Cost Analysis Smarter: Building FinOps Intelligent Agents with Strands and AgentCore
- Session 37: Transform Conversational Agentic AIOps for K8s Using CNCF Kagent, K8sGPT, and Nova Sonic
Session Notes
Key Challenges in LLM Operations
- Tracking usage across tenants and models
- Preventing abuse and prompt injection
- Optimizing cost without sacrificing SLA
- Blind spots in usage, security, and cost can sink LLM scale. Real-Time Insights Driving AIOps Decisions
- Monitor every prompt, token, and latency
- Detect anomalies and abuse patterns
- Drive intelligent automation
- Live metrics turn anomalies into instant, automated fixes. Fair Pricing Through Smart Observability
- Align cloud spend with true LLM usage
- Identify under-utilized resources and right-size automatically
- Trigger cost-saving actions (scale-to-zero, burst capacity)
- Pay only for value – usage-driven metering that right-sizes itself. From Observability to Optimization with AIOps
- Smarter automation drives faster incident resolution
- Continuous cost efficiency without manual tuning
- High-performing AI workloads
- transform LLM observability into intelligent AIOps actions. —- Architecture 1 Chat / AI Services Usage with Customers using Toby AI
- SaaS Cluster
- Toby AI Services
- Application Performance
- Monitoring
- Real User Monitoring
- Logs and Metrics Analytics
- Synthetic Monitoring
- Monitoring Rules and Alerts 2 Telemetry Data SaaS (Self-Monitoring Cluster)
- Toby AI Services
- Application Performance
- Monitoring
- Real User Monitoring
- Logs and Metrics Analytics
- Synthetic Monitoring
- Monitoring Rules and Alerts 3 Subscription Check with AIOps Orchestrator
- Context Enrichment
- Policy Decision
- Trigger Actions 4 Runbook, Scale, Throttle, Optimize with DevOps Orchestrator
- GitOps Runbooks