CNCF-hosted Co-located Events North America 2024

LLMs are huge to deploy and manage. Have you ever wondered if there's a way to pinpoint exactly which parts of your code are draining resources, causing latency, and hurting performance? In a world where efficiency is crucial, dynamically inspecting application behavior and performance at runtime can be transformative. Join this session to explore how to leverage OpenTelemetry’s profiling feature to optimize LLM code at a much deeper level.

We'll cover how to:
1. Identify specific pieces of code that consume excessive CPU and memory, or cause memory leaks and OOM errors.
2. Improve LLM performance by understanding model behavior, reducing latency, and meeting SLAs and SLOs.
3. Achieve efficient deployments on Kubernetes, ensuring optimal resource utilization and cost savings.