CNCF-hosted Co-located Events North America 2024

Confluent Cloud is a data streaming platform built on thousands of Kubernetes clusters across AWS, Azure & GCP. Confluent migrated clusters to use Cilium for its advanced security features like transparent encryption and DNS name-based network policies, along with performance, scalability & observability improvements. The main challenge was executing a live migration without disrupting stateful workloads, complicated by the risks of replacing a low-level component like the CNI. The process required meticulous planning to ensure intra-cluster connectivity during migration, while accommodating each cloud provider's unique network config. This talk shares the journey of migrating to Cilium, highlighting obstacles and lessons learned. We will explore uninstalling pre-existing CNIs, setting up Cilium & addressing cloud-specific issues to maintain connectivity. Benefits like transparent encryption, policies, and Hubble observability, along with the challenges faced, will also be discussed.