Fortifying AI Security in Kubernetes with Confidential Containers (CoCo)

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io Fortifying AI Security in Kubernetes with Confidential Containers (CoCo) - Suraj Deshmukh, Microsoft & Pradipta Banerjee, Red Hat AI models have become valuable intellectual property that can provide organizations with a competitive edge. Users are searching for ways to secure their AI models without implicitly trusting third-party platform providers. While encryption is available to keep models secure when they’re stored & transferred, they’re still decrypted & loaded into memory during inferencing, potentially exposing them to unintentional or intentional exfiltration. This is where "confidential computing" comes in. This technology encrypts memory to protect data in use. Confidential Containers (CoCo) is a CNCF sandbox project that aims to bring confidential computing to k8s. The k8s AI/ML ecosystem is mature & offers many AI/ML training & inferencing options. The focus is on using CoCo with Kserve project to show how CoCo strengthens AI model protection. Apart from inferencing, we will explore broader application of CoCo, emphasizing its role in providing general memory protection for foundational platforms.