Optimizing AI Workloads with NVIDA GPUs, Time Slicing, and Karpenter (Part 2) Dhanush Gowda Vidyaranya on January 22, 2025 at 9:13 pm

Introduction: Overcoming GPU Management Challenges   In Part 1 of this blog series, we explored the challenges of hosting large language models (LLMs) on CPU-based workloads within an EKS cluster. We discussed the inefficiencies associated with using CPUs for such tasks, primarily due to the large model sizes and slower inference speeds. The introduction of GPU […]

​[#item_full_content]  Introduction: Overcoming GPU Management Challenges   In Part 1 of this blog series, we explored the challenges of hosting large language models (LLMs) on CPU-based workloads within an EKS cluster. We discussed the inefficiencies associated with using CPUs for such tasks, primarily due to the large model sizes and slower inference speeds. The introduction of GPU  Read More Cisco Blogs