Inference with Claudie
In this blog post, we’ll walk through how to connect an on-premise server, equipped with custom hardware, with the AWS cloud provider, forming a hybrid cluster, and further use this infrastructure for running an AI workload.
Our experience running an AI workload in Kubernetes - Part 4 <em>The Scaling Challenges</em>
In the previous part of this series, we walked through our migration from the RayCluster CRD to the RayService CRD. To complete the picture, this post covers the challenges we’ve faced and the improvements made to our setup running in a cost-optimized multi-cloud Kubernetes cluster.
Our experience running an AI workload in Kubernetes – Part 3 <em>Migration to RayService</em>
Brief outages caused by Ray head node restarts were no longer acceptable. In this post, we dive into our migration from the RayCluster CRD to the RayService CRD, which enabled rolling updates, external GCS storage, and more. We share how we tackled challenges such as unpredictable deployments, slow Ray worker nodes start-up, and ensuring high availability with Dragonfly. If you want to understand how to make Ray workloads more resilient, predictable, and production-ready on Kubernetes, this post walks through our practical solutions and lessons learned.
Our experience running an AI workload in Kubernetes – Part 2 <em>Limitations & Pitfalls of our solution with RayCluster CRD</em>
In this part of our series, we share the challenges we faced running Ray Serve Deployments in production using the RayCluster CRD. Along the way, we tackled issues like ephemeral head nodes, RayCluster’s autoscaling quirks, and the limitations of rolling updates. If you’re curious about bridging the gap between traditional Kubernetes workloads and the unique demands of AI applications on Ray, this post dives deep into using the RayCluster CRD in K8s.
Our experience running an AI workload in Kubernetes – Part 1 <em>Lift & Shift Ray applications to K8s</em>
In this post, we share our hands-on experience helping our client, Mixedbread, run their AI applications on Kubernetes using the KubeRay Operator. During the migration from a hyperscaler to a multi-cloud environment powered by claudie.io, we cut infrastructure costs by 70% while tackling challenges around RayCluster resilience, Ray Serve Deployments.
Kubernetes on Genesis Cloud with Claudie!
Claudie is a platform for managing multi-cloud and hybrid-cloud Kubernetes clusters. These Kubernetes clusters can mix and match nodepools from various cloud providers, e.g. a single cluster can have a nodepool in AWS, another in GCP and another one on-premises. In one of the recent releases 0.7.0, Claudie brings support for Genesis Cloud and its EU-hosted GPU instances. This article explains the process of setting up a Kubernetes cluster on Genesis.
AWS SSO backed by Google Workspace: Troubleshooting and surprising resolution
Configuring SSO configuration in a multi-cloud environment is often anything but straightforward. Our colleague Miroslav Kohutik encountered a vague error message while setting up SSO. This blog post dives into his troubleshooting journey and unveils the solution to get your AWS/Google Workspace SSO up and running.
Egress traffic in multi-cloud Kubernetes: do I need to worry?
Egress traffic represents the network data that exits an entity or a private network within a cloud infra and heads toward the public internet. Many cloud providers charge you for such outbound data transmission. Let’s look at its impact on multi-cloud Kubernetes clusters.
Exploring Multi-Tenancy Solutions for my Kubernetes Learning Platform
Introduction Multi-tenancy in Kubernetes presents various complex challenges, including security, fairness, and resource allocation. This blog discusses the challenges associated with multi-tenancy and the technology choices made for a Kubernetes-based …
A different method to debug Kubernetes Pods
By Adam Stawski 15 August 2023. In this blog, I will demonstrate a step-by-step guide on how to access a running Kubernetes Pod by examining its namespace.
Interview with Bernard Halas on Claudie and DevOps Engineering
Bernard has been interviewed on multi-cloud Kubernetes topics and SW engineering in general. He also gave the context of why Claudie has been build to help in this regards.
Evaluating etcd’s performance in multi-cloud
By Adam Stawski 12 May 2023 1. Intro Many companies are focusing on making their workload as highly available as possible. The intention behind moving their workloads from local datacenters
Introducing Cluster Autoscaler into Claudie
By Miroslav Repka 02 May 2023 Excitingly, Claudie has recently introduced integration with Cluster Autoscaler, providing Claudie users with full functionality of the autoscaler across any cloud or mixture of
DevOps Team With No Security
By Mário Uhrík 09 December 2022 A friend is going through an interview process at a well-known IT security company. This is one of the interview questions: Imagine a situation
Cloud-agnostic Kubernetes Clusters
By Bernard Halas 23 November 2022 Intro Kubernetes is often referred to as “the operating system of the cloud”. It gives freedom to build feature-rich platforms for operating your application stacks.
Traffic Encryption Performance in Kubernetes Clusters
By Samuel Stolicny January 18, 2021 Building a hybrid Kubernetes cluster among various environments (public providers and on-premise devices) requires a layer of reliable and secure network connectivity. Choosing the …