Migrate from AWS EKS to Kubernetes on Quake AI

Service mapping#

1. Overview#

Amazon EKS gives you a managed Kubernetes control plane with deep AWS integration: VPC-native pod networking (VPC CNI), IAM Roles for Service Accounts (IRSA), EBS/EFS-backed persistent storage, ALB/NLB ingress through the AWS Load Balancer Controller, and Karpenter for sub-minute node autoscaling.

EKS also offers Auto Mode (generally available since December 2024), which manages both the control plane and the worker nodes through Karpenter, running EC2 managed instances. Auto Mode clusters add managed CRDs (NodePool and EC2NodeClass). EKS Pod Identity (generally available since November 2023) is the newer pod-to-IAM mechanism: it uses PodIdentityAssociation resources instead of the IRSA eks.amazonaws.com/role-arn annotation. If your source cluster uses Auto Mode or Pod Identity, add the matching cleanup items to the checklist in Section 3.

On Quake AI, you provision Kubernetes two ways. You can run self-managed RKE2 or k3s on Nova instances, installing and operating the control plane yourself and wiring in the OpenStack cloud-provider stack (CCM + Cinder CSI), or you can provision a cluster through Magnum, the Container Infrastructure Management service, which builds the cluster from a curated template (openstack coe cluster create). Both are supported; see the Kubernetes migration overview for the trade-offs.

For an EKS migration, self-managed often makes it easier to match your source cluster's Kubernetes version and CNI exactly: the Magnum template catalog provides a fixed set of versions that can differ from the versions EKS supports (1.30 through 1.35 as of June 2026). Run openstack coe cluster template list to see the templates available in your region before you choose Magnum.

Migration complexity: Medium-High. The primary drivers are IRSA removal, EBS-to-Cinder data migration, ALB-to-nginx-ingress conversion, VPC CNI replacement with Calico/Cilium, Fargate pod migration to regular node pools, and Karpenter replacement. Expect 4-8 weeks for a production migration depending on IRSA and EFS usage.

2. Workload portability matrix#

3. Pre-migration: export and audit#

Audit your existing EKS cluster to identify all provider-specific resources before you make changes.

Confirm your EKS kubeconfig is current and that kubectl can reach the source cluster:

aws eks update-kubeconfig --region REGION --name CLUSTER_NAME
kubectl get nodes

A modern EKS kubeconfig uses aws eks get-token as the exec credential plugin. You no longer need the older aws-iam-authenticator binary with AWS CLI 1.16.156 or later.

kubectl get all --all-namespaces -o yaml > cluster-export.yaml

kubectl get pvc --all-namespaces -o yaml > pvcs.yaml

kubectl get sa --all-namespaces -o yaml | grep -l 'eks.amazonaws.com' > irsa-accounts.txt

kubectl get ingress --all-namespaces -o yaml > ingresses.yaml

helm list --all-namespaces > helm-releases.txt

kubectl get pods --all-namespaces -o json | \
  jq '.items[] | select(.metadata.annotations["eks.amazonaws.com/compute-type"]=="fargate") | .metadata.name'

Inventory checklist:

• All EBS-backed PVCs and their StorageClasses (gp2, gp3, custom)
• All IRSA-annotated ServiceAccounts (eks.amazonaws.com/role-arn)
• All ALB/NLB Ingress resources and their annotations
• All Fargate-scheduled pods (eks.amazonaws.com/compute-type: fargate)
• All ECR image references
• EKS Pod Identity associations (PodIdentityAssociation resources, in addition to IRSA-annotated ServiceAccounts)
• EKS Auto Mode NodePools and EC2NodeClass CRDs (if applicable)
• Helm releases and their versions
• EFS PVCs (RWX): these require special handling

4. Provision Kubernetes on Quake AI#

4.1 Option A: self-managed RKE2 or k3s (recommended)#

Build the cluster yourself on Nova instances. Use OpenTofu with the openstack provider to create the infrastructure, then install Kubernetes. This path matches any Kubernetes version, CNI, or CRI your workloads require, including the versions EKS supports today.

Infrastructure (OpenTofu):

• Neutron network + subnet
• Security groups (allow K8s API on port 6443, NodePort range 30000-32767, inter-node communication)
• Nova instances: 3 control plane nodes (for HA) + N worker nodes
• Stable endpoint for the Kubernetes API

Install RKE2 (one supported self-managed distribution):

curl -sfL https://get.rke2.io | INSTALL_RKE2_TYPE=server sh -
systemctl enable --now rke2-server

Install the OpenStack cloud-provider stack on the self-managed cluster:

1. Deploy openstack-cloud-controller-manager with a cloud.conf using application credentials
2. Deploy the Cinder CSI driver with the cinder-flash StorageClass
3. Install Calico (VXLAN mode) or Cilium as the CNI

Verify the cluster:

kubectl get nodes
kubectl get csinodes
kubectl get storageclass

4.2 Option B: Magnum#

Provision a cluster from a Magnum template. The platform creates the control plane VMs, worker VMs, security groups, networking, and a stable endpoint for the Kubernetes API. Run openstack coe cluster template list first to confirm which templates and Kubernetes versions your region offers.

openstack coe cluster template list
openstack coe cluster create production-k8s \
  --cluster-template Standard-v2.0-k8s-calico-fc38_v1.24.16 \
  --master-count 3 \
  --node-count 3 \
  --keypair MY_KEYPAIR

For the full walkthrough including Console and quota guidance, see Create a Kubernetes cluster. To roll your own template (custom flavors, alternate Kubernetes version), see Create a cluster template.

Once the cluster reports CREATE_COMPLETE, fetch the kubeconfig:

openstack coe cluster config production-k8s --dir ~/.kube
kubectl get nodes
kubectl get storageclass

The Cinder CSI driver and the OpenStack cloud-provider stack are pre-installed by the template; no manual CCM, CSI, or CNI install is required for the Magnum path.

5. Adapt provider-specific resources#

CSI driver: EBS to Cinder#

Replace all references to the EBS CSI driver and its StorageClasses:

# Before (EKS)
storageClassName: gp2

# After (Quake AI)
storageClassName: cinder-flash

The cinder-flash StorageClass uses the Cinder CSI provisioner with Flash_Premium volume type:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: cinder-flash
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: cinder.csi.openstack.org
parameters:
  type: Flash_Premium
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

EFS (RWX) workloads: Amazon EFS gives you ReadWriteMany access with no direct Cinder equivalent. Your options on Quake AI:

• NFS server provisioner (dedicated Nova VM running NFS)
• Rook-Ceph with CephFS (production-grade RWX, operationally complex)
• Restructure the application to use RWO volumes with application-level coordination

Data migration with Velero#

Use Velero to back up your EKS workloads and restore them onto the Quake AI cluster, including filesystem-level backup of PersistentVolume data. Install the Velero provider plugin that matches your Velero version instead of pinning an old release:

velero plugin add velero/velero-plugin-for-aws:CURRENT_VERSION

Use the plugin version compatible with your Velero installation; see the Velero compatibility matrix.

To remap EBS StorageClasses to cinder-flash during the restore, create a change-storage-class ConfigMap before you run velero restore create:

apiVersion: v1
kind: ConfigMap
metadata:
  name: change-storage-class-config
  namespace: velero
  labels:
    velero.io/change-storage-class: RestoreItemAction
data:
  gp2: cinder-flash
  gp3: cinder-flash

Create this ConfigMap before running velero restore create. The change-storage-class RestoreItemAction is built into Velero core (Velero 1.10 and later). You still need provider plugins for the backup storage location, but not for StorageClass remapping.

Ingress controller: ALB to nginx-ingress#

Replace AWS Load Balancer Controller annotations with nginx-ingress annotations:

# Before (EKS ALB)
annotations:
  kubernetes.io/ingress.class: alb
  alb.ingress.kubernetes.io/scheme: internet-facing
  alb.ingress.kubernetes.io/target-type: ip

# After (Quake AI nginx-ingress)
annotations:
  kubernetes.io/ingress.class: nginx
  nginx.ingress.kubernetes.io/ssl-redirect: "true"

TLS termination moves from ACM (AWS Certificate Manager) to cert-manager with Let's Encrypt on Quake AI. You deploy cert-manager and create a ClusterIssuer before you migrate Ingress resources.

Pod identity: IRSA to application credentials#

IRSA injects AWS credentials into pods by annotating ServiceAccounts with eks.amazonaws.com/role-arn. This mechanism does not work outside AWS.

EKS Pod Identity (generally available since November 2023) is an alternative to IRSA that uses PodIdentityAssociation resources instead of ServiceAccount annotations. If your source cluster uses Pod Identity, list the associations during the audit and remove the EKS Pod Identity add-on before migration:

aws eks list-pod-identity-associations --cluster-name CLUSTER_NAME

For applications accessing OpenStack APIs: Create application credentials and store them in Kubernetes Secrets:

openstack application credential create MYAPP_CREDENTIAL --role member
kubectl create secret generic openstack-app-creds \
  --from-literal=OS_APPLICATION_CREDENTIAL_ID=APPLICATION_CREDENTIAL_ID \
  --from-literal=OS_APPLICATION_CREDENTIAL_SECRET=APPLICATION_CREDENTIAL_SECRET

Replace APPLICATION_CREDENTIAL_ID and APPLICATION_CREDENTIAL_SECRET with the values from the openstack application credential create output.

For applications that still need AWS access (hybrid period): Replace IRSA with static IAM credentials in Kubernetes Secrets. This is less secure than IRSA but works during the transition.

You remove the IRSA annotation from every affected ServiceAccount:

kubectl annotate sa MY_SERVICE_ACCOUNT eks.amazonaws.com/role-arn- -n MY_NAMESPACE

Service type LoadBalancer#

Kubernetes Services with type: LoadBalancer receive a public endpoint through the cluster's cloud controller. Remove AWS-specific annotations and keep the standard Service fields. The assigned address appears in status.loadBalancer.ingress.

Fargate pods#

Remove the eks.amazonaws.com/compute-type: fargate annotation and any Fargate profile references. Deploy these workloads to regular Nova-backed worker nodes. Note the isolation change: Fargate enforces hard resource isolation per pod (dedicated vCPU and memory), while Quake AI node pools enforce limits through cgroups and let pods share node resources. Apply LimitRange resources to namespaces to enforce minimum resource requests and prevent unbounded consumption.

Container images: ECR to permanent registry#

ECR authentication tokens expire every 12 hours and cannot be used from outside AWS long-term. Mirror all images to a permanent registry (Docker Hub, self-hosted Harbor, or Quay.io) before cutover:

docker pull AWS_ACCOUNT_ID.dkr.ecr.AWS_REGION.amazonaws.com/IMAGE_NAME:IMAGE_TAG
docker tag AWS_ACCOUNT_ID.dkr.ecr.AWS_REGION.amazonaws.com/IMAGE_NAME:IMAGE_TAG docker.io/DOCKER_ORG/IMAGE_NAME:IMAGE_TAG
docker push docker.io/DOCKER_ORG/IMAGE_NAME:IMAGE_TAG

Replace AWS_ACCOUNT_ID, AWS_REGION, IMAGE_NAME, IMAGE_TAG, and DOCKER_ORG with your registry values. You update all Deployment and StatefulSet manifests with the new image references.

6. Apply and validate#

You deploy adapted manifests to the Quake AI cluster:

kubectl apply -f adapted-manifests/ --namespace production
kubectl get pods -n production -w

Verify each layer:

kubectl get pods --all-namespaces -o wide
kubectl get pvc --all-namespaces
kubectl get svc --all-namespaces
kubectl get ingress --all-namespaces

Test application endpoints through the ingress controller's public LoadBalancer Service. Confirm PVCs bind to Cinder volumes.

7. Observability setup#

You replace AWS managed observability (CloudWatch Container Insights, X-Ray, CloudWatch Logs) with a self-managed stack:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm upgrade --install kube-prometheus-stack \
  prometheus-community/kube-prometheus-stack \
  --version 72.9.1 \
  --namespace monitoring --create-namespace

helm repo add grafana https://grafana.github.io/helm-charts
helm upgrade --install loki grafana/loki-stack \
  --namespace monitoring \
  --set promtail.enabled=true \
  --set loki.persistence.enabled=true \
  --set loki.persistence.storageClassName=cinder-flash

Match the chart version to your cluster's Kubernetes version. From chart version 73.0.0 the kube-prometheus-stack chart requires Kubernetes 1.25 or later. Quake AI Magnum cluster templates run Kubernetes 1.24.16, so pin --version 72.9.1, the last chart release that supports 1.24. On a cluster running Kubernetes 1.25 or later, omit the --version flag to install the current chart.

The Helm releases install Prometheus (metrics), Grafana (dashboards), Alertmanager (alerting), and Loki with Promtail (log aggregation). For distributed tracing, you add Grafana Tempo to replace X-Ray.

8. Auto-scaling alternatives#

HPA and VPA stay fully portable and behave the same on Quake AI.

For the Cluster Autoscaler on OpenStack, you set the --cloud-provider=openstack flag and define node groups by Nova server groups or Cluster API (CAPO).

9. Validation checklist#

• All pods in Running or Completed state
• PVCs bound to Cinder volumes with correct data
• Ingress routes return expected responses through the public LoadBalancer Service
• DNS resolves to the new Service address
• Prometheus scraping all targets (check Targets page in Grafana)
• Loki ingesting logs from all namespaces
• cert-manager certificates issued and valid
• Application health checks passing
• StatefulSet data integrity verified (database queries, object counts)
• No IRSA-related errors in pod logs
• No EBS/ALB/ECR references remaining in manifests

10. Provider-specific gotchas#

See also#

• Kubernetes migration overview: all provider guides and portability matrix
• Migrate from AWS: cross-service AWS migration hub
• Network migration from AWS VPC: networking-specific migration
• Compute migration from EC2: VM-level migration