How to run blue/green or canary deployments on Quake AI
Coming from another cloud?
▸AWS·Elastic Load Balancing, ALB Listener Rules
Elastic Load Balancing
- Quake AI workloads use a self-managed reverse proxy or Kubernetes LoadBalancer Service.
- AWS hourly/LCU billing.
- OpenStack VM billing.
- AWS global endpoints.
▸Azure·Load Balancer
Azure Load Balancer
- Pure L4 (TCP/UDP) pass-through; no L7 features (use App Gateway) ().
- SKUs: Standard w/ zones/HA, health probes, outbound SNAT rules; Basic free but retiring 2025.
- Azure provides a regional managed service integrated with VNets. Quake AI workloads use a self-managed reverse proxy or Kubernetes LoadBalancer Service.
- API via ARM; frontend configs w/ multiple IPs/ports.
▸Google Cloud·Cloud Load Balancing
Cloud Load Balancing
- GCP offers external and internal, global and regional, HTTP, TCP, and UDP load balancers. Quake AI workloads use a self-managed reverse proxy or Kubernetes LoadBalancer Service.
- GCP global HTTP(S) load balancing uses one anycast IP across regions. Self-managed Quake AI edges use the public IP and topology you provision.
- GCP integrates Cloud CDN, Cloud Armor (WAF/DDoS) with LBs; OpenStack has no native CDN/WAF integration.
- GCP backend services use health checks and instance groups or NEGs. Self-managed Quake AI edges use proxy upstreams or Kubernetes service selectors.
How to run blue/green or canary deployments on Quake AI
Cut over to a new application version by routing traffic through a reverse proxy that you operate. This guide uses HAProxy because its runtime API can change backend weights without restarting the proxy. Nginx, Caddy, Traefik, and Envoy can implement the same pattern with their own configuration and control interfaces.
Prerequisites
- CLIOpenStack CLI installed and authenticated (
clouds.yamloropenrcsourced)
Windows: CLI examples use bash. Set up a Linux CLI environment on Windows before proceeding.
- A reverse proxy instance with a floating IP
- Running instances for the current ("blue") version and the new ("green") version on a private network
- A health endpoint, such as
/healthz, on both versions - Security group rules that allow the proxy instance to reach the application port
Configure the proxy backends#
Install HAProxy on the reverse proxy instance:
sudo apt update
sudo apt install -y haproxy socatAdd blue and green servers to one backend in /etc/haproxy/haproxy.cfg:
frontend public_https
bind :443 ssl crt /etc/haproxy/certs/example.com.pem
default_backend application
backend application
option httpchk GET /healthz
http-check expect status 200
server blue-1 10.0.0.11:8080 check weight 100
server green-1 10.0.0.21:8080 check weight 0Enable the HAProxy runtime socket in the global section:
global
stats socket /run/haproxy/admin.sock mode 660 level adminValidate and reload the configuration:
sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxyVerify the green version#
Test the green instance from the proxy before sending public traffic to it:
curl -fsS http://10.0.0.21:8080/healthzCheck HAProxy's view of both backends:
echo "show stat" | sudo socat stdio /run/haproxy/admin.sockThe green server must report UP before you change its weight.
Run a blue/green cutover#
Move traffic to green by setting green to full weight and blue to zero:
echo "set weight application/green-1 100%" | sudo socat stdio /run/haproxy/admin.sock
echo "set weight application/blue-1 0%" | sudo socat stdio /run/haproxy/admin.sockKeep the blue version running during the verification window. The proxy's floating IP and DNS records stay unchanged.
To roll back:
echo "set weight application/blue-1 100%" | sudo socat stdio /run/haproxy/admin.sock
echo "set weight application/green-1 0%" | sudo socat stdio /run/haproxy/admin.sockRun a canary rollout#
Start green at 10%:
echo "set weight application/blue-1 90%" | sudo socat stdio /run/haproxy/admin.sock
echo "set weight application/green-1 10%" | sudo socat stdio /run/haproxy/admin.sockWatch application errors, latency, and business metrics. Increase green in stages after each observation window:
echo "set weight application/blue-1 50%" | sudo socat stdio /run/haproxy/admin.sock
echo "set weight application/green-1 50%" | sudo socat stdio /run/haproxy/admin.sockComplete the rollout by setting green to 100% and blue to zero. Roll back at any stage by restoring blue to 100%.
Persist the final state#
Runtime weight changes do not survive a proxy restart. After the rollout, update the server weights in /etc/haproxy/haproxy.cfg, validate the file, and reload HAProxy:
sudo haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxySee also#
Usage Guidelines
The sample code, software libraries, command line tools, proofs of concept, templates, and other related technology on this page (including any of the foregoing that is provided by Quake AI personnel) is provided to you as Quake AI Content under the Quake AI Customer Agreement, or the relevant written agreement between you and Quake AI (whichever applies). Do not use this Quake AI Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the Quake AI Content (such as sample code) as appropriate for production grade use based on your specific quality control practices and standards. Deploying Quake AI Content may incur Quake AI charges for creating or using Quake AI chargeable resources, such as running Compute instances or storing data in Object Storage. Your use is also subject to the Acceptable Use Policy.
For the full policy, see Usage Guidelines.