Plan your object storage migration

Quake AI object storage exposes an S3-compatible API, so many tools and SDKs built for AWS S3 also work with Quake AI. Before migrating data, review what is and is not supported so you can plan around feature gaps.

S3 feature compatibility#

Quake AI object storage runs on OpenStack Swift with a Ceph backend and exposes an S3-compatible API. Core object operations work as expected. These advanced S3 features are not available.

Supported#

Not supported#

Inventory your data#

Before starting a transfer, collect these numbers from your source provider:

Estimate transfer time and cost#

Egress fees from your source provider#

Egress pricing varies widely across providers and changes regularly. Confirm current rates on each provider's pricing page before estimating the migration bill:

Transfer time estimate#

Rough formula for bulk transfers over a stable connection:

Hours = (Total GB × 8) / (Bandwidth Mbps × 3600 × Utilization)

At 1 Gbps with 60% utilization, 1 TB takes approximately 3.7 hours; 10 TB takes about 37 hours. At 100 Mbps with 60% utilization, 1 TB takes ~37 hours. Add 20-30% overhead for S3 API request latency on high object counts (over 1 million objects).

Object storage cost#

Object storage on Quake AI is a per-TB add-on. Backups you store there add to stored capacity; extra terabytes are priced per additional TB. For current add-on rates, see the pricing reference and the Quake AI pricing page.

Set up Quake AI S3 credentials#

Migration tools need an access key and secret key for the Quake AI S3 endpoint. Generate these before starting any transfer.

1. Follow create S3 credentials to generate a key pair.
2. Note your region endpoint:
   • object.us-east-1.rumble.cloud
   • object.us-east-2.rumble.cloud
   • object.us-west-1.rumble.cloud
3. Set the S3 region to us-east-1 for AWS Signature Version 4 compatibility (required regardless of which Quake AI region you use).

Choose a migration strategy#

For large data sets, a phased runbook looks like this: run an initial bulk copy, schedule daily delta syncs to catch changes, then run a final sync and cut over during a maintenance window.

Retention without lifecycle policies#

Since Quake AI does not support S3 lifecycle policies, implement retention externally:

With rclone:

rclone delete quakeai:my-bucket --min-age 90d

Run this on a cron schedule to expire objects older than your retention period.

With a shell script:

#!/bin/bash
BUCKET="my-bucket"
RETENTION_DAYS=90
rclone delete "quakeai:${BUCKET}" --min-age "${RETENTION_DAYS}d" --verbose

Validation checklist#

Run these checks after each migration or sync pass:

• Object count matches: Compare source and destination: rclone size source:bucket vs rclone size quakeai:bucket
• Total size matches: Same commands. Allow for minor differences due to metadata overhead.
• Spot-check file integrity: Download 5-10 random objects from both sides and compare checksums: rclone check source:bucket quakeai:bucket --one-way
• ACLs are correct: Verify public/private access state on critical buckets.
• Application connectivity: Point a test instance at the Quake AI endpoint and confirm retrieval and upload operations succeed.
• Versioning state: If versioning was enabled on the source, enable it on the destination: see enable versioning.
• Encryption state: If objects were encrypted, verify encryption is configured on Quake AI: see server-side encryption.

Rollback plan#

Keep your source data intact for at least 30 days after cutover. During this period:

1. Run daily rclone check between source and destination to catch drift.
2. If issues arise, revert your application's endpoint configuration to the source provider.
3. After 30 days with no issues, decommission the source and cancel the source provider subscription.

Next steps#

• Choose a migration tool: feature comparison of rclone, AWS CLI, s3cmd, MinIO Client, and restic
• Migrate from AWS S3: step-by-step guide for the most common migration path
• Set up Quake AI as a backup target: use Quake AI for 3-2-1 backups without a full migration