How to migrate from Azure blob storage to Quake AI

This guide walks through migrating objects from an Azure Blob Storage container to Quake AI object storage using rclone for direct cloud-to-cloud transfer.

Service mapping#

Prerequisites#

• A Quake AI account with S3 credentials
• An Azure Storage Account SAS token with Read and List permissions, or an Azure service principal with Storage Blob Data Reader role
• rclone installed (v1.65+)

Before you start#

Read the migration planning guide to review S3/Swift feature compatibility and build a validation checklist.

Azure-specific considerations:

• Terminology mapping: Azure uses "containers" (equivalent to S3 buckets) and "blobs" (equivalent to S3 objects). Quake AI uses both "containers" and "buckets" interchangeably.
• Access tiers: Azure Blob offers Hot, Cool, Cold, and Archive tiers, plus a smart tier option that automatically moves data across the Hot, Cool, and Cold tiers based on access patterns (it does not cover Archive). Data on Quake AI is stored on a single tier. Two Azure tier behaviors affect migration timing and cost:
  • Minimum retention and early deletion: Azure sets a minimum retention period for cooler tiers: 30 days for Cool, 90 days for Cold, and 180 days for Archive. If you delete source objects after migrating them, objects still inside their minimum period incur an early deletion fee. Check each object's tier and age before you clean up the source.
  • Archive rehydration: Objects in the Archive tier must be rehydrated before they can be copied. Standard-priority rehydration can take up to 15 hours; high-priority rehydration can finish in under 1 hour for objects under 10 GB but costs more. Rehydration incurs a per-GB fee.
• RBAC and ABAC: Azure uses Entra ID (Azure AD) with RBAC and ABAC policies. These do not translate to Swift ACLs. Re-apply access controls on Quake AI using grant access control.
• Lifecycle management: Azure lifecycle policies (tier transitions, expiration) are not supported on Quake AI. See retention without lifecycle policies.
• Hierarchical namespace (ADLS Gen2): If your Azure Storage Account has hierarchical namespace enabled (Azure Data Lake Storage Gen2), rclone's azureblob backend supports it but may require adls = true in the remote config, and SAS token scoping can differ. Verify your settings against the rclone Azure Blob documentation.

Configure rclone remotes#

Using a SAS token#

Generate a SAS token in the Azure portal: Storage Account > Shared access signature > grant Read and List > generate.

Add both remotes to ~/.config/rclone/rclone.conf:

[azure]
type = azureblob
account = YOUR_STORAGE_ACCOUNT_NAME
sas_url = https://YOUR_ACCOUNT.blob.core.windows.net/?YOUR_SAS_TOKEN

[quakeai]
type = s3
provider = Ceph
access_key_id = YOUR_RUMBLE_ACCESS_KEY
secret_access_key = YOUR_RUMBLE_SECRET_KEY
endpoint = object.us-east-2.rumble.cloud
acl = private

Using a service principal#

[azure]
type = azureblob
account = YOUR_STORAGE_ACCOUNT_NAME
tenant = YOUR_TENANT_ID
client_id = YOUR_CLIENT_ID
client_secret = YOUR_CLIENT_SECRET

Verify both remotes:

rclone lsd azure:
rclone lsd quakeai:

Create the destination bucket#

rclone mkdir quakeai:my-bucket

Run the migration#

Initial copy#

rclone copy azure:my-source-container quakeai:my-bucket \
  --transfers 16 \
  --checkers 32 \
  --s3-chunk-size 64M \
  --progress \
  --log-file migration.log \
  --log-level INFO

Delta syncs#

rclone sync azure:my-source-container quakeai:my-bucket \
  --transfers 16 \
  --checkers 32 \
  --progress

Schedule daily until cutover.

Estimate egress cost#

Azure charges per-GB for egress, with rates that vary by region. Azure does not currently run a free egress program for migrations. For current rates, see the Azure bandwidth pricing page.

For large data sets, consider a phased migration or check if your Azure support plan includes migration credits.

Verify the migration#

rclone size azure:my-source-container
rclone size quakeai:my-bucket

Check file integrity:

rclone check azure:my-source-container quakeai:my-bucket \
  --one-way \
  --log-file check.log

See the validation checklist for the full procedure.

Cut over#

1. Run a final rclone sync.
2. Update your application's storage endpoint. See update application endpoint.
3. Keep the Azure source container for 30 days as a rollback target.

See also#

• Plan your migration: compatibility matrix, cost estimation, validation checklist
• Migration tools comparison: rclone configuration and alternatives
• Update application endpoint: swap SDK config to Quake AI