How to migrate from Backblaze B2 to Quake AI

Backblaze B2 supports both its native API and an S3-compatible API. rclone handles both, copying data directly to Quake AI with no local staging required.

Prerequisites#

• A Quake AI account with S3 credentials
• A Backblaze B2 application key with read access to the source bucket (generated in the B2 Cloud Storage console under App Keys)
• rclone installed (v1.65+)

Before you start#

Read the migration planning guide for the S3/Swift compatibility matrix.

Backblaze-specific considerations:

• Egress pricing: B2 meters direct egress per GB. If your B2 bucket is fronted by Cloudflare (Bandwidth Alliance), egress from B2 to Cloudflare is free, so a Cloudflare-fronted bucket migrates without egress charges. Verify the current integration terms on the Backblaze Cloudflare integration page and current rates on the Backblaze B2 pricing page.
• B2 native vs S3-compatible API: rclone supports both. The B2-native remote (type = b2) is faster for listing operations. The S3-compatible remote works but has slightly higher API overhead.
• File caps: B2 has no hard per-bucket file count limit. The B2 web console only displays up to 10 million files, so manage larger buckets through the CLI or API. B2 limits each account to 100 buckets, which can affect planning if you split data across many buckets.
• Lifecycle rules: B2 lifecycle rules (file version retention, hide-after-days) do not transfer. Implement retention on Quake AI with external tooling. See retention without lifecycle policies.
• Multipart uploads: B2 supports multipart uploads, which it calls large files, for files from 5 MB to 10 TB. The recommended part size is 100 MB and the minimum is 5 MB. rclone selects multipart thresholds and part sizes automatically.

Configure rclone remotes#

Using the B2-native remote for the source (faster listing):

[b2]
type = b2
account = YOUR_B2_KEY_ID
key = YOUR_B2_APPLICATION_KEY

[quakeai]
type = s3
provider = Ceph
access_key_id = YOUR_RUMBLE_ACCESS_KEY
secret_access_key = YOUR_RUMBLE_SECRET_KEY
endpoint = object.us-east-2.rumble.cloud
acl = private

Alternatively, use B2's S3-compatible endpoint. The B2-native remote above is the simpler option because it needs no region-specific endpoint. B2's S3-compatible endpoints are both region-specific and bucket-specific, so look yours up before configuring this remote.

[b2s3]
type = s3
provider = Other
access_key_id = YOUR_B2_KEY_ID
secret_access_key = YOUR_B2_APPLICATION_KEY
endpoint = s3.YOUR_B2_REGION.backblazeb2.com

To find your bucket's endpoint, open the B2 console and go to Buckets > your bucket > Bucket Settings > Endpoint. The value follows the format s3.<region>.backblazeb2.com (for example, s3.us-west-004.backblazeb2.com). The b2_authorize_account API call also returns it. Replace the Quake AI endpoint with your region.

Verify:

rclone lsd b2:
rclone lsd quakeai:

Create the destination bucket#

rclone mkdir quakeai:my-bucket

Run the migration#

Initial copy#

rclone copy b2:my-source-bucket quakeai:my-bucket \
  --transfers 16 \
  --checkers 32 \
  --s3-chunk-size 64M \
  --progress \
  --log-file migration.log \
  --log-level INFO

Delta syncs#

rclone sync b2:my-source-bucket quakeai:my-bucket \
  --transfers 16 \
  --checkers 32 \
  --progress

Schedule daily until cutover.

Estimate egress cost#

B2 meters direct egress per GB. If your B2 bucket is already fronted by Cloudflare (Bandwidth Alliance), egress from B2 to Cloudflare is free. For current B2 egress rates, see the Backblaze B2 pricing page. For the integration terms, see the Backblaze Cloudflare integration page.

Verify the migration#

rclone size b2:my-source-bucket
rclone size quakeai:my-bucket

Check file integrity:

rclone check b2:my-source-bucket quakeai:my-bucket \
  --one-way \
  --log-file check.log

See the validation checklist for the full procedure.

Cut over#

1. Run a final rclone sync.
2. Update your application's S3 endpoint configuration. See update application endpoint.
3. Keep the B2 source bucket for 30 days as a rollback target.

See also#

• Plan your migration: compatibility matrix, validation checklist
• Migration tools comparison: rclone configuration and alternatives
• Update application endpoint: swap SDK config to Quake AI