Migration tools comparison

Five tools cover the most common object storage migration and backup scenarios with Quake AI. Each has trade-offs in capability, speed, and ease of use.

Feature comparison#

Checksum verification differs by tool. rclone check and mc diff run a standalone, bidirectional integrity comparison between source and destination. The AWS CLI --checksum flag and s3cmd --check-md5 verify checksums during a transfer or sync, not as a separate audit pass. For a full integrity audit after a migration, run rclone check or mc diff.

When to use each tool#

Decision guide: match your migration scenario to the right tool. For full feature details, see the table above.

Quake AI configuration#

Each tool needs the Quake AI S3 endpoint, access key, and secret key. Generate credentials via create S3 credentials.

rclone#

Create a remote interactively with rclone config, or add directly to ~/.config/rclone/rclone.conf:

[quakeai]
type = s3
provider = Ceph
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
endpoint = object.us-east-2.rumble.cloud
acl = private

Replace the endpoint with your region: object.us-east-1.rumble.cloud, object.us-east-2.rumble.cloud, or object.us-west-1.rumble.cloud.

Verify the connection:

rclone lsd quakeai:

AWS CLI#

Configure a named profile in ~/.aws/credentials and ~/.aws/config:

# ~/.aws/credentials
[quakeai]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY

# ~/.aws/config
[profile rumble]
region = us-east-1
output = json

Pass the endpoint on every command:

aws s3 ls --profile rumble \
  --endpoint-url https://object.us-east-2.rumble.cloud

s3cmd#

Create or edit ~/.s3cfg:

[default]
access_key = YOUR_ACCESS_KEY
secret_key = YOUR_SECRET_KEY
host_base = object.us-east-2.rumble.cloud
host_bucket = %(bucket)s.object.us-east-2.rumble.cloud
use_https = True
signature_v2 = False

Verify:

s3cmd ls

MinIO Client#

mc alias set rumble https://object.us-east-2.rumble.cloud \
  YOUR_ACCESS_KEY YOUR_SECRET_KEY

Verify:

mc ls rumble

restic#

Initialize a repository on Quake AI S3:

export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_KEY"
export RESTIC_REPOSITORY="s3:https://object.us-east-2.rumble.cloud/my-backup-repo"
export RESTIC_PASSWORD="YOUR_ENCRYPTION_PASSWORD"

restic init

Performance tuning#

rclone (performance tuning)#

rclone sync source:bucket quakeai:bucket \
  --transfers 16 \
  --checkers 32 \
  --s3-upload-concurrency 8 \
  --s3-chunk-size 64M \
  --progress

• --transfers: Parallel file transfers. Start with 16, increase for small files.
• --checkers: Parallel integrity checkers. 2x --transfers is a reasonable default.
• --s3-chunk-size: Multipart chunk size. Increase to 64M or 128M for large files.
• --bwlimit: Throttle bandwidth (e.g., --bwlimit 100M for 100 MB/s).

AWS CLI (performance tuning)#

aws configure set default.s3.max_concurrent_requests 20
aws configure set default.s3.multipart_chunksize 64MB

MinIO client (performance tuning)#

mc mirror --overwrite --watch \
  source/bucket rumble/bucket

The --watch flag enables continuous sync after the initial mirror.

See also#

• Plan your migration: feature compatibility, cost estimation, and validation checklist
• Migrate from AWS S3: step-by-step guide using these tools
• Set up Quake AI as a backup target: rclone and restic backup automation