# Migration Tools Comparison

Source: https://docs.quake.ai/docs/object/migration/tools-comparison
Markdown: https://docs.quake.ai/docs/object/migration/tools-comparison.md

---

# Migration tools comparison

Five tools cover the most common object storage migration and backup scenarios with Quake AI. Each has trade-offs in capability, speed, and ease of use.

## Feature comparison

| Feature | rclone | AWS CLI | s3cmd | MinIO Client (mc) | restic |
|---|---|---|---|---|---|
| Cloud-to-cloud copy | Yes | No; requires local staging | No; requires local staging | Yes | No; backup only |
| Preserves custom metadata | Yes (`--metadata`) | No | Server-side copy only (2.2.0+) | **Yes** | N/A |
| Multipart upload | Yes | Yes | Yes | Yes | Yes |
| Bandwidth limiting | Yes (`--bwlimit`) | No | Yes (`--limit-rate`) | Yes (`--limit-upload`) | No |
| Checksum verification | Yes (`rclone check`) | Yes (`--checksum`) | Yes (`--check-md5`) | Yes (`mc diff`) | Yes (built-in) |
| Incremental sync | Yes | Yes | Yes | Yes (`mc mirror --watch`) | Yes (dedup) |
| Encryption at rest | Yes (crypt remote) | No | No | No | Yes (built-in AES-256) |
| Deduplication | No | No | No | No | Yes (content-defined chunking) |
| Scheduling built-in | No; use cron | No; use cron | No; use cron | `--watch` mode for continuous | No; use cron |
| S3-compatible endpoint support | Yes | Yes (`--endpoint-url`) | Yes (`host_base`) | Yes (`mc alias set`) | Yes (via S3 backend) |

**Checksum verification differs by tool.** `rclone check` and `mc diff` run a standalone, bidirectional integrity comparison between source and destination. The AWS CLI `--checksum` flag and s3cmd `--check-md5` verify checksums during a transfer or sync, not as a separate audit pass. For a full integrity audit after a migration, run `rclone check` or `mc diff`.

## When to use each tool

Decision guide: match your migration scenario to the right tool. For full feature details, see the table above.

| Scenario | Recommended tool | Why |
|---|---|---|
| **Full migration** from another S3-compatible provider | **rclone** | Direct cloud-to-cloud copy, no local staging, parallel transfers, checksum verification |
| **Custom metadata must be preserved** | **MinIO Client** | Copies `x-amz-meta-*` headers between providers with `mc mirror`; current rclone (`--metadata`) also preserves them cross-provider. s3cmd preserves metadata only on same-endpoint server-side copies (2.2.0+) |
| **Encrypted, deduplicated backups** | **restic** | Built-in encryption, content-defined chunking, retention policies via `forget` |
| **AWS-centric workflow** (already using AWS CLI) | **AWS CLI** | Familiar commands, but requires local staging for cross-provider transfers |
| **Legacy S3 workflows** | **s3cmd** | Lightweight, config-file based, but requires local staging for cross-provider transfers |
| **Simple file-level backup** to Quake AI | **rclone** | `rclone copy` with cron: reliable, well-documented, cross-platform |

## Quake AI configuration

Each tool needs the Quake AI S3 endpoint, access key, and secret key. Generate credentials via [create S3 credentials](/docs/object/how-to/create-s3-credentials).

### rclone

Create a remote interactively with `rclone config`, or add directly to `~/.config/rclone/rclone.conf`:

```ini
[quakeai]
type = s3
provider = Ceph
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
endpoint = object.us-east-2.rumble.cloud
acl = private
```

Replace the endpoint with your region: `object.us-east-1.rumble.cloud`, `object.us-east-2.rumble.cloud`, or `object.us-west-1.rumble.cloud`.

Verify the connection:

```bash
rclone lsd quakeai:
```

### AWS CLI

Configure a named profile in `~/.aws/credentials` and `~/.aws/config`:

```ini
# ~/.aws/credentials
[quakeai]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
```

```ini
# ~/.aws/config
[profile rumble]
region = us-east-1
output = json
```

Pass the endpoint on every command:

```bash
aws s3 ls --profile rumble \
  --endpoint-url https://object.us-east-2.rumble.cloud
```



Set the region to `us-east-1` regardless of which Quake AI region you use. This is required for AWS Signature Version 4 compatibility.



### s3cmd

Create or edit `~/.s3cfg`:

```ini
[default]
access_key = YOUR_ACCESS_KEY
secret_key = YOUR_SECRET_KEY
host_base = object.us-east-2.rumble.cloud
host_bucket = %(bucket)s.object.us-east-2.rumble.cloud
use_https = True
signature_v2 = False
```

Verify:

```bash
s3cmd ls
```

### MinIO Client

```bash
mc alias set rumble https://object.us-east-2.rumble.cloud \
  YOUR_ACCESS_KEY YOUR_SECRET_KEY
```

Verify:

```bash
mc ls rumble
```

### restic

Initialize a repository on Quake AI S3:

```bash
export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY"
export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_KEY"
export RESTIC_REPOSITORY="s3:https://object.us-east-2.rumble.cloud/my-backup-repo"
export RESTIC_PASSWORD="YOUR_ENCRYPTION_PASSWORD"

restic init
```



Store your restic encryption password securely. If you lose it, your backups are unrecoverable.



## Performance tuning

### rclone (performance tuning)

```bash
rclone sync source:bucket quakeai:bucket \
  --transfers 16 \
  --checkers 32 \
  --s3-upload-concurrency 8 \
  --s3-chunk-size 64M \
  --progress
```

- `--transfers`: Parallel file transfers. Start with 16, increase for small files.
- `--checkers`: Parallel integrity checkers. 2x `--transfers` is a reasonable default.
- `--s3-chunk-size`: Multipart chunk size. Increase to 64M or 128M for large files.
- `--bwlimit`: Throttle bandwidth (e.g., `--bwlimit 100M` for 100 MB/s).

### AWS CLI (performance tuning)

```bash
aws configure set default.s3.max_concurrent_requests 20
aws configure set default.s3.multipart_chunksize 64MB
```

### MinIO client (performance tuning)

```bash
mc mirror --overwrite --watch \
  source/bucket rumble/bucket
```

The `--watch` flag enables continuous sync after the initial mirror.

## See also

- [Plan your migration](/docs/object/migration/plan-your-migration): feature compatibility, cost estimation, and validation checklist
- [Migrate from AWS S3](/docs/object/migration/migrate-from-s3): step-by-step guide using these tools
- [Set up Quake AI as a backup target](/docs/object/migration/backup-to-quake-ai): rclone and restic backup automation