# How to Back Up to Quake AI Object Storage Source: https://docs.quake.ai/docs/object/migration/backup-to-quake-ai Markdown: https://docs.quake.ai/docs/object/migration/backup-to-quake-ai.md --- # How to back up to Quake AI object storage Use Quake AI object storage as an off-site backup target in a 3-2-1 strategy: three copies of your data, on two different media, with one copy off-site. This guide covers file-level backups with rclone, encrypted backups with restic, database backups, and cron-based automation. You do not need to migrate your primary storage to Quake AI to use this guide. Quake AI works as a standalone backup destination alongside any primary provider. ## Prerequisites - A Quake AI account with [S3 credentials](/docs/object/how-to/create-s3-credentials) - A dedicated backup bucket on Quake AI (see [create a bucket](/docs/object/how-to/create-container)) - [rclone](https://rclone.org/install/) installed (v1.65+) for file-level backups - [restic](https://restic.readthedocs.io/en/latest/020_installation.html) installed (v0.16+) for encrypted, deduplicated backups - A Linux server, macOS workstation, or VM with cron or systemd available ## Configure rclone for Quake AI Add Quake AI as an rclone remote. Edit `~/.config/rclone/rclone.conf`: ```ini [quakeai] type = s3 provider = Ceph access_key_id = YOUR_ACCESS_KEY secret_access_key = YOUR_SECRET_KEY endpoint = object.us-east-2.rumble.cloud acl = private ``` Replace the endpoint with your region. Verify the connection: ```bash rclone lsd quakeai: ``` See [tools comparison](/docs/object/migration/tools-comparison) for detailed rclone configuration. ## Copy from an SFTP source Some providers expose file storage over SFTP instead of S3. [Hetzner Storage Box](https://docs.hetzner.com/storage/storage-box/) is one example: it speaks SFTP, FTP, SMB, and WebDAV, not S3. Add an SFTP remote as the source alongside your existing `rumble` remote: ```ini [storagebox] type = sftp host = u123456.your-storagebox.de user = u123456 pass = YOUR_PASSWORD ``` Replace the host and user with the values from your Storage Box settings. For key-based auth, use `key_file` instead of `pass`. Run `rclone config` if you prefer an interactive setup. Copy from the SFTP remote to Quake AI: ```bash rclone copy storagebox:backup-path quakeai:my-backup-bucket/storagebox \ --transfers 8 \ --progress ``` Use `copy`, not `sync`, so deletions on the source do not propagate to your backup bucket. ## File-level backup with rclone ### One-time backup Copy a local directory to Quake AI. Use `copy`, not `sync`: `copy` only adds and updates files on the destination and never deletes, which protects against propagating local deletions to your backup. ```bash rclone copy /path/to/data quakeai:my-backup-bucket/daily \ --transfers 8 \ --progress ``` ### Exclude temporary files ```bash rclone copy /path/to/data quakeai:my-backup-bucket/daily \ --exclude "*.tmp" \ --exclude ".cache/**" \ --exclude "node_modules/**" \ --transfers 8 \ --progress ``` ### Verify the backup ```bash rclone check /path/to/data quakeai:my-backup-bucket/daily --one-way ``` This compares every local file against the remote copy and reports mismatches. ## Encrypted backup with restic restic provides built-in AES-256 encryption, content-defined deduplication, and retention policy management. Backups are encrypted before leaving your machine. ### Initialize the repository ```bash export AWS_ACCESS_KEY_ID="YOUR_ACCESS_KEY" export AWS_SECRET_ACCESS_KEY="YOUR_SECRET_KEY" export RESTIC_REPOSITORY="s3:https://object.us-east-2.rumble.cloud/my-backup-repo" export RESTIC_PASSWORD="YOUR_ENCRYPTION_PASSWORD" restic init ``` Store your restic password in a password manager or secrets vault. If you lose it, your backups are permanently unrecoverable. Consider also backing up the password to a separate, secure location. ### Run a backup ```bash restic backup /path/to/data \ --exclude ".cache" \ --exclude "node_modules" \ --verbose ``` restic deduplicates at the chunk level; subsequent backups transfer only changed blocks, saving bandwidth and storage. ### Apply a retention policy ```bash restic forget \ --keep-daily 7 \ --keep-weekly 4 \ --keep-monthly 12 \ --prune ``` This keeps 7 daily, 4 weekly, and 12 monthly snapshots, then removes unreferenced data with `--prune`. ### Verify backup integrity ```bash restic check ``` Run this periodically (monthly) to verify that all data in the repository is intact and readable. ### Restore from backup List available snapshots: ```bash restic snapshots ``` Restore the latest snapshot to a local directory: ```bash restic restore latest --target /path/to/restore ``` Restore a specific file or directory: ```bash restic restore latest --target /tmp/restore --include "/path/to/specific/file" ``` ## Database backups Object storage is a natural destination for database dumps. The pattern is: dump → compress → upload → manage retention. ### PostgreSQL ```bash #!/bin/bash TIMESTAMP=$(date +%Y%m%d-%H%M%S) BACKUP_DIR="/tmp/db-backups" BUCKET="quakeai:my-backup-bucket/postgres" mkdir -p "$BACKUP_DIR" pg_dump -U postgres -h localhost my_database \ | gzip > "${BACKUP_DIR}/my_database-${TIMESTAMP}.sql.gz" rclone copy "${BACKUP_DIR}/" "${BUCKET}/" --progress rm -f "${BACKUP_DIR}/my_database-${TIMESTAMP}.sql.gz" ``` ### MySQL / MariaDB ```bash #!/bin/bash TIMESTAMP=$(date +%Y%m%d-%H%M%S) BACKUP_DIR="/tmp/db-backups" BUCKET="quakeai:my-backup-bucket/mysql" mkdir -p "$BACKUP_DIR" mysqldump -u root --single-transaction --quick my_database \ | gzip > "${BACKUP_DIR}/my_database-${TIMESTAMP}.sql.gz" rclone copy "${BACKUP_DIR}/" "${BUCKET}/" --progress rm -f "${BACKUP_DIR}/my_database-${TIMESTAMP}.sql.gz" ``` ### SQLite SQLite databases are single files. Use the `.backup` command for a consistent copy, then upload: ```bash #!/bin/bash TIMESTAMP=$(date +%Y%m%d-%H%M%S) BACKUP_DIR="/tmp/db-backups" BUCKET="quakeai:my-backup-bucket/sqlite" mkdir -p "$BACKUP_DIR" sqlite3 /path/to/database.db ".backup '${BACKUP_DIR}/database-${TIMESTAMP}.db'" gzip "${BACKUP_DIR}/database-${TIMESTAMP}.db" rclone copy "${BACKUP_DIR}/" "${BUCKET}/" --progress rm -f "${BACKUP_DIR}/database-${TIMESTAMP}.db.gz" ``` ### Database backup retention Delete database dumps older than your retention period: ```bash rclone delete quakeai:my-backup-bucket/postgres --min-age 90d rclone delete quakeai:my-backup-bucket/mysql --min-age 90d ``` ## Automate with cron ### rclone nightly backup Edit your crontab with `crontab -e`: ```cron # File backup: nightly at 02:00 0 2 * * * /usr/bin/rclone copy /path/to/data quakeai:my-backup-bucket/daily --transfers 8 --log-file /var/log/rclone-backup.log --log-level INFO # Database backup: nightly at 02:30 30 2 * * * /usr/local/bin/backup-postgres.sh >> /var/log/db-backup.log 2>&1 # Retention cleanup: weekly on Sunday at 04:00 0 4 * * 0 /usr/bin/rclone delete quakeai:my-backup-bucket/daily --min-age 90d --log-file /var/log/rclone-cleanup.log --log-level INFO ``` ### restic nightly backup ```cron # Encrypted backup: nightly at 02:00 0 2 * * * /usr/bin/restic backup /path/to/data --exclude ".cache" --quiet >> /var/log/restic-backup.log 2>&1 # Retention policy: weekly on Sunday at 04:00 0 4 * * 0 /usr/bin/restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --prune --quiet >> /var/log/restic-forget.log 2>&1 # Integrity check: first of the month at 05:00 0 5 1 * * /usr/bin/restic check --quiet >> /var/log/restic-check.log 2>&1 ``` For restic cron jobs, set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `RESTIC_REPOSITORY`, and `RESTIC_PASSWORD` as environment variables in the crontab or in a sourced env file. Do not hardcode credentials in scripts committed to version control. ### systemd timer alternative For systems using systemd, create a service and timer pair instead of cron. This provides better logging via `journalctl` and automatic retry on failure. `/etc/systemd/system/rclone-backup.service`: ```ini [Unit] Description=Rclone backup to Quake AI [Service] Type=oneshot ExecStart=/usr/bin/rclone copy /path/to/data quakeai:my-backup-bucket/daily --transfers 8 --log-level INFO ``` `/etc/systemd/system/rclone-backup.timer`: ```ini [Unit] Description=Run rclone backup nightly [Timer] OnCalendar=*-*-* 02:00:00 Persistent=true [Install] WantedBy=timers.target ``` Enable and start: ```bash sudo systemctl enable --now rclone-backup.timer ``` ## Enable versioning and encryption For additional protection on your backup bucket: - **Versioning** preserves previous versions of overwritten objects. Enable it so accidental overwrites do not destroy backup data. See [enable versioning](/docs/object/how-to/enable-versioning). - **Server-side encryption** encrypts data at rest on Quake AI's storage layer. See [server-side encryption](/docs/object/how-to/configure-server-side-encryption). If using restic, data is already encrypted client-side before upload; server-side encryption is an additional layer. ## Monitor and test - **Check logs** after each automated run. Both rclone and restic write structured logs. Monitor for errors, especially auth failures and network timeouts. - **Test restores quarterly.** A backup you have never restored from is a backup you cannot trust. Download a sample of files and verify integrity. - **Alert on failure.** Wrap your cron jobs in a script that sends a notification (email, Slack webhook, or monitoring system alert) if the exit code is non-zero. ## Cost estimation Object storage on Quake AI is a per-TB add-on, and each dedicated vCPU subscription includes 1 TB at no additional cost. Backup sets that fit within the included 1 TB do not add to the bill; sets that exceed the allowance scale per additional TB. For current rates, see the canonical [Quake AI pricing page](https://www.quake.ai/pricing/). restic's deduplication typically reduces stored data to 30-60% of the source size after multiple backup cycles, lowering effective storage requirements. ## See also - [Plan your migration](/docs/object/migration/plan-your-migration): S3 compatibility matrix and feature gap workarounds - [Migration tools comparison](/docs/object/migration/tools-comparison): rclone vs restic vs other tools - [Multi-cloud sync](/docs/object/migration/multi-cloud-sync): continuous replication between Quake AI and other providers - [Enable versioning](/docs/object/how-to/enable-versioning) - [Server-side encryption](/docs/object/how-to/configure-server-side-encryption)