# Store and retrieve files with S3-compatible object storage

Source: https://docs.quake.ai/docs/quickstart/object-storage-upload
Markdown: https://docs.quake.ai/docs/quickstart/object-storage-upload.md

---

# Store and retrieve files with S3-compatible object storage

In this tutorial, we create S3 credentials, configure the AWS CLI to point at Quake AI's object storage endpoint, create a storage container, upload and download files, and apply a basic access policy. By the end, you will have a working object storage workflow that you can use from any S3-compatible tool or SDK.

**What you will learn:**

- What S3 credentials are and how they differ from your Quake AI login
- How to configure the AWS CLI to work with Quake AI's S3-compatible endpoint
- How containers (buckets) work in Quake AI object storage
- How to upload, list, and download files using both the console and the AWS CLI
- How to apply a read-only access policy to a container

**Time estimate:** 20–30 minutes

## Prerequisites

You need:

- A Quake AI account with an active project
- The AWS CLI installed on your local machine:

```bash
pip install awscli
```

Verify the installation:

```bash
aws --version
```

You do not need an AWS account. The AWS CLI works with any S3-compatible storage endpoint, including Quake AI's.

## Why S3 compatibility matters

Quake AI Object Storage implements the S3 API. This means every tool built for Amazon S3, the AWS CLI, `s3cmd`, `rclone`, `boto3`, and dozens of backup and sync utilities, works with Quake AI storage without modification. You point the tool at a different endpoint URL and use Quake AI's credentials instead of AWS credentials.

This compatibility is deliberate. Migrating data to and from Quake AI does not require rewriting integrations, only reconfiguring endpoints.

## Step 1: Create S3 credentials

S3 credentials are separate from your Quake AI console login. They consist of an access key and a secret key, scoped to your project, used exclusively for S3-compatible storage access.

1. In the Quake AI console, go to **Storage** > **Object Storage** > **S3 Credentials**.
2. Select **Create S3 Credential**.
3. Enter a name for the credential set, for example `tutorial-credentials`.
4. Select **OK**.
5. The console displays the **Access Key** and **Secret Key**. Copy both values and store them somewhere secure. The secret key is shown only once and cannot be retrieved again.
6. Select **Close**.



The secret key is not stored by Quake AI after you close this dialog. If you lose it, you will need to create a new credential set.



## Step 2: Configure the AWS CLI

Configure the AWS CLI with your Quake AI S3 credentials and endpoint. We use a named profile (`rumble`) to keep these settings separate from any existing AWS configuration on your machine.

```bash
aws configure --profile rumble
```

At each prompt, enter:

```text
AWS Access Key ID [None]: YOUR_ACCESS_KEY
AWS Secret Access Key [None]: YOUR_SECRET_KEY
Default region name [None]: us-east-1
Default output format [None]: json
```

Replace `YOUR_ACCESS_KEY` and `YOUR_SECRET_KEY` with the values from Step 1.

Now set the S3 endpoint for this profile. The AWS CLI reads a per-profile `endpoint_url` from `~/.aws/config`. The endpoint URL is specific to the region your project is in:

| Region | Endpoint |
|--------|----------|
| US East 1 | `https://object.us-east-1.rumble.cloud` |
| US East 2 | `https://object.us-east-2.rumble.cloud` |
| US West 1 | `https://object.us-west-1.rumble.cloud` |

Open `~/.aws/config` and add the `endpoint_url` line to the `[profile rumble]` section that `aws configure` created:

```ini
[profile rumble]
region = us-east-1
output = json
endpoint_url = https://object.us-east-1.rumble.cloud
```



The AWS CLI v2 ignores the legacy nested `s3.endpoint_url` config key for the `s3 ls` command and several other high-level S3 commands, so setting it with `configure set` silently sends the request to Amazon S3 and fails with `InvalidAccessKeyId`. Set the per-profile `endpoint_url` key shown above. For scripts and CI, export `AWS_ENDPOINT_URL=https://object.us-east-1.rumble.cloud` instead, which the CLI reads natively.



Verify the configuration by listing your containers (the list is empty, which is expected):

```bash
aws s3 ls --profile rumble
```

No error means the credentials and endpoint are configured correctly.

## Step 3: Create a container

Object storage organizes files into containers. Containers are the equivalent of S3 buckets: a flat namespace for storing objects. Container names must be unique within your project.

**Console:**

1. Go to **Storage** > **Object Storage** > **Create Container**.
2. Enter the name `tutorial-bucket`.
3. Leave the access set to private (not publicly accessible).
4. Select **OK**.

The container appears in the list immediately.

**CLI equivalent:**

```bash
aws s3 mb s3://tutorial-bucket --profile rumble
```

Both methods create the same result. The console is easier for one-off creation; the CLI is better for scripting and automation.



Quake AI's console uses the term "container" (from the underlying Swift API). The S3 API calls the same concept a "bucket." They are the same thing. When you create a container in the console, it is accessible as a bucket via the S3 endpoint, and vice versa.



## Step 4: Upload a file

**Console:**

1. Go to **Storage** > **Object Storage** and select `tutorial-bucket`.
2. Select **Upload File**.
3. Choose a file from your local machine (any small file works: a text file or image is fine).
4. Select **OK**.

The file appears in the container listing.

**CLI:**

Upload a file with the AWS CLI:

```bash
echo "hello from Quake AI" > /tmp/hello.txt
aws s3 cp /tmp/hello.txt s3://tutorial-bucket/hello.txt --profile rumble
```

Upload an entire local directory:

```bash
aws s3 sync /tmp/my-folder s3://tutorial-bucket/my-folder --profile rumble
```

The `sync` command uploads only files that are new or changed, making it efficient for incremental backups and deployments.

## Step 5: List and download files

List the contents of the container:

```bash
aws s3 ls s3://tutorial-bucket --profile rumble
```

Download a file:

```bash
aws s3 cp s3://tutorial-bucket/hello.txt /tmp/hello-downloaded.txt --profile rumble
cat /tmp/hello-downloaded.txt
```

The downloaded file contains the same content you uploaded. Files in object storage are retrieved via key (their path within the container) and are stored with full consistency: what you put in is exactly what you get back.

## Step 6: Set a basic access policy

By default, objects in your container are private: only requests authenticated with your S3 credentials can access them. You can make a container or individual objects publicly readable using a bucket policy.

The following policy makes all objects in `tutorial-bucket` publicly readable via HTTPS, without credentials:

```bash
aws s3api put-bucket-policy \
  --bucket tutorial-bucket \
  --policy '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": "*",
        "Action": "s3:GetObject",
        "Resource": "arn:aws:s3:::tutorial-bucket/*"
      }
    ]
  }' \
  --profile rumble
```

After applying the policy, objects in `tutorial-bucket` are accessible at:

```text
https://object.us-east-1.rumble.cloud/YOUR_PROJECT_ID:tutorial-bucket/OBJECT_KEY
```

Replace `YOUR_PROJECT_ID` with your cloud project ID and `OBJECT_KEY` with the filename or path of the object. For `hello.txt`, the URL is:

```text
https://object.us-east-1.rumble.cloud/YOUR_PROJECT_ID:tutorial-bucket/hello.txt
```

The `YOUR_PROJECT_ID:` prefix is required for anonymous reads on Quake AI's S3-compatible endpoint. The plain `tutorial-bucket/OBJECT_KEY` form is reserved for authenticated S3 API access and returns `NoSuchBucket` to anonymous callers.



Your cloud project ID appears on the project detail page in the [portal console](/docs/account/portal), under **Projects**. It is the same identifier that prefixes object storage URLs in the console.





A public-read policy allows anyone on the internet to download any object in the container without authentication. Use this only for assets you intend to be publicly accessible, such as static site files, documentation assets, or public software releases. For private data, keep the default private access.



For more policy patterns, see [Bucket policy examples](/reference/object-storage/bucket-policies#public-read).

## What you learned

In this tutorial, we:

- **Created S3 credentials** scoped to our Quake AI project
- **Configured the AWS CLI** to use Quake AI's S3-compatible endpoint with a named profile
- **Created a container** via both the console and the CLI
- **Uploaded, listed, and downloaded files** using `aws s3 cp` and `aws s3 ls`
- **Applied a bucket policy** to make objects publicly readable

Any tool or library that speaks S3 works with this setup. The `endpoint_url` configuration is the only Quake AI-specific change required.

## Next steps

- [Create a volume and add persistent storage to your server](/docs/quickstart/create-block-volume): the previous tutorial in this series, if you have not completed it
- [Mount S3 storage as a filesystem](/docs/object/how-to/mount-s3-storage): make object storage browseable on Linux with s3fs
- [Back up to Quake AI](/docs/object/migration/backup-to-quake-ai): 3-2-1 backups with rclone and restic
- [Object storage concepts](/docs/object/concepts/object-storage): deeper background on the storage model, consistency guarantees, and pricing

## Clean up

To remove the resources we created:

Delete all objects in the container:

```bash
aws s3 rm s3://tutorial-bucket --recursive --profile rumble
```

Delete the container:

```bash
aws s3 rb s3://tutorial-bucket --profile rumble
```

Optionally revoke the S3 credentials:

1. In the Quake AI console, go to **Storage** > **Object Storage** > **S3 Credentials**.
2. Select `tutorial-credentials` and select **Delete**.

After deletion, any tools configured with those credentials will no longer have access to your object storage.