Deploy Airbyte with the airbyte template

Stand up Airbyte, an open-source extract-and-load platform, on a single Quake AI instance using the validated OpenTofu template airbyte. You apply the template, reach the web UI over the floating IP, configure a source and a destination, and run a replication job.

Airbyte is the extract-and-load layer in a self-operated data stack. You run it yourself; this is a self-hosted tool you operate, not a hosted ELT cloud.

Prerequisites#

You need:

• OpenTofu 1.6.0 or later (or Terraform 1.6.0 or later) installed locally.
• Your OpenStack credentials sourced into the shell (source openrc.sh). See the OpenStack CLI guide.
• An SSH keypair that already exists in your project. Record its name for the key_name variable.
• A copy of the airbyte template directory from the template reference page.
• Your workstation's public IP address, so you can open the web UI port for first-boot setup. Find it with curl -sS https://api.ipify.org.

A destination is optional for first boot. You add a PostgreSQL warehouse or Object Storage bucket in step 3.

Step 1: Set the variables and apply the template#

The web UI listens on port 8000 over plain HTTP. The template's security group restricts port 8000 to ui_allowed_cidr, which defaults to the private network only, so the raw UI stays off the public internet. To reach the UI from your workstation for first-boot setup, set ui_allowed_cidr to your own address.

Copy the template's example variables file and open it:

cp terraform.tfvars.example terraform.tfvars

Set key_name to the SSH keypair already in your project, and ui_allowed_cidr to your workstation's public IP with a /32 suffix:

key_name         = "YOUR_KEY_NAME"
ui_allowed_cidr  = "YOUR_IP/32"

Initialize the working directory, preview the plan, and apply:

tofu init
tofu plan
tofu apply

OpenTofu provisions a private network, a router, a security group, a block volume mounted at /var/lib/docker, an instance, and a floating IP. On first boot, cloud-init mounts the data volume, installs Docker Engine, and starts the Airbyte stack from docker compose on port 8000.

When the apply finishes, read the outputs:

tofu output

Record floating_ip and ui_url.

Step 2: Open the web UI#

Airbyte does not ship default login credentials in this template. cloud-init downloads the pinned platform release and starts the stack in detached mode; the first boot takes several minutes while containers pull images and run migrations.

Open ui_url (for example http://YOUR_FLOATING_IP:8000) in your browser. If the page does not load yet, wait and retry; you can watch containers start over SSH:

ssh ubuntu@YOUR_FLOATING_IP "sudo docker ps --filter name=airbyte"

When the Airbyte home screen appears, you are ready to add connections.

Step 3: Configure a source and destination#

You connect a sample source and a warehouse destination, then run a sync.

1. In the Airbyte UI, select Sources > + New source. Choose Sample data (Faker) for a quick test, or pick a connector that matches your workload.
2. Select Destinations > + New destination. For a PostgreSQL warehouse, choose Postgres and point it at a self-managed PostgreSQL instance on the same private network (host, port, database, user, password). For Object Storage, choose S3 and set the Quake AI S3-compatible endpoint and access keys from the Console.
3. Select Connections > + New connection, link the source and destination, choose a sync frequency, and save.
4. Select Sync now and wait for the job to complete. Confirm rows landed in the destination (query the Postgres table or list objects in the bucket).

What you built#

• Applied the airbyte template to provision a network, security group, data volume, instance, and floating IP, and let cloud-init install Docker and start the Airbyte stack
• Opened the web UI and confirmed the multi-container stack is running
• Configured a source, a destination, and a connection, then ran a replication sync into PostgreSQL or Object Storage
• Confirmed data landed in the destination you chose

Scope of this deployment#

This template runs a single-VM Airbyte host, not a hosted ELT cloud. The instance is CPU-only and runs in one region. You operate the instance, Docker, Airbyte, and the data volume yourself: back them up, patch them, and watch resource use as sync volume grows. Snapshot the volume before you resize or rebuild the host. For heavy replication workloads, size the instance up and point destinations at a dedicated warehouse.

Next steps#

• Airbyte template: the template reference, parameters, and resource map
• self-managed PostgreSQL template: a warehouse destination to replicate into
• S3 storage ACL template: Object Storage for raw landing zones
• data pipelines and analytics brief: how extract, transform, and load layers compose
• How to store application secrets and inject them at runtime: keep connector credentials out of plain environment variables

Clean up#

When you no longer need the deployment, destroy everything the template created:

tofu destroy

Because Airbyte metadata, connector workspaces, and sync logs all live on the instance and its attached volume, tofu destroy removes them along with the infrastructure. Export any connection configurations you want to keep before you destroy.