feat: add Garage as self-hosted S3-compatible storage, replacing MinIO
MinIO's community edition has been trimming features and its licensing direction has grown less predictable. Garage (AGPL, purpose-built for self-hosting) is a drop-in S3-compatible replacement — no changes needed to backend/app/minio_client.py, which talks to the S3 API generically. Deployed independently from the app stack (storage/docker-compose.yml), same separation MinIO had. Single-node config (replication_factor = 1) matching this project's learning/small-team scope. init-garage.sh handles the one-time layout assignment + bucket creation Garage requires that MinIO didn't (MinIO auto-creates buckets via API; Garage needs an explicit `garage bucket create` step).
This commit is contained in:
parent
b13810d68b
commit
0f0b7e9d32
42
storage/README.md
Normal file
42
storage/README.md
Normal file
@ -0,0 +1,42 @@
|
||||
# DocHub Storage (Garage)
|
||||
|
||||
S3-compatible object storage for DocHub, deployed independently from the
|
||||
app stack in `docker-compose.yml` at the repo root — same separation as
|
||||
was used for MinIO, just backed by [Garage](https://garagehq.deuxfleurs.fr/)
|
||||
instead. Garage was chosen because MinIO's community edition has been
|
||||
scaling back features and its licensing direction has grown less
|
||||
predictable; Garage is fully open source (AGPL) and purpose-built for
|
||||
self-hosting. Both speak the S3 API, so `backend/app/minio_client.py`
|
||||
needed no code changes — only connection settings differ.
|
||||
|
||||
Single-node setup, matching this project's learning/small-team scope
|
||||
(`replication_factor = 1` in `garage.toml` — no data redundancy across
|
||||
nodes, since there's only one node).
|
||||
|
||||
## First-time setup
|
||||
|
||||
```bash
|
||||
cd storage
|
||||
docker compose up -d
|
||||
./init-garage.sh
|
||||
docker compose exec garage /garage key info dochub-backend
|
||||
```
|
||||
|
||||
The last command prints an access key ID and secret key. Set those as
|
||||
`MINIO_ACCESS_KEY` / `MINIO_SECRET_KEY` for the DocHub backend (the env
|
||||
var names stay `MINIO_*` because `backend/app/config.py` was written
|
||||
against the S3 API, not against MinIO specifically).
|
||||
|
||||
## Connecting the backend
|
||||
|
||||
From the repo root, when running `docker compose up` there:
|
||||
|
||||
```bash
|
||||
MINIO_ENDPOINT=<this-server-ip>:3900 \
|
||||
MINIO_ACCESS_KEY=<from init-garage.sh> \
|
||||
MINIO_SECRET_KEY=<from init-garage.sh> \
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Garage's S3 API listens on port `3900` (vs MinIO's `9000`) — that's the
|
||||
only endpoint difference the rest of the stack needs to know about.
|
||||
13
storage/docker-compose.yml
Normal file
13
storage/docker-compose.yml
Normal file
@ -0,0 +1,13 @@
|
||||
services:
|
||||
garage:
|
||||
image: dxflrs/garage:v1.0.1
|
||||
network_mode: host
|
||||
volumes:
|
||||
- ./garage.toml:/etc/garage.toml:ro
|
||||
- garage-meta:/var/lib/garage/meta
|
||||
- garage-data:/var/lib/garage/data
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
garage-meta:
|
||||
garage-data:
|
||||
23
storage/garage.toml
Normal file
23
storage/garage.toml
Normal file
@ -0,0 +1,23 @@
|
||||
metadata_dir = "/var/lib/garage/meta"
|
||||
data_dir = "/var/lib/garage/data"
|
||||
db_engine = "sqlite"
|
||||
|
||||
replication_factor = 1
|
||||
|
||||
rpc_bind_addr = "[::]:3901"
|
||||
rpc_public_addr = "127.0.0.1:3901"
|
||||
rpc_secret = "62b86336239b3fd482f05e0caed739198a9601917cb1f80a7d8d429775be9e5c"
|
||||
|
||||
[s3_api]
|
||||
s3_region = "garage"
|
||||
api_bind_addr = "[::]:3900"
|
||||
root_domain = ".s3.garage.localhost"
|
||||
|
||||
[s3_web]
|
||||
bind_addr = "[::]:3902"
|
||||
root_domain = ".web.garage.localhost"
|
||||
index = "index.html"
|
||||
|
||||
[admin]
|
||||
api_bind_addr = "[::]:3903"
|
||||
admin_token = "fad952cc1c31f6e53b63b24d9dbbae75809906cfa61a933a91545e6130e32a31"
|
||||
35
storage/init-garage.sh
Executable file
35
storage/init-garage.sh
Executable file
@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-time Garage initialization: assigns this single node to the cluster
|
||||
# layout, then creates the DocHub bucket and an access key for the backend.
|
||||
#
|
||||
# Run this once after `docker compose up -d` in this directory.
|
||||
set -euo pipefail
|
||||
|
||||
GARAGE="docker compose exec garage /garage"
|
||||
|
||||
echo "Waiting for garage to be ready..."
|
||||
until $GARAGE status >/dev/null 2>&1; do
|
||||
sleep 1
|
||||
done
|
||||
|
||||
NODE_ID=$($GARAGE status | awk '/NODE ID/{found=1; next} found && NF {print $1; exit}')
|
||||
if [ -z "$NODE_ID" ]; then
|
||||
echo "Could not determine node ID from 'garage status' output." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Assigning layout to node $NODE_ID (single node, capacity 10G)..."
|
||||
$GARAGE layout assign -z dochub -c 10G "$NODE_ID"
|
||||
$GARAGE layout apply --version 1
|
||||
|
||||
echo "Creating bucket 'dochub'..."
|
||||
$GARAGE bucket create dochub
|
||||
|
||||
echo "Creating access key 'dochub-backend'..."
|
||||
$GARAGE key create dochub-backend
|
||||
$GARAGE bucket allow --read --write --owner dochub --key dochub-backend
|
||||
|
||||
echo
|
||||
echo "Done. Run 'docker compose exec garage /garage key info dochub-backend'"
|
||||
echo "to retrieve the access key ID and secret key, then set them as"
|
||||
echo "MINIO_ACCESS_KEY / MINIO_SECRET_KEY for the DocHub backend."
|
||||
Loading…
Reference in New Issue
Block a user