feat: add Garage as self-hosted S3-compatible storage, replacing MinIO
MinIO's community edition has been trimming features and its licensing direction has grown less predictable. Garage (AGPL, purpose-built for self-hosting) is a drop-in S3-compatible replacement — no changes needed to backend/app/minio_client.py, which talks to the S3 API generically. Deployed independently from the app stack (storage/docker-compose.yml), same separation MinIO had. Single-node config (replication_factor = 1) matching this project's learning/small-team scope. init-garage.sh handles the one-time layout assignment + bucket creation Garage requires that MinIO didn't (MinIO auto-creates buckets via API; Garage needs an explicit `garage bucket create` step).
This commit is contained in:
parent
b13810d68b
commit
0f0b7e9d32
42
storage/README.md
Normal file
42
storage/README.md
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
# DocHub Storage (Garage)
|
||||||
|
|
||||||
|
S3-compatible object storage for DocHub, deployed independently from the
|
||||||
|
app stack in `docker-compose.yml` at the repo root — same separation as
|
||||||
|
was used for MinIO, just backed by [Garage](https://garagehq.deuxfleurs.fr/)
|
||||||
|
instead. Garage was chosen because MinIO's community edition has been
|
||||||
|
scaling back features and its licensing direction has grown less
|
||||||
|
predictable; Garage is fully open source (AGPL) and purpose-built for
|
||||||
|
self-hosting. Both speak the S3 API, so `backend/app/minio_client.py`
|
||||||
|
needed no code changes — only connection settings differ.
|
||||||
|
|
||||||
|
Single-node setup, matching this project's learning/small-team scope
|
||||||
|
(`replication_factor = 1` in `garage.toml` — no data redundancy across
|
||||||
|
nodes, since there's only one node).
|
||||||
|
|
||||||
|
## First-time setup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd storage
|
||||||
|
docker compose up -d
|
||||||
|
./init-garage.sh
|
||||||
|
docker compose exec garage /garage key info dochub-backend
|
||||||
|
```
|
||||||
|
|
||||||
|
The last command prints an access key ID and secret key. Set those as
|
||||||
|
`MINIO_ACCESS_KEY` / `MINIO_SECRET_KEY` for the DocHub backend (the env
|
||||||
|
var names stay `MINIO_*` because `backend/app/config.py` was written
|
||||||
|
against the S3 API, not against MinIO specifically).
|
||||||
|
|
||||||
|
## Connecting the backend
|
||||||
|
|
||||||
|
From the repo root, when running `docker compose up` there:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
MINIO_ENDPOINT=<this-server-ip>:3900 \
|
||||||
|
MINIO_ACCESS_KEY=<from init-garage.sh> \
|
||||||
|
MINIO_SECRET_KEY=<from init-garage.sh> \
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Garage's S3 API listens on port `3900` (vs MinIO's `9000`) — that's the
|
||||||
|
only endpoint difference the rest of the stack needs to know about.
|
||||||
13
storage/docker-compose.yml
Normal file
13
storage/docker-compose.yml
Normal file
@ -0,0 +1,13 @@
|
|||||||
|
services:
|
||||||
|
garage:
|
||||||
|
image: dxflrs/garage:v1.0.1
|
||||||
|
network_mode: host
|
||||||
|
volumes:
|
||||||
|
- ./garage.toml:/etc/garage.toml:ro
|
||||||
|
- garage-meta:/var/lib/garage/meta
|
||||||
|
- garage-data:/var/lib/garage/data
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
garage-meta:
|
||||||
|
garage-data:
|
||||||
23
storage/garage.toml
Normal file
23
storage/garage.toml
Normal file
@ -0,0 +1,23 @@
|
|||||||
|
metadata_dir = "/var/lib/garage/meta"
|
||||||
|
data_dir = "/var/lib/garage/data"
|
||||||
|
db_engine = "sqlite"
|
||||||
|
|
||||||
|
replication_factor = 1
|
||||||
|
|
||||||
|
rpc_bind_addr = "[::]:3901"
|
||||||
|
rpc_public_addr = "127.0.0.1:3901"
|
||||||
|
rpc_secret = "62b86336239b3fd482f05e0caed739198a9601917cb1f80a7d8d429775be9e5c"
|
||||||
|
|
||||||
|
[s3_api]
|
||||||
|
s3_region = "garage"
|
||||||
|
api_bind_addr = "[::]:3900"
|
||||||
|
root_domain = ".s3.garage.localhost"
|
||||||
|
|
||||||
|
[s3_web]
|
||||||
|
bind_addr = "[::]:3902"
|
||||||
|
root_domain = ".web.garage.localhost"
|
||||||
|
index = "index.html"
|
||||||
|
|
||||||
|
[admin]
|
||||||
|
api_bind_addr = "[::]:3903"
|
||||||
|
admin_token = "fad952cc1c31f6e53b63b24d9dbbae75809906cfa61a933a91545e6130e32a31"
|
||||||
35
storage/init-garage.sh
Executable file
35
storage/init-garage.sh
Executable file
@ -0,0 +1,35 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# One-time Garage initialization: assigns this single node to the cluster
|
||||||
|
# layout, then creates the DocHub bucket and an access key for the backend.
|
||||||
|
#
|
||||||
|
# Run this once after `docker compose up -d` in this directory.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
GARAGE="docker compose exec garage /garage"
|
||||||
|
|
||||||
|
echo "Waiting for garage to be ready..."
|
||||||
|
until $GARAGE status >/dev/null 2>&1; do
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
NODE_ID=$($GARAGE status | awk '/NODE ID/{found=1; next} found && NF {print $1; exit}')
|
||||||
|
if [ -z "$NODE_ID" ]; then
|
||||||
|
echo "Could not determine node ID from 'garage status' output." >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Assigning layout to node $NODE_ID (single node, capacity 10G)..."
|
||||||
|
$GARAGE layout assign -z dochub -c 10G "$NODE_ID"
|
||||||
|
$GARAGE layout apply --version 1
|
||||||
|
|
||||||
|
echo "Creating bucket 'dochub'..."
|
||||||
|
$GARAGE bucket create dochub
|
||||||
|
|
||||||
|
echo "Creating access key 'dochub-backend'..."
|
||||||
|
$GARAGE key create dochub-backend
|
||||||
|
$GARAGE bucket allow --read --write --owner dochub --key dochub-backend
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "Done. Run 'docker compose exec garage /garage key info dochub-backend'"
|
||||||
|
echo "to retrieve the access key ID and secret key, then set them as"
|
||||||
|
echo "MINIO_ACCESS_KEY / MINIO_SECRET_KEY for the DocHub backend."
|
||||||
Loading…
Reference in New Issue
Block a user