Server setup (manual) — mirrors the automated provisioner

The SaaS control plane provisions a tenant automatically (see scripts/stage_deploy_linode.py and provisioner/). This guide is for operators who self-host and want to perform the same five phases by hand, or who need to understand exactly what the automation does to a box.

Each phase below maps 1:1 to a module in provisioner/phases/. The automation is idempotent; these manual steps are too (safe to re-run).

Target OS: Debian 12 / Ubuntu 22.04+. Run as root (or via sudo).

---

Phase 1 — System (phase1_system.py)

Fully patch the freshly-booted image before installing anything.

export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get -y -o Dpkg::Options::=--force-confold dist-upgrade
apt-get -y autoremove --purge

Phase 2 — Packages (phase2_packages.py)

Install only what the server's role needs.

# base (all roles)
apt-get install -y ca-certificates curl gnupg ufw fail2ban apache2-utils jq git

# docker (app / registry / edge / all-in-one)
command -v docker >/dev/null || curl -fsSL https://get.docker.com | sh
systemctl enable --now docker

# wireguard (vpn / edge / all-in-one)
apt-get install -y wireguard wireguard-tools

# caddy (app / edge / all-in-one) — auto-HTTPS reverse proxy
#   add the Caddy apt repo, then: apt-get install -y caddy

Role	docker	caddy	wireguard
app	✅	✅	—
registry	✅	—	—
vpn	—	—	✅
edge / all-in-one	✅	✅	✅

Phase 3 — Hardening (phase3_hardening.py)

SSH, firewall, kernel. Validate sshd before reloading so a typo can't lock you out.

# Operator key + SSH drop-in (use a NON-standard port, e.g. 2222)
install -d -m 700 /root/.ssh
cat your_operator_key.pub >> /root/.ssh/authorized_keys && chmod 600 /root/.ssh/authorized_keys
cat >/etc/ssh/sshd_config.d/99-forge.conf <<'EOF'
Port 2222
PermitRootLogin prohibit-password
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
X11Forwarding no
MaxAuthTries 3
EOF
sshd -t && systemctl reload ssh   # ONLY reload if validation passes

# Firewall: default-deny, allow SSH FIRST, then role ports
ufw --force reset
ufw default deny incoming
ufw default allow outgoing
ufw allow 2222/tcp            # SSH (do this before enabling!)
ufw allow 80/tcp; ufw allow 443/tcp   # app/edge
ufw allow 51820/udp          # vpn/edge
ufw --force enable

# Kernel/network hardening
cat >/etc/sysctl.d/99-forge.conf <<'EOF'
net.ipv4.conf.all.rp_filter = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
kernel.kptr_restrict = 2
kernel.yama.ptrace_scope = 1
kernel.randomize_va_space = 2
EOF
sysctl --system

Phase 4 — Services (phase4_services.py)

WireGuard gateway:

install -d -m 700 /etc/wireguard
test -f /etc/wireguard/wg0.key || (umask 077 && wg genkey > /etc/wireguard/wg0.key)
# write /etc/wireguard/wg0.conf (Address 10.13.13.1/24, ListenPort 51820, PostUp sets key)
systemctl enable --now wg-quick@wg0

Private registry (distribution v3 + htpasswd):

install -d -m 755 /opt/forge/registry/auth /opt/forge/registry/data
htpasswd -Bbn forge "$REGISTRY_PASS" > /opt/forge/registry/auth/htpasswd
docker run -d --restart=always --name forge-registry \
  -p 127.0.0.1:5000:5000 \
  -v /opt/forge/registry/data:/var/lib/registry \
  -v /opt/forge/registry/auth:/auth \
  -e REGISTRY_AUTH=htpasswd -e 'REGISTRY_AUTH_HTPASSWD_REALM=CTFHive Registry' \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  distribution/distribution:3

App stack: render /opt/forge/app/.env (SITE_URL https://{slug}.{BASE_DOMAIN}, BASE_DOMAIN, SECRET_KEY, REGISTRY_HOST/USER/PASS, LAB_ENABLED=true) and docker compose --env-file .env up -d.

TLS (Caddy, provisioner/tls.py): write /etc/caddy/Caddyfile routing the apex → 127.0.0.1:8000, registry.{slug}.{base} → 127.0.0.1:5000, and the challenge wildcard *.{slug}.{base} (DNS-01 via the Cloudflare plugin), then caddy validate --config /etc/caddy/Caddyfile && systemctl reload caddy.

Phase 5 — Healthchecks (phase5_healthcheck.py)

docker info                                            # docker up
curl -fsS -o /dev/null -w '%{http_code}' https://{slug}.{base}/   # app over TLS
echo | openssl s_client -servername {slug}.{base} -connect {slug}.{base}:443 \
  | openssl x509 -noout -checkend 0                    # cert valid/not expired
curl -fsS -u forge:$REGISTRY_PASS https://registry.{slug}.{base}/v2/   # registry auth
wg show wg0                                             # vpn handshake

All green ⇒ the tenant is ready. Any red ⇒ the automation rolls back / refunds.

---

DNS

Point these at the server IP (the control plane does this via the Cloudflare API): - {slug}.{BASE_DOMAIN} → A → server IP - registry.{slug}.{BASE_DOMAIN} → A → server IP - *.{slug}.{BASE_DOMAIN} → A → server IP (challenge containers)