Homelab

Why

To learn infrastructure by doing it, and because a QA who does not know where the thing under test runs misses half the bugs. The lab also serves as a sandbox: a deployment regression shows up here before it shows up in production.

The architecture

Proxmox VE 9.2, a single host, and one LXC container per service rather than one big VM: independent updates, isolated failures, selective backups. It runs AdGuard (DNS), Vaultwarden, Immich, Jellyfin, Meilisearch, Miniflux, the *arr media stack, OpenCloud, and a few home-made tools.

Caddy handles the reverse proxy and certificates: one block per service, automatic TLS, HSTS. Every domain points to the same public IP, and Caddy decides where the request goes.

This site runs in container 217: Node and Express on port 3000, Caddy in front, PM2 to restart the process. Proxmox Backup Server for backups, rclone for the off-site copy.

What broke

A Proxmox upgrade changed container networking behaviour: no outbound internet, while everything else answered. Diagnosis took an hour, less because of the problem than because I was looking in the wrong place — inside the containers, while the fault was on the host.

Second classic: a “successful” backup that restored nothing. The job ran, the file existed, and the restore failed on a missing dependency. Since then a backup is only valid once a restore has been tested.

What I keep from it

Three habits that came from this and that I use daily in QA: isolate before fixing, read the logs of whoever reports the error — not just whoever produces it — and never trust a green light before making it lie once.