Operating a Multi-Node Linux Cluster

I operate three Linux nodes connected through a private Tailscale network: a primary GPU server, a Beelink mini-PC, and a Raspberry Pi 5. Together they run public applications, replicated data, local AI inference, monitoring, logging, and backups.

The point is not to imitate a hyperscale cloud. It is to understand deployment constraints directly and build a repeatable system that I can operate myself.

Service isolation

Applications run in Docker, and each public service shares a network namespace with its own Tailscale sidecar. The application containers do not publish host ports. Cloudflare Tunnels and Nginx route approved traffic over the private network.

This pattern gives each service its own private identity and keeps application routing separate from the host network.

Repeatable operations

Ansible playbooks handle deployments, health checks, MongoDB replica checks, backups, and routine maintenance. Keeping the application pattern consistent across nodes makes improvements reusable instead of one-off shell history.

Observability

Prometheus, Grafana, Loki, Node Exporter, cAdvisor, MongoDB Exporter, and GPU metrics provide one view across the cluster. I use those measurements to find actual bottlenecks—for example, reducing cAdvisor CPU use by 93–95% after changing its collection behavior.

Local inference

The GPU node runs Qwen3.5-35B-A3B through llama.cpp at approximately 101 generated tokens per second on a down-tuned RTX 3060. In that setup, PCIe model movement is the primary measured constraint.

Read the full Homelab case study →

Node Usage

On my Raspberry Pi, I run analytics for Docker containers using Prometheus, backups, and an n8n server, among other services. The external SSD provides lower-end read write speed, so this node is used for backups, testing among other services. The other two servers are used for deployed services with docker.

Tech Stack

  • Networking: Tailscale wire-mesh private network
  • Operating System: Ubuntu Server
  • Containers & Orchestration: Docker
  • Reverse Proxy / Load Balancing: Nginx
  • Secure Remote Access: Cloudflare Tunnel
  • Database: MongoDB (WiredTiger)
  • Backend: FastAPI
  • Frontend: Next.js or Vanilla React
  • Mobile: React Native with Expo
  • Scripting & Automation: Python, Shell scripts

Building a multi-node cluster has allowed me made me a better programmer. From understanding the multiple layers of security, networking, deployment and more.