update(2024-10-01): I’ve been getting busier with work recently, and I haven’t taken care of the devices at home for a long time. Now I only keep one server with decent hardware spec for occasional remote ssh development…
Revamping My HomeLab During the Shanghai Lockdown#
While under lockdown in Shanghai, I took the opportunity to reorganize my home servers — hoping it serves as a reference and sparks discussion for mutual improvement.
Hardware#
I’ve set up a 22U rack in my basement, populated with a collection of decommissioned servers sourced from Taobao, piecing together this somewhat precarious cluster.
Server Hardware Spec#
| Model | CPU | Cores/Threads | Memory | Storage | Height | Primary Role | Quantity |
|---|---|---|---|---|---|---|---|
| Dell R620 | E5-2620 v2 | 24/48 | 64GB | 2T 600GB*3 + 300GB | 1U | ESXi Host | 2 |
| HUAWEI RH1288 V2 | E5-2620 v3 | 24/48 | 64GB | 2T 600GB*3 + 300GB | 1U | ESXi Host | 1 |
| Dell R720xd | E5-2620 v2 | 24/48 | 128GB | 32T 4TB*8 | 2U | Storage Database Redis | 1 |
| HUAWEI 1288H v5 | Xeon Silver 4110 | 16/32 | 128GB | 4.8T 600GB*8 | 1U | ESXi Host | 1 |
Currently, all machines except the two R720xd units (used for storage) are virtualized using ESXi.
Network#
- Switch: Unmanaged Gigabit Switch
- External Access: Public IPv6 + ddns-go for dynamic DNS
Software Architecture#
Currently running a Kubernetes cluster comprised of 20 virtual machines (each 8 vCPUs / 16GB RAM).
Runtime#
- containerd
K8s Deployment#
- KubeKey: An open-source Kubernetes deployment tool from KubeSphere – simple and effective
Logging#
- Loggie: Open-source log collector from NetEase, deployed as DaemonSet
- ElasticSearch: Central log storage
- Grafana: Log visualization (lighter than Kibana)
Ingress#
- Traefik: Go-based, direct K8s integration, easier plugin development
Storage#
- SeaweedFS: Distributed file system with multi-protocol support (POSIX, S3, WebDAV, K8S CSI)
CI/CD#
| Component | Function |
|---|---|
| Tekton | Pipeline orchestration, artifact building, CD |
| Helm | Container deployments |
| Ansible | Host deployments |
| Buildkit | Image building |
| SeaweedFS | Artifact storage |
| Harbor | Image registry |
Monitoring#
- Prometheus: Metrics scraping only
- VictoriaMetrics: Long-term storage (1M writes/sec)
- Grafana: Metrics visualization
Networking#
- Calico-BGP: Pure routing solution, direct Pod IP access
Serverless#
- OpenFaaS: Simple architecture meets current function/script needs
- Note: Prefer KNative + Dapr but too complex for current setup