My Homelab Evolution: Single Server to 3-Node Cluster

How my homelab grew from one server to a 3-node cluster — the upgrades, mistakes, and lessons along the way.

Getting startedhomelabnetworkingciscoserverrack

Introduction

My homelab started with a single Supermicro 1U server and a Cisco switch. It now runs a 3-node VMware vSphere cluster with vSAN, giving me high availability and redundancy for everything from Kubernetes clusters to network lab environments. This post covers how I expanded the setup, what hardware I added, and what I’d do differently next time.

My Homelab Beginnings: The Single Server

My initial foray into homelabbing began with a single server. As described in the previous article, this setup allowed me to explore and play with the basics of networking, servers, and other technologies. However, as my interests and needs grew, I realized that I needed a more powerful and flexible infrastructure.

That’s when I decided to venture into the world of VMware vSphere and vSAN, which opened up new possibilities for virtualization and storage.

The Evolution: A 3-Node Cluster

As my homelab journey progressed, I decided it was time to expand my setup to take full advantage of the capabilities offered by a 3-node cluster. This upgrade allowed me to reap the benefits of high availability, redundancy, improved performance, and scalability.

To create my 3-node cluster, I added two HPE DL380 Gen9 24xSFF servers to my existing Supermicro 1U server.

Server 1: Supermicro 1U - codename: “Kimchi”

  • CPUs: 2 x Intel Xeon E5-2697v3 running at 2.60 GHz with 14 Cores / 28 Threads
  • Memory: 256 GB of DDR4 running at 2333MHz
  • Storage: 4 x 500 GB SSD hard drives + 1 x 1 TB NVMe PCIe + 1 TB NVMe SATA.
  • Networking: 4x Gigabit Ethernet ports

Server 2: HPE DL380 Gen9 24xSFF - codename: “Tonkotsu”

  • CPUs: 2 x Intel Xeon E5-2697v4 running at 2.3 GHz with 18 Cores / 36 Threads
  • Memory: 256 GB of DDR4 running at 2400MHz
  • Storage: 12 x 900 GB SAS hard drives + 2x 500 GB PCIe NVMe SSD hard drives in RAID-0 (vSAN cache tier — see note below)
  • Networking: 4x Gigabit Ethernet ports + 2x SFP+ ports

Server 3: HPE DL380 Gen9 24xSFF - codename: “Okonomiyaki”

  • CPUs: 2 x Intel Xeon E5-2696v4 running at 2.2 GHz with 22 Cores / 44 Threads (OEM-only SKU, common in HPE/Dell pulls — don’t confuse with the E5-2698v4, which is 20C/40T)
  • Memory: 256 GB of DDR4 running at 2400MHz
  • Storage: 12 x 900 GB SAS hard drives + 1x 500 GB PCIe NVMe SSD
  • Networking: 4x Gigabit Ethernet ports + 2x SFP+ ports The server

A note on the RAID-0 NVMe pair on Tonkotsu: that’s deliberate, and it’s only safe because those drives are the vSAN cache tier, not capacity. vSAN replicates cache writes across hosts before they land on the capacity tier, so losing a cache device takes a single disk group offline but doesn’t lose data — the other nodes in the cluster still hold the replica. Striping two NVMes as RAID-0 doubles the write bandwidth into the cache, which is where the tier is usually bottlenecked. Do not copy this pattern for your capacity tier or for non-replicated VM data. RAID-0 on data means a single drive failure loses the whole stripe, and on a homelab that’s almost always a worse trade than a single drive’s throughput ceiling.

The rack cabinet

Adding 2 more servers to my homelab allowed me to realise that the rack cabinet I had was not big enough to accommodate the new hardware. So, I decided to upgrade to a 29U rack cabinet with 1000mm depth, which provided me with more space and better cable management and airflow.

The server

The networking

To support the increased networking demands of the 3-node cluster, I also added a Mikrotik CcS326-24G-2S+RM switch, which supports 10Gbps networking for higher data transfer speeds on the data VLAN. This switch was crucial in ensuring smooth communication between the servers and the efficient management of data traffic.

The networking

With this powerful 3-node cluster in place, I’ve been able to run multiple Kubernetes clusters in vSphere, utilize vSan for storage, and experiment with automating VM provisioning and management in vCenter. This expanded setup has provided me with countless opportunities to explore new technologies and use cases, fueling my passion for homelabbing even further.

The Benefits of a 3-Node VMware vSphere and vSAN Cluster

  1. Personal Learning and Growth: One of the most significant benefits of building a 3-node cluster in my homelab has been the opportunity to learn and grow as an IT professional. I’ve gained hands-on experience with VMware vSphere and vSAN, networking, and other technologies, which has proven invaluable in my career.

  2. High Availability and Redundancy: A 3-node cluster provides high availability and redundancy, ensuring that my homelab continues to function even if one of the nodes fails. With vSphere’s High Availability (HA) feature and vSAN’s data redundancy, I can rest assured that my virtual machines and data are safe.

  3. Improved Performance and Load Balancing: The 3-node cluster has allowed me to achieve better performance and load balancing in my homelab. VMware vSphere’s Distributed Resource Scheduler (DRS) ensures that resources are optimally allocated across the cluster, preventing bottlenecks and enhancing overall performance.

  4. Scalability: As my homelab needs have grown, the 3-node cluster has provided me with a scalable foundation. I can easily add more nodes, storage, or other resources as required, ensuring that my homelab remains adaptable and capable of handling my evolving requirements.

  5. Experimentation and Exploration: A 3-node cluster has enabled me to experiment with various technologies and use cases, such as running multiple k8s clusters in vSphere, automating VM provisioning, and testing VM migration within the cluster. This hands-on experience has been both enjoyable and informative, helping me expand my knowledge and skills.

The Future of My Homelab

The DL380 Gen9s are loud — 1U/2U enterprise fans do not care about your home office. My concrete next step is a Noctua fan swap on both DL380s using iLO-compatible PWM adaptors to stop the BMC from spinning the replacements to 100% on boot. Failing that, the cluster moves out of the office into a dedicated closet with acoustic foam and forced intake, and I run 10GbE fibre back to the desk. Both paths are cheap compared to the alternative (buying quieter hardware), and I’ll write up whichever one I actually ship.

Conclusion

And so, my homelab odyssey continues to be an exciting and fulfilling pursuit. With every new piece of equipment and every configuration tweak, my passion for homelabbing grows stronger. Who knows what the next chapter will bring? But one thing’s for sure – the journey has been a thrilling ride so far, and I can’t wait to see where it takes me next!

← Back to blog