HOME LAB RENEWAL - PT 2
In our last encounter, I had teased you with my need for more. Specifically, more storage since I had ditched all those spinning disks. I've been a YouTube Premium subscriber for like ten years and had somehow NEVER been introduced to "TechTube." Boy did I go all in:
- Linus Tech Tips got me going, specifically these glorious episodes: link and link and a few others.
- That channel brought me to all of his other channels like Techquickie, TechLinked, MacAddress, ShortCircuit, etc.
- Then I decided to get real - so I started looking around at understanding more about TrueNas and Proxmox. That lead me to Jeff Geerling and Craft Computing.
Those led me down the rabbit hole to lots of Reddit research, particularly in /r/Homelab. I had decided WHAT I wanted to do, just not HOW. Here's the what:
Proxmox seems better suited for running Linux VMs. Sure HyperV does them fine, but they felt snappier in my tiny test lab I built with a left over Surface Book. It also had native "just run it" container support and a stupidly awesome repo of container templates called TurnKey - wow are those fancy! Sure HyperV could run containers, but not very well natively. Also, and this makes me feel bad saying, Windows Admin Center is slow as hell and the console makes me sad. Proxmox's entirely webby interface is comically responsive and the console is a chef's kiss. Okay, so, I'm gonna switch from HyperV to Proxmox - how will I do that? That's for later. Next I needed some storage.
I love a good Ebay deal in the morning. Retail therapy got me and I ended up with a delightful little toy:
That thar is Supermicro 2U 24 Bay SFF X10DRU-i with 4x960GB NVMe drives, 2x Intel E5-2683v3 28 Core, and 128GB of RAM. Last gen but the price!!!!! So, I picked one of those up. The store was reputable and they made things right when I arrived with a lot less than what I had ordered. Hat tip to them! To finish that dandy up, I needed to fill in 20 of those 2.5" bays - enter the IBM 900 GB SAS drive (I forget the model number). Not the hugest capacity, but based on my new learnins from TrueNas, I knew I could get about 17TB out of it. More on storage setup in a bit. One bummer about this server is the fans, just like the Dell, were "airplane revved to max" loud, so, out with them and in with some cheap o USB fans I can just hover over the open case for now. Learnin me about static air pressure I was!
So, that's gonna be my storage box, and my Dell from Part 1 will be my compute. I needed to also finagle some of my M.2's out of the Dell since I was going to do all my storage over on this new Supermicro. Turns out, I ran into a few problems. First, the Dell supports something called 'bifurcation' - that lets me get full benefit of the 4 M.2's in that x16 PCIe card. The Supermicro, well, it only has x8 PCIe and also doesn't do bifurcation. So, I picked up a few more x4 M.2 adapters and just slung them into the open slots - I had plenty and a specific purpose for them. These work great.
For this part, we'll round out our discussion of how the heck I'm gonna hook these bad boys together. I played around with upgrading my entire Ubiquiti network to 10GBE, but that was horse poop expensive. Both servers had dual 10GBE ports, so - I just stuck a crossover cable on both ports and attached and set up a two node network between the two (no switch) and bonded them together for a saucy 20GBE compute to storage network.
Before taking the Dell server down to build up Proxmox, and before installing TrueNas onto the new Supermicro, I set up a teensy lab to see what kind of throughput I could actually get. Neither of these servers are amazingly powerful, but I suspected with an M.2 on one side and an NVME on the other, I could get some decent push. Turns out - around 10 GIGABYTES per second is where I maxed out...not gigabits, GIGABYTES. That's only half the rated speed but holy crap I'll take it!
Next time, we'll discuss the teardown, build, and all the other goodies I discovered along the way!