Pursuant to my master’s in educational sociology I’ve been coding a fair bit of R recently. I’ve quickly run into resource bottlenecks though — the intersection of a fairly large dataset and a mere X1C6 turns out to… not be great. So… what better excuse to buy retired enterprise hardware? It’s for my degree!
| Base | HP DL380e gen 8 | | Chassis | 12 LFF bays, 1x750W PSU | | CPU | Dual Xeon E5-2450L | | RAM | 96 GB (4x8 + 4x16) | | HBA(ish) | P420, B120i | | Data | 5x 6TB HGST NL-SAS | | Cache | 500GB WD Red NAS SSD |
Went whole hog on the RAM, as that’s where my programming efforts are stymied. I’ve got room to grow - HPE reports I can increase up to 196 GB; while unRAID reports 384 GB max capacity.
The disks were used, and a steal at ~15 USD per gigabyte. All report A-OK.
To use the B120i for cache SSDs, I needed to find extra power somewhere. My 380e came without the rear drive cage option; but did come with the rear drive cage cable. Measurements yielded this pinout:
|-| +---------+ 1: 8v ground 4: 12/8v (yellow) | 1 2 3 | 2: empty 5: 12v/5v ground | 4 5 6 | 3: 1v ground 6: 5v/1v (red) +---------+
Also, the cable fits a female 6-pin PCI Express connector perfectly. So, a massacre of a 6-to-8 pin adapter as well as a molex extender later, we have power!
The yellow and red leads on the molex adapter go to pins 4 and 6, and the two black leads meet and go to pin 5.
HP servers are notorious for having an… aggressive approach to fan profiles. This means they can be hard to share a small home with. But never fear! Nerds to the rescue — turns out, there’s a hack for that.
Do note: THIS IS A HACK. IT MAY NOT WORK. IT MAY BRICK YOUR SERVER.
I hate noise more than I have sense, and I was fine in the end. YMMV.
The P420 is a hot chip, and largely responsible for the baseline fan levels. iLO does all it can to keep it at or below 85 degrees C; which means running the fans hard. I ziptied a Noctua 40mm fan (the A4-N20FLX) to the heatsink, with great results. Powered from the rear drive bay power cable (another adapter in the chain), this keeps the RAID card at a comfortable 65-67 degrees C, and iLO can stop worrying.
Except that, in their infinite wisdom, any PCI card detected means fans 3, 4, and 5 will run at a minimum of 35-40 %. More steps must be taken!
wget https://downloads.hpe.com/pub/softlib2/software1/sc-linux-fw-ilo/p192122427/v112485/CP027911.scexe CP027911.scexe --unpack=ilo
Install the 2.50 firmware however you like. I used the web interface.
git clone email@example.com:airbus-seclab/ilo4_toolbox.git yay -S keystone hexdump cd ilo4_toolbox/scripts/iLO4/eploits wget https://uc2e993615a24a6915b40d722b8c.dl.dropboxusercontent.com/cd/0/get/A1CIhVjQEhr9ukukz8Qw_dHKizKB0RGgnFjfrp6z1rUtvBFclCvn4t6LErPcGVl0At3NQKzgezKAb8eV9-W5eg1P_0lRnZ47R-d5u0r4VvTpbmRBuItsv5RL2b2aKbyY7_M/file?_download_id=16760008867236312560412850928972566356913390752513665509633372074&_notify_domain=www.dropbox.com&dl=1 python2 exploit_write_flash.py 250 ilo4_healthcommands.bin
I had the exploit stall the first time I ran it. Tried again, the planets were aligned.
This will reduce the base speed of the fans to a more bearable level cross the board; while allowing the firmware to respond as designed to high temperatures.
It really quiets down around sensor 32; which, incidentally, is the P420.
Adding disks can cause the fans to spin up; which requires a re-run of the command.
Change user and iLO hostname to suit your environment.
If private keys are not set up, add
sshpass password after
for I in `seq 1 65`; do ssh -o KexAlgorithms=+diffie-hellman-group1-sha1 martin@nas-ilo "fan pid $I lo 125"; done
I decided to run this command periodically, in case the box gets confused and ramps up the fan profiles again.