Homebrew NAS, v2.0

Almost 3 years ago, I built myself a homebrew 10TB NAS for what I considered to be quite a reasonable price for such a compact, flexible system.

Having felt some envy towards to Pim van Pelt and the NAS setup he built himself in December, I realised that my setup was rapidly approaching it’s third anniversary. Even though it’s not full, it is showing it’s age, as well as having a few quirks (like no working serial port to console in on), not to mention a flakey BIOS which has occasional issues booting, etc. It had also become my workhorse box of choice in the house, even though it was a lowly 1.5GHz Intel Atom CPU. But my setup had an eSATA connector, so I started to think about extending it with a disk enclosure.

Then, at the beginning of February, I tried to flash the BIOS to get off the no longer supported OEM branded BIOS (which I thought was just a custom splash screen), only to run into BIOS checksum errors, followed by new and novel booting problems. Ultimately, the system was no longer booting and I was dead in the water.

Some failed attempts at debugging the problem turned into researching components for a replacement system, which arrived a week later. After a few false starts due to attempts of being thrifty, reusing disks and USB keys that I had lying around as the system disk, I now have an awesome new Intel H67 MicroATX based setup, with a 3.6GHz quad core Intel IvyBridge CPU, 16GB 32GB of RAM (and space for another 16GB when I get around to expanding it even more) and a Sharkoon 5 bay eSATA enclosure. While one of the requirements of v1.0 was a neat form factor, physical space is no longer an issues so the main requirement is modular design, allowing for expansion (aka eSATA expansion modules) and a PCI or PCIe slot to provide a serial COM port for my homebrew console server setup.

A few weeks on and things are almost back to normal, in no small part due to ZFS’ ability to be imported from one system to the next. All the usual services I was running before we’re fairly easy to setup gain. The one thorn in my side is the Squeezebox server, whose port in FreeBSD is stale and unloved, resulting in dependency hell on a recent system. Despite being EOLed by Logitech, I’ve started playing around, trying to create a port the newest release of the recently renamed Logitech Media Server since it supports SQLite instead of MySQL (MySQL schema problems being the problems I ran into) and the community developed replacement is still vapourware.

I still need to get around to running benchmarks on the eSATA setup (which I’ve deferred until I install a 64GB SSD I’ve picked up as the long term system disk) and tuning ZFS to the system to the system settings. I’m still mildly in awe of the CPU bump. The v1.0 system used to take 4 hours to do a fairly predictable ffmpeg based transcoding job, which was long and intensive enough that I’d schedule them to run overnight. The v2.0 system with it’s 8 effective (HyperThreading enabled) cores, the same workload take a mere 14 minutes! And I still can’t find a way to get the system to draw more than 49W.

I <3 living in the future.

DIY Low Power Serial Console Servers, on a Shoestring Budget

I try to keep my home computer setup minimal, low power and out of sight. There is no dedicated computer space, everything is either portable, resides in the media centre and/or runs headless by default. It’s is great for keeping the house neat, but a pain when something goes wrong and I need to debug it in place.

For years, I had kept a USB serial cable ready to go beside the Soekris router, in case I needed to get in the console. But hauling the floating monitor up to the attic whenever the NAS had an issue became an inconvenience. So I came up with an idea from my professional life, console servers.

But there were two problems, what cheap, low power device to use, and how to get a serial port on the NAS? Turns out the second question was easily addressed when I noticed the COM connector in the main board in the NAS. I just needed to add a DB9 connector, or so I thought. Despite trying a variety of settings and speeds, I could never quite get the output at the correct speed and settings, which is a solvable problem, with enough time. I was prepared to live without serial support into the BIOS.

For the low power devices themselves, I considered SheevaPlugs, GuruPlugs etc but always held off since I didn’t want to have to spend over €100 (plus shipping) for each device. So I kept hauling the monitor around, up until last year when the Raspberry Pi was released. The 1 unit per order slowed me down for a few months, but I managed to amass 3 of them (2 console servers, and 1 toy) by the end of 2012. I got a handful of 16GB, recognizably branded SD cards from Amazon and I was making progress. I also needed cases for them, since the Soekris is in a cat accessible places, and I’m sure a cat would be curious about a circuit board at some point. So I got 3x of the elegantly designed AdaFruit cases from a UK reseller^Wcutter, which did the job very nicely indeed.

The result?


3 Raspberry Pis, in cases

I now have a Raspbian (eventually I’ll swap it out for FreeBSD after some testing) console server on my Soekris, which has already been invaluable flashing it’s BIOS and getting in via the console via SSH. I have another Raspberry Pi in the attic, ready for action once I solve the serial port issues on the NAS (which is in progress, but worthy of it’s own post). I’m even toying with a 3G modem on the Raspberry Pi on the Soekris as a complete out of band solution to fixing my home internet when I’m not there. Once I get that working, I’ll declare my home network over engineered. 🙂

Installing FreeBSD 9.0 amd64 on a Soekris net6501 (the long way)…

I figured I should document this here, since it really does belong somewhere on the internet.

I’ve been using Soekris systems as my home route/firewall/misc Unix box for about 5 years now and I’m quite a fan. I’ve slowly been coming to the realisation that my trusty old net5501-70 is going to become more and more of a bottleneck for me, I decided to jump to the new shiny, a net6501-70

It seems there is a known issue with FreeBSD 9.0 amd64 and the net6501, the GENERIC kernel fails to boot since an optional driver is required. So the options seem to be either install FreeBSD 9.0 in 32 bit, or do something a little creative to install 64 bit mode. In the end, I did both.

Originally, I tried in vain to replace the GENERIC kernel on the memstick installer with a custom compiled kernel, which didn’t work out so well. Next I tried to build a custom memstick image on one of my other FreeBSD amd64 systems, using the make scripts provided within /usr/src/release/. It seemed to be going well, but it was taking a number of days, which seemed way too long to build a 700MB iso image, so I gave up on that. I came very close to simply installing a freshly built world and custom kernel, but ran into issues I now know are FreeBSD boot loader stage issues (they’re just very poorly documented).

In the end, I installed FreeBSD 9.0 i386 on the net6501-70, and then using i366->amd64 migration steps, I did a side step over to amd64 (ignoring the parts about broken ports since I hadn’t installed anything from ports). I now have a freshly installed amd64 system I can continue to setup, it just took me a number of weeks to get here.

At least it wasn’t entirely wasted, I’ve learnt a lot more about FreeBSD boot stages and symptoms debugging problems. I’ve also learnt to use gpart, which is so much more intuitive to use than bsdlabel

Posted from Dublin, County Dublin, Ireland.

Homebrew NAS Box

Last June, I built myself a homebrew NAS box. Every so often people ask me about it, so I might as well document what I did once and for all.

For a few years, I was storing media and backing up my photos onto a single disk in a USB caddy. There are a few problems with this approach, if like me you’re a sysadmin (no redundancy, no SMART monitoring, limited capacity, etc) so I started thinking about getting a NAS.

I looked at the Thecus and QNAP models and found the higher end ones had some of the features I was looking for. But I’m a FreeBSD user and I wanted to start trying ZFS, so firmware hackability was one of the most important features I was looking for. None of the models I was looking at were very hackable. So now, I was in homebrew territory, which concerned me since I didn’t want a full sized system with a stack of disks in it.

Then on the recommendation from Colm Buckley, I ended up looking into Tranquil PC, who make the BBS2, a dual core 1.66GHz Intel Atom CPU, up to 4GB RAM system in the form factor of a NAS. I ended up getting one with 4GB RAM and filling it with 5x Western Digital 2TB disks.

Since I wanted to use the drive bays purely for storage, I was originally running the system off a 16GB CF card connected via USB, while I sorted out an IDE Disk On Module. But after opening my BBS2 up, I couldn’t find an IDE slot for the Disk On Module to go into, so I resorted back to the USB storage solution, but using a new 16GB SanDisk Cruzer USB key as the system disk.

The result? I have a 10TB NAS which takes ~40W of electricity to run (I should really confirm that with my power meter), runs FreeBSD 8.x, stores everything in ZFS and can interface with pretty much anything I want, from my Mac Mini powered Multimedia Setup, to my new Squeezebox Touch. It speaks NFS, SMB and AFP, so pretty much any client can talk to it, and it didn’t take anything more than a standard FreeBSD install to set it up. In the (somewhat) distant future, when I manage to use most of the free space I may end up getting the SQA-EX to add another 5 disks to the system.