Category Archives: NAS

NetBSD NAS software setup

Once boot process was solved, I needed to setup ZFS for my hard drives and several services like NFS and Samba, setup network bonding using the new dedicated lagg(4) interface with LACP protocol, which in turn would complete the major NAS functionality.

ZFS setup

Initial ZFS setup was pretty easy. Luckily, configuring ZFS modules (solaris, zfs) in modules.conf(5) file was enough to make it work, even if configuration file was located in chrooted root partition. I mainly followed the Creating ZFS pools on NetBSD article from the UnitedBSD , or more exactly the “Create a ZFS pool” section by creating a ZFS pool and datasets with the mount points. After that, adding “zfs=YES” line in rc.conf(5) file does the final trick in order to mount ZFS partitions in their intended locations. I also enabled de-duplication and gzip compression features. The main mistake followed after adding a second hard drive to the zpool, which was intended to be a mirrored drive, for which attach command should have been used. NetBSD’s zpool command doesn’t allow removal of the drive without proper redundancy and doesn’t support forced removal, thus I need to recreate the pool, which was a bit of painful setback. According to recommendations I scheduled once a month zpool scrub operation, but still need to setup scheduled snapshots in the nearest future.

Link aggregation

As mentioned previously QNAP TVS-675 system contains two expansion card slots using PCIe Gen3 x4 interface. Thanks to the cut at the back of the port, cards with x8 physical slot can also fit into the port (longer cards may not fit though without modifications, metal frame can block the insertion). The challenge in doing so that it was a very short distance between PSU and a slot bracket, which made it very difficult to unscrew it without removing the PSU first. Once the network card was physically in place, the major problem occurred while setting up the lagg(4) interface using link aggregation control protocol (LACP). It appeared that mcx(4) driver wasn’t setting up full-duplex data transmission required by the protocol. Fortunately, the fix was relatively easy and made it into NetBSD 10 release by enforcing full-duplex in the driver. Additionally, missing lagg parameters were documented in ifconfig(8) man page. After the fix, network bonding setup was pretty straightforward by creating /etc/ifconfig.lagg0 file:

create
laggproto lacp laggport mcx0 laggport mcx1
up

I also created ifconfig.mcx0 and ifconfig.mcx1 with only up command and added net_interfaces=”mcx0,mcx1,lagg0″ line to /etc/rc.conf to control initialization order and ensure that mcx interfaces are up before lagg0 interface is created and initialized. For dhcpcd I added dhcpcd_flags=”-qM lagg0″ flags. After reboot my switch was happy to accept link aggregation with my NAS system.

Samba setup

Next step was to install and enable samba(8). The small issue was the missing xbase set required by the samba installation, since I was not aware of this requirement. I simply downloaded the required set and extracted it manually. Other than that I could mainly reuse configuration file from Linux system by only adjusting required paths and placing it in /usr/pkg/etc/samba/smb.conf location. Of course I needed to create a samba user (smbpasswd). Also worth to note that smbd, nmbd and winbindd services should be enabled instead of samba service, contrary to typical Linux experience.

NFS setup

NFS configuration was not compatible with the Linux one on the other hand and I needed to rewrite it from scratch. NetBSD “how to set up nfs and nis” and UnitedBSD NFS on NetBSD: server and client side wiki articles were a great guidance. To enable NFS properly I needed to add these line to /etc/rc.conf:

# NFS server
rpcbind=YES
nfs_server=YES
nfsd_flags=’-n8′
mountd=${nfs_server}
lockd=${nfs_server}
statd=${nfs_server}

And configure /etc/exports according to examples from the articles above. From client side I just needed to adjust /etc/fstab to the new NFS path.

Other notes

I also installed rtorrent as my torrent service and utilized cron to enable it after boot by adding this line to crontab configuration:

@reboot /usr/bin/tmux new-session -d '/usr/pkg/bin/rtorrent -D -I -n -o import=$
PWD/.rtorrent.rc'

.rtorrent.rc file just need to point to the session path, otherwise I kept it standard. After that tmux attach -d -t 0 command can be used to attach to the rtorrent session and CTRL+B D to detach from it.

In order to monitor HDD and its temperature I installed smartmontools utility. Current temperature can be retrieved using sudo smartctl -a /dev/rwd1|grep -i temp command (raw device should be used, thus rwd1 instead of wd1).

Finally I created the patch for the CPU temperature which didn’t make it to NetBSD 10 release, but hopefully will be available with NetBSD 10.1 through envstat(8) command. For time being I am using the patched kernel.

Overall my experience is quite positive with the new setup. There are still few improvements required, mainly for the data redundancy and potentially better power management and improved CPU support. The NAS device doesn’t like enclosed environments without proper airflow and my become extremely noisy in such case, however it works pretty silently be default. Actually the main noise was coming from 10-Gbit ethernet card, which has an active fan. Hard drives are also pretty audible unfortunately, thus SSDs would be preferred if silence is the priority.

Boot into SSD in QNAP TVS-675

Historically, my first NAS system was running NetBSD operating system (5th release if I am not mistaken), however after a hardware update, I was forced to switch to the Debian Linux due to hardware support issues at that time. Since then I have used few different distributions for over a decade (mainly Arch and Artix). Several years ago I upgraded the underlying platform again with the hopes to use it for at least as long as my previous system. This NAS setup is based on Biostar FX9830M motherboard. My goal in doing so is to make setup as cheap as possible by scrapping various parts from my closet. Unfortunately, it didn’t workout as planned, mainly due to uncomfortably noisy operation of aforementioned motherboad’s CPU fan, which I have never managed to remedy (unusual fan connector and cooling solution is a limiting factor, requiring to do custom solutions without knowing if that solution will work). In addition, the compact case didn’t provide enough airflow leading to slightly overheating HDDs which may affect their operational lifetime. Thus, I decided that it may be a good chance to come back to a long time dream of switching back to NetBSD in my NAS setup and I acquired QNAP TVS-675 NAS system for this purpose.

Under the hood it uses Zhaoxin KaiXian KX-U6580 8-core x86-64 CPU, has 6 HDD bays, has an option to add two PCIe Gen 3 x4 cards (the slot allows to attach longer cards thanks to the cut at the back of it), and supports NVMe expansion slots. It is almost a typical x86-64 PC, just wrapped into a “fancy” case specifically designed for NAS purposes. It operates pretty silently by default, with only audible noise coming from HDDs themselves during heavy operations (once high capacity SSDs are cheap enough I may switch to them).

The installation of the NetBSD 10.0 didn’t bring any surprises and went smoothly, however I faced the first issue just after the first reboot. There was no support to boot directly from SATA SSD (limitation of the system’s BIOS). Only the USB boot devices are supported. As a result, I started to look into options somehow to start booting from USB, to switch the root system to the SSD partition at the subsequent stages of the boot process. After reading a few resources (will mention them at the end of the article) and consulting the NetBSD developers, I decided to use a pretty simple approach with SSD root partition being mounted to /altroot, where kernel eventually is being instructed to chroot to it. The approach is similar – to root on ZFS or CGD functionality, however I didn’t configure a separate ramdisk for that, at least now.

In order to achieve that, I needed to prepare a bootable USB flash drive. One can use a lengthy instructions with the UEFI in mind, or install a minimal system using installation media, however I took a lazy approach by dissecting provided installation media itself. I just mounted it on my main NetBSD system and deleted all unnecessary folders and files, updated netbsd kernel (I am building a custom one, but GENERIC can be used) and updated /etc/rc script by mounting required partitions, including SSD root to /altroot and finally setting init.root kernel option to /altroot, so that it would switch to it during boot process:

PATH=/sbin:/bin
export PATH

mount -u /
mount -t tmpfs tmpfs /tmp
mount /altroot
sysctl -w init.root=/altroot

# done, move on to multiuser mode
exit 0

To complete a setup, I added required fields to /etc/fstab by setting correct /altroot mount point to the SSD’s root partition.

NAME=<usbpartionid>		/		ffs	rw		1 1
tmpfs			/tmp		tmpfs	rw		0 0
NAME=<ssdpartitionid>			/altroot	ffs	ro,log	1 1

The important part was to mount /altroot as read-only (ro), otherwise automatic filesystem check kicked-in and disrupted the boot process (it can’t check rw mounted partitions), requiring for the manual interaction. It is not an issue, since root partition will remount in rw mode by reading SSDs /etc/fstab. If there’s a need to change some files in /altroot before chroot (like rndseed for example), automount can be still rw but rc script should remount /altroot manually as read only shortly before sysctl command. Alternative approach may be to skip remounting root partition at all, but I haven’t tried it.

After reboot the system successfully booted and switched to SSD root folder!

Ideally I would like to avoid having USB flash drive being attached at all. I believe that integrated USB DISK module can be utilized somehow, but I haven’t explored that just yet, since i I don’t want to mess up the default boot configuration for the their own custom OS and it needs more time and skills to implement. For the time being, I would likely buy a very short USB flash driver, similar to Intenso Slim Line to make protrusion from the case as minimal as it can be.

The next step will be to setup my hard drives to using ZFS file system, setup NFS and samba as well as network with lagg(4) link aggregation. I am unsure at the moment, where I should load zfs modules, should it be before the chroot operation happens or it can be loaded after. Currently I am utilizing SSD root partition’s /etc/modules.conf file, which loads modules correctly, however I am not certain if it won’t be too late for mounting ZFS partitions automatically. I will definitely explore that in the next post.

Few useful resources on this topic:
HowTo: Making a bootable ISO w/ miniroot.kmod etc.
Building a NetBSD ramdisk kernel
Root On ZFS
NetBSD Full-Disk Encryption with CGD
Building NetBSD system components