RU version is available. Content is displayed in original English for accuracy.
Advertisement
Advertisement
⚡ Community Insights
Discussion Sentiment
60% Positive
Analyzed from 2270 words in the discussion.
Trending Topics
#zfs#https#nas#drives#more#linux#don#need#storage#freebsd
Discussion (48 Comments)Read Original on HackerNews
I ended up on shucking 4x the 14 TB WD Elements Desktop. They contain helium drives, the WD140EDGZ in my case, and are about a third cheaper than 4x the 12 TB WD Red Plus drives (which are air-filled). The shucking was easier than I expected too, and the performance seems very comparable. The warranty is a definite downside (European, so no Magnuson-Moss), but I think I can even get them back in their enclosure should they fail during the 2-year warranty period.
I've put some second hand 256 GB M.2 SSDs in there as boot drives. It was a bit of a struggle to get it to work in a way that failure of one of the drives doesn't hold up booting, combined with LUKS, TPM keys and ZFS on root. Learned a lot about systemd-boot which I have never used before, but feels a lot saner to me than grub ever was. So now I have a large script which debootstraps a Debian based NAS into being.
I noticed that there are a lot of ZFS myths and cargo culting. For example TFA mentions ECC RAM, which in some circles is a must-have because ZFS would wreck your pool during a scrub otherwise, which is a myth. It's also very expensive, especially this year. You also don't need much RAM for ZFS, L2ARC doesn't use much RAM at all, to name a few others.
Still doubting about setting `dnodesize=auto` (which is the default), because there are some horror stories about that [1]. And it seems impossible to find a cloud storage provider with reasonable prices that supports `zfs send`. Rsync.net upped their minimum order to 10 TiB recently, which is far too much for my use case.
[1] https://github.com/openzfs/zfs/issues/11353
[2] https://www.rsync.net/products/zfsintro.html
Like I’m already giving up two full drives for redundancy (which saved my ass - I recently had two drives fail on me in quick succession — both SSDs from what looks like an identical batch) but then the advice is kinda saying I need to keep at least another drive worth of space free for the pool to perform well and not crap itself. That hurts with current prices for sure.
That under-states the matter. It is a terrible time, price-wise, to build a NAS.
I'd almost rather have no AI whatsoever and have storage 1/10 the price of pre-AI times.
(If there were a magical choice between having AI and significantly more expensive storage, and having no AI and some program to dump that investment money into getting and somehow leveraging significantly more available storage, that is.)
You have SATA or SAS to pick from. The CPU requirements for a storage server are not high. On a typical ATX board you have motherboard SATA and can put SAS controllers in the spare PCIe slots.
My first "NAS" was two 22TB hard drives in a ZFS pool on my motherboard SATA
It's a set and forget OS that will run for years without requiring your attention. But these days it has decent container support for hosting services on.
I don't know much about ZFS, but it sounds like I need to learn. Docker may have conquered the world, but I plan to stay with LXD for services.
The one thing I take issue with: an appliance like this runs 24/7. It should be low power and fanless. A processor like the N100 seems like the obvious choice.
But the specs also said ECC RAM and I don't think the N100 supports that.
https://HomeFree.host
(And I know I have to do that, because when the disk fails it beeps and lights a led near the bad disk)
It’s easy to build a NAS such as the one described in this article, but in the long run, data loss is significantly more likely.
Also, any guide like this that doesn’t guide you through “disk 3 failed, this is how you safely replace it” is imho incomplete, even if it doesn’t go through telling you how you know a disk has failed.
> I am creating a RAIDZ1 (RAID 5) zpool. That means 1 drive redundancy in-case of failure
A friend once told me that RAID5 has a high latency cost, because every Write requires a Read to update the stripes across all drives, and while this made sense when drives were expensive, nowadays you might as well do a RAID10 instead, and trade space for latency.
Is this still true with ZFS RAIDZ1?
``` Unlike traditional RAID5 and RAID6 implementations, ZFS supports partial-stripe writes. This has a number of important advantages but also presents some implications for space calculation that we'll need to consider. Supporting partial stripe writes means that in our 7wZ2 vdev example, we can support a write of 12 total sectors even though 12 is not an even multiple of our stripe width (7). 12 is evenly divisible by +1 (3 in this case), so we don't even need any padding. We would have a single full stripe of 7 sectors (2 parity sectors plus 5 data sectors) followed by a partial stripe with 2 parity sectors and 3 data sectors. This will be important because even though we can support partial stripe writes, every stripe (including those partial stripes) need a full set of p parity sectors. ```
I don't have the answer to the latency question, but HDDs have shot up in $/TB over the last couple years too. They are once again kind of expensive.
Please - dont - use - consumer ssd's - with zfs raidz1
They support ECC ram, 4 caddies, one extra PCIe slot, and to my knowledge you're not cpu limited for a zfs file server usecase.
Keep in mind though, all you need is linux* support, iDRAC, ECC if you're a snob, and drive bays ... and that's basically any free server.
In my extremely opinionated opinion I would only get used enterprise server gear, because a zfs file server will just work unless hardware fails. And a UPS.
*ZFS will be a more natural choice on FreeBSD. It's better documented, and will meet Linux 1:1 in hardware compatibility for this.
It works well enough though and has lasted me over a decade at this point. 16GB DDR3 ECC, an old 4 core/8 thread Xeon, 4x14TB drives and the Mellanox NIC.
Along with various other devices (including a large Synology which I wouldn’t buy today), I run Proxmox on a small two bay+two nvme Terramaster. I have a bare bones Ubuntu LXC running Samba configured for Apple Time Machine, an VM running Scrypyed, and PBS for Proxmox backups. Nothing on it is critical so I don’t bother with any storage redundancy.
Recently replaced the internal USB boot drive with a small NVMe USB enclosure; using a 90-degree USB connector and using a dremel to sand away an opening for the cable to come out so I could mount the enclosure externally.
You're describing ZFS.
Truenas for example add replication, external cloud sync, gui (yes it’s important not to wipe your data with wrong command), HA, custom caching, containers to extend ot with other apps, raid-z expansion with openzfs, and other for monitoring, user management and a lot more.
Other solutions also add more than just a zfs with ssh access.
Running ZFS on anything but Solaris/Illumos/FreeBSD is asinine.
ZFS is a permanent second-class citizen on Linux (due to usual open-source politics). This will never resolve.
I don't want to trust my data to some half-assed out-of-tree solution that may or may not break in a week.
FreeBSD ZFS support has matured and is outstanding. Quality-wise it has reached parity with Illumos.
If you can afford Solaris then you're probably not building your own NAS from parts of lesser computers.
Run ZFS on Raspberry Pi, on home builds, on Intel, on AMD, on other ARM chipsets.
I think you're over-stating things. Debian is fine for this. I do think FreeBSD is a better platform for myself.
The code bases adhere (modulo ZFS version numbers) to a spec and you can safely migrate the pools between OS. I've done it multiple times both directions.
You can not do this with BTRFS and other Linux things, I consider this feature of (Open)ZFS a killer-context for me: It's OS portable. I wish Mac OSX hadn't walked out of the room when Oracle went legal.
There is actually a btrfs driver for Windows [0]. I've used it a few times before, and it works surprisingly well. You probably wouldn't want to use it for any serious work, but that's not because it's technically flawed, but more because it isn't extensively tested or commercially-supported.
[0]: https://github.com/maharmstone/btrfs
> ZFS is a permanent second-class citizen on Linux
Linux is the primary target of OpenZFS [0] [1], and has been since 2020 [2]. It may not be supported by the Linux kernel developers, but it's supported by the ZFS developers, and that's all that really matters.
> I don't want to trust my data to some half-assed out-of-tree solution that may or may not break in a week.
Sure, it's an out-of-tree module, but that doesn't mean that it will randomly break all the time; it just means that you may occasionally need to wait for a new OpenZFS release before upgrading your kernel.
> FreeBSD ZFS support has matured and is outstanding.
Agreed, but Linux and FreeBSD both use the same ZFS [3], so I don't really see how the ZFS in FreeBSD can be better than the one in Linux. The tooling and install procedure is certainly better on FreeBSD, but the actual filesystem code is the same (and is probably slightly more robust on Linux since that's going to be where most of the testing occurs).
[0]: https://github.com/openzfs/zfs#supported-kernels-and-distrib...
[1]: https://github.com/openzfs/zfs/pull/8987
[2]: https://github.com/openzfs/zfs/releases/tag/zfs-2.0.0
[3]: https://docs.freebsd.org/en/books/handbook/zfs/