OS and Software > All Things Unix > btrfs or zfs - Which way to the future?

btrfs or zfs - Which way to the future?

Any project that relies on a single person is doomed in the long run.
Since Oracle is relying on this for their Linux I'd find it hard to imagine they would let development on it slow down.

...such as Hans Reiser?

Is there something fundamentally wrong with ext4 then? I don't know the subject too deeply...why are the choices stated as only those two?

Ext4 is kinda hackish. The ext4 driver is capable of running an ordinary ext2 or 3 filesystem.

btrfs is a completely new system with snapshot support, internal compression, and a boatload of other features. ZFS is similar but it's more mature as long as you're using Solaris or maybe BSD; it's available for Linux as a FUSE driver (somewhat poor performance) and a kernel driver (which still deadlocks itself under pressure).

I would imagine the devs of each filesystem couldn't turn down a feature request. It's good for features but means a hard development cycle. Neither are ready for primetime under Linux yet. That said I've been playing with ZFS anyway and do like what I've been seeing so far. That's more-or-less where I'd throw my money, mainly because it's already mature and has a seemingly higher level of portability with the FUSE driver, native BSD and all that.

I do hope btrfs surprises me though, but in the short term I don't see that happening. Then again ZFS is struggling for stability.
--
That's odd...

reply to rchandra

reply to rchandra

ZFS is open source (depending on which version you're looking at) but not with a GPL-compatible license. It also supports RAID5, 6 and, umm, I'll call it RAID7 (tolerates 3 dead drives) along with the usual mirroring and striping.

brfs is obviously GPL since it ships with the linux kernel right now (and as of several years ago).

I'm going to put my money on zfs for now, but look forward to being pleasantly surprised by btrfs.
--
That's odd...

reply to timcuth
I'm must say, I'm kind of surprised. The Unix philosophy is do (really only) one thing, and do it well, with simple (as in the opposite of convoluted) interfaces in and out to make integration (such as pipelining) easy. The one thing to do is store and retrieve data, and provide access controls, and to a certain extent, organize it. The method I'm used to has the RAID and LVM pieces as a more-or-less totally independent layer between filesystem and block devices. It allows plug-n-substitute-n-play, whereas integrating dm into a filesystem takes away that modularity.

Hmmmm....
--
English is a difficult enough language to interpret correctly when its rules are followed, let alone when a writer chooses not to follow those rules.

Jeopardy! replies and randomcaps REALLY suck!

reply to timcuth
The following were published within the last few years and may be helpful:

● Benchmarking ZFS On FreeBSD vs. EXT4 & Btrfs On Linux

● Benchmarks Of ZFS-FUSE On Linux Against EXT4, Btrfs

● Workload Dependent Performance Evaluation of the Btrfs and ZFS Filesystems [pdf]

● Benchmarking ZFS, XFS, ext4 and btrfs with PostgreSQL 9.0 and 9.1

● ReiserFS vs ext4 vs XFS vs ZFS vs Btrfs – Linux filesystems compared

reply to timcuth

Thank you, KodiacZiller . That is very helpful info. Software freedom is quite important to me.

@FF4m3, I am not so much interested in current performance comparisons. I am sure both of these filesystems' performance will be inproved. But I do appreciate your posting them.

My main interest is in getting a feel for which one will become more prevalent on Linux in, say, the next two years. As Yogi Berra once said, "If you come to a fork in the road, take it."

Tim
--
"Life is like this long line, except at the end there ain't no merry-go-round." - Arthur on The King of Queens
~ Project Hope ~

reply to timcuth
Since ZFS is not GPL compatible, I believe that makes btrfs the future for Linux by default.

Well, like Broadcom wireless or nVidia graphics drivers, it's completely allowed to load non-GPL drivers into the kernel at runtime as modules. ZFS would qualify and that's how the work is being done now.

People who want ZFS can and will get it. The question is, will it be worth the additional hassle? My personal opinion is that, if both were magically made stable right now, I'd be all over ZFS.
--
That's odd...

reply to KodiacZiller

ZFS doesn't eat memory so much as it takes a more active roll by doing its own cache management. If you use ZFS on linux, you'll find the usual "cache" memory not being used. You've really only changed ownership of the filesystem cache. (Linux is currently my only experience with any significant weight to it)

As for IO speed, ZFS has a couple of interesting... uhh.. hacks (!) to speed it up - using an SSD as a secondary read cache and having a designated fsync() storage space for speed reasons whether you use the external ZIL feature or not. Then again raidz is notoriously bad for seek performance and makes software raid look good.

And I don't think checksumming is the number one reason to use either ZFS or btrfs. It's a high selling point but if you imagine the day they (or either one) become standard installs for Fedora, Ubuntu, etc. the typical workstation doesn't have several hard drives in their computer and the thought of using copies=2 (or whatever btrfs' equivalent is called) isn't going to appeal to anybody. Snapshot-assisted backups, in place compression and protection against any power outage are far more likely to be the killer features to the average user. Yes it's a different target than what enterprise has in mind but I think this model has to be taken into account as well. If ext4 really is just the crutch until btrfs is ready, this has to be the direction we're going and I'm not entirely sure it's the intended use case.
--
That's odd...

Using some ZFS terminology here because I'm not up to speed on Btrfs:

Checksumming is important even in the case of systems with a single-disk ZFS pool: it allows you to detect underlying storage subsystem issues. Yes, with a single disk you can't auto-repair the problem (mirrors and raidzX solve that), but you can still detect the problem. Furthermore, with detection on ZFS, it's even kind enough to tell you the filename of what has been impacted (if available; it will report hexadecimal offsets for metadata corruption or files which have since been deleted).

With ext4, ext3, UFS, NTFS, FAT/FAT32, blah blah, you can't even detect the situation, which is horrible.

So I'll make it clear: it is very, very important that a filesystem be able to detect if the bits it submit to the controller to write to the disk didn't get written correctly. This is not the same thing as a write failure; what I'm describing is a successful write (meaning the controller and disk both said "yep wrote things fine") of, say, a value of 0xFA (%11111010) -- but what actually got written to the platters was 0xEA (%11101010).

As for the the other features you mention -- I can't talk about ZFS on Linux as I have never used it, but:

- Filesystem snapshots (including incrementals) -- yes, very useful. However at least on FreeBSD these are quite possibly the #1 source of crashes and filesystem problems, even as of this writing. I have avoided them since ZFS was originally ported to FreeBSD and still continue to do so. They work beautifully/reliably on Solaris (used them at my previous job constantly).

- Compression and data deduplication -- useful, but presently only plausible on Solaris. On FreeBSD, use of either of these features completely destroys system usability. (I'm Jeremy Chadwick ). I have tested both of these features on FreeBSD which is partially how this crap came to light.

- Protection against power outages -- journalling solves this, which is something reiserfs, ext3 and ext4 all provide. FreeBSD, on the other hand, is a complete clusterf*** when it comes to journalling. I'm not even sure where to begin. All of the below things are independent of one another:

First we have softupdates (SU), which should not be misunderstood as a journalling replacement -- they aren't. Journalling does a lot more than SU. SU has been enabled on UFS/UFS2 (except root filesystem) on FreeBSD since the 5.x days, if I remember right. But wait until I get to FreeBSD 9...

Secondly we have gjournal. This is a journalling layer provided by GEOM. This was introduced in FreeBSD 7.0, and isn't commonly used given some of its requirements and configuration complexities.

Finally we have softupdate journalling (SUJ or SU+J), which was introduced starting with FreeBSD 9.0 (9.0-RELEASE) The very bad, brash, and idiotic decision was made to enable this by default in the 9.0 installer because it was deemed stable. Very quickly people began reporting filesystem inconsistencies and silent corruption, and kernel panics especially when using snapshots. Disabling the journalling part of SUJ (meaning switching to just SU) solved the 2 former problems. The problems were admitted by one of the authors of the code, but who couldn't fix the problem promptly because his cohort who wrote most of the code had disappeared/went on hiatus. A fix was eventually committed, but there have been no 9.0-STABLE snapshots provided anywhere, and there are still reports of problems even after the fact (some involving UFS filesystem snapshots). The same author also confirms those problems.

So at least in the FreeBSD world, things are not in very good shape. ZFS has the most attention (understandably), which is good, but I still would not call it "stable" given the existing problems with it. It still requires kernel tuning to get stability as well (kmem map exhaustion is still a problem -- really!). And SUJ should be avoided like the plague (I'm left with the impression after this nobody is going to dare use it given what it has done).

Oh, I forgot to mention one of the ZFS features that drives a lot of system administrators to it (myself included): the integration of a LVM. I know rchandra pointed out the true UNIX philosophy (and I agree with it), but the LVM-esque integration in ZFS makes managing filesystems so incredibly easy. I know the true UNIX way would be to make many small things all work together, but sadly that is not easily do-able with the kernel; the more you push out to userland the worse performance you get (Linux went through this same issue back in the late 90s; am I the only one who remembers the huge NFS-in-kernel vs. NFS-in-userland debacle during Linux 1.2 and 1.3? ).

So if I was to switch to, say, graid, I would lose checksumming and the LVM-esque features, forcing me to make separate filesystems (rather than have multiple filesystems that share a single pool of disks). Folks may not know that FreeBSD does not have a decent volume manager; gvinum and vinum are neglected and nobody bothers to use them any more.

On the other hand, scrapping ZFS would allow me to get my performance back, I wouldn't need 8GB+ of RAM in systems or dedicated servers, and so on. Of course, on my current hosting server I use UFS2 (SU, not SUJ) exclusively on all filesystems and I "just deal with it". Thankfully it works, including on FreeBSD 9, as long as you remember to disable journalling during the install. Don't get me started on the other horrible decisions in bsdinstall though, like the default now being a single filesystem (/) which completely breaks booting into single-user to do installworld. Whoever decided that did not think it through -- still want to wring the neck of that person...
--
Making life hard for others since 1977.
I speak for myself and not my employer/affiliates of my employer.