ZFS is a combined file system and logical volume manager designed by Sun Microsystems. The features of ZFS include data integrity verification against data corruption modes, support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software, licensed under the Common Development and Distribution License (CDDL). The ZFS name was a trademark of Oracle^[3] until September 20, 2011.^[4]

History[link]

ZFS was designed and implemented by a team at Sun led by Jeff Bonwick. It was announced on September 14, 2004.^[5] Source code for ZFS was integrated into the main trunk of Solaris development on October 31, 2005^[6] and released as part of build 27 of OpenSolaris on November 16, 2005. Sun announced that ZFS was included in the 6/06 update to Solaris 10 in June 2006, one year after the opening of the OpenSolaris community.^[7]

The name originally stood for "Zettabyte File System"^[8] but today does not stand for anything.^[9] A ZFS file system can store up to 256 quadrillion zettabytes (ZB), where a zettabyte is 2⁷⁰ bytes.

Release history[link]

As new features are introduced the version number of the ZPool and Z file system are incremented to designate the format and features available.^[10][11]

Legend:

Features[link]

Data Integrity[link]

One major feature that distinguishes ZFS from other file systems is that ZFS is designed from the ground up with a focus on data integrity. That is, protect the user's data on disk, against silent data corruption caused by e.g., bit rot, current spikes, bugs in disk firmware, ghost writes, etc.

Data Integrity is a high priority in ZFS because recent research shows that none of the currently widespread file systems — such as Ext, XFS, JFS, UFS, or NTFS — nor Hardware RAID provide sufficient protection against such problems.^{[12][13][14][15][16]} It is well known that Hardware RAID has some issues with data integrity. Initial research indicates that ZFS clearly protects data better than earlier solutions.^[17]

A modern hard disk devotes a large portion of its capacity to error detection data. Many errors occur during normal usage, but are corrected by the disk's internal software, and thus are not visible to the host software. A tiny fraction of the detected errors are not correctable. For example, a modern Enterprise SAS disk specification estimates this fraction to be one uncorrected error in every 10¹⁶ bits.^[18] A modern Enterprise Fibre Channel disk which uses the new 520 byte sectors to support the new DIF Data Integrity Field standard to combat data corruption, reports similar error rates.^[19] The worst type of errors are those that go unnoticed, and are not even detected by the disk firmware or the host operating system. This is known as "silent corruption". A real life study of 1.5 million HDDs in the NetApp database found that on average 1 in 90 SATA drives will have silent corruption which is not caught by hardware RAID verification process; for a RAID-5 system that works out to one undetected error for every 67 TB of data read.^[20]

However, there are many other external error sources than the disk itself. For instance, the disk cable might be slightly loose, the power supply might be flaky,^[21] external vibrations such as a loud sound,^[22] the Fibre Channel switch might be faulty,^[23] cosmic radiation and many other types of soft errors, etc. In 39,000 storage systems that were analyzed, firmware bugs accounted for between 5–10% of storage failures.^[24] All in all, the error rates as observed by a CERN study on silent corruption, are far higher than one in every 10¹⁶ bits.^[25]

Silent data corruption has not been a serious concern while storage devices remained relatively small and slow. Hence, a user very rarely faced silent corruption, so it was not deemed to be a problem that required a solution. With the advent of larger drives and very fast RAID setups, a user is capable of transferring 10¹⁶ bits in a reasonably short time. In particular, ZFS creator Jeff Bonwick stated that the fast database at Greenplum — a database software company located in San Mateo, California specializing in enterprise data cloud solutions for large-scale data warehousing and analytics – faces silent corruption every 15 minutes,^[26] which is one of the reasons that Greenplum now base their fast database solution on ZFS. These large and fast raid setups require new file systems that focus on data integrity. This is one of the design goals of ZFS, as explained by Jeff Bonwick.^[26]

For ZFS, data integrity is achieved by using a (Fletcher-based) checksum or a (SHA-256) hash throughout the file system tree.^[27] Each block of data is checksummed and the checksum value is then saved in the pointer to that block—rather than at the actual block itself. Next, the block pointer is checksummed, with the value being saved at its pointer. This checksumming continues all the way up the file system's data hierarchy to the root node, which is also checksummed, thus creating a Merkle tree.^[27] When a block is accessed, regardless of whether it is data or meta-data, its checksum is calculated and compared with the stored checksum value of what it "should" be. If the checksums match, the data is passed up the programming stack to the process that asked for it. If the values do not match, then ZFS can heal the data if the storage pool has redundancy via ZFS mirroring or RAID.^[28] If the storage pool consists of a single disk it is possible to provide such redundancy by specifying "copies=2" (or "copies=3") which means that data will be stored twice (thrice) on the disk, effectively halving (1/3) the storage capacity of the disk.^[29] If redundancy exists, then ZFS fetches the second copy of the data (or recreates it via a RAID recovery mechanism), and recalculates the checksum—hopefully reproducing the original value this time. If the data passes the integrity check, the system can then update the first copy with known-good data so that redundancy can be restored.

ZFS cannot fully protect the user's data when using a hardware RAID controller, as it is not able to perform the automatic self-healing unless it controls the redundancy of the disks and data.^[30] ZFS prefers direct, exclusive access to the disks, with nothing in between that interferes. If the user insists on using hardware-level RAID, the controller should be configured as JBOD mode (i.e. turn off RAID-functionality) for ZFS to be able to guarantee data integrity. Note that hardware RAID configured as JBOD may still detach disks that do not respond in time; and as such may require TLER/CCTL/ERC-enabled disks to prevent drive dropouts:^[31]

These limitations do not apply when using a non-RAID controller, which is the preferred method of supplying disks to ZFS. A non-RAID controller is generally called a Host Bus Adapter (HBA) and allows the operating system to control timeout and error control, rather than the RAID controller which generally has very strict timeout control.

ZFS has no "fsck" repair tool, common on Unix/Linux filesystem, which examines and repairs data. Instead, ZFS has a repair tool called "scrub" which examines and repairs Silent Corruption and other problems. Some differences are:

• fsck must be run on an offline filesystem, which means the filesystem must be unmounted and not usable while being repaired.
• fsck usually only checks metadata (such as the journal log) but never checks the data itself. This means, after an fsck, the data might still be corrupt.
• scrub does not need the ZFS filesystem to be taken offline. scrub is designed to be used on a working, mounted alive filesystem.
• scrub checks everything, including metadata and the data.

The official recommendation from Sun/Oracle is to scrub once every month with Enterprise disks, because they have much higher reliability than cheap commodity disks. If using cheap commodity disks, scrub every week.

However, no system is immune to bugs or hardware not following standards.

"...For example: FLUSH CACHE should only return, when the cache is flushed. But there are dirt cheap converter chips that sends the FLUSH CACHE to disk, but returns a successful FLUSH CACHE in the same moment back to the OS (of course without having NVRAM on disk or in a controller as this would allow to ignore CACHE FLUSH). Or interface converters reordering commands in really funny ways. By such reordering it may happen, that the uberblock is written to disk, before the rest of the structure has been written to disk..." ^[32]

Thus, there are known cases where ZFS has had problems. Therefore, as an extra safety measure, it is possible to go back in time by using the "-F" flag with the "zpool" command. ZFS use Copy-On-Write, which means old data is not altered. Whenever data is edited and updated, the old data is always left intact, and only the edits are stored, on a new place on the disk. This means every change can be traced back in time. This allows the user to discard the latest change which caused the problem, and instead go back to an earlier functioning state. This is also how ZFS Snapshots works.

Storage pools[link]

Unlike traditional file systems, which reside on single devices and thus require a volume manager to use more than one device, ZFS filesystems are built on top of virtual storage pools called zpools. A zpool is constructed of virtual devices (vdevs), which are themselves constructed of block devices: files, hard drive partitions, or entire drives, with the last being the recommended usage.^[33] Block devices within a vdev may be configured in different ways, depending on needs and space available: non-redundantly (similar to RAID 0), as a mirror (RAID 1) of two or more devices, as a RAID-Z (similar to RAID-5) group of three or more devices, or as a RAID-Z2 (similar to RAID-6) group of four or more devices.^[34] In July 2009, triple-parity RAID-Z3 was added to OpenSolaris.^[35][36]

Thus, a zpool (ZFS storage pool) is vaguely similar to a computer's RAM. The total RAM pool capacity depends on the number of RAM memory sticks and the size of each stick. Likewise, a zpool consists of one or more vdevs. Each vdev can be viewed as a group of hard disks (or partitions, or files, etc.). Each vdev should have redundancy because if a vdev is lost, then the whole zpool is lost. Thus, each vdev should be configured as RAID-Z1, RAID-Z2, mirror, etc. It is not possible to change the number of drives in an existing vdev (Block Pointer Rewrite will allow this, and also allow defragmentation), but it is always possible to increase storage capacity by adding a new vdev to a zpool. It is possible to swap a drive to a larger drive and resilver (repair) the zpool. If this procedure is repeated for every disk in a vdev, then the zpool will grow in capacity when the last drive is resilvered. A vdev will have the same capacity as the smallest drive in the group. For instance, a vdev consisting of three 500 GB and one 700 GB drive, will have a capacity of 4 x 500 GB.

In addition, pools can have hot spares to compensate for failing disks. When mirroring, block devices can be grouped according to physical chassis, so that the filesystem can continue in the case of the failure of an entire chassis.

Storage pool composition is not limited to similar devices but can consist of ad-hoc, heterogeneous collections of devices, which ZFS seamlessly pools together, subsequently doling out space to diverse filesystems as needed. Arbitrary storage device types can be added to existing pools to expand their size at any time. ^[37]

The storage capacity of all vdevs is available to all of the file system instances in the zpool. A quota can be set to limit the amount of space a file system instance can occupy, and a reservation can be set to guarantee that space will be available to a file system instance.

ZFS cache: L2ARC, ZIL[link]

ZFS uses different layers of disk cache to speed up read and write operations. Ideally, all data should be stored in RAM but that is too expensive. Therefore, the data is automatically cached in a hierarchy to optimize performance vs cost http://dtrace.org/blogs/brendan/2008/07/22/zfs-l2arc/. Frequently accessed data is stored in RAM, and less frequently accessed data can be stored on slower media, such as SSD disks. Data that is not often accessed, is not cached and left on the slow hard drives. If old data is suddenly read a lot, ZFS will automatically move it to SSD disks or to RAM.

The first level of disk cache is RAM, which uses a variant of the ARC algorithm. It is similar to a level 1 CPU cache. RAM will always be used for caching, thus this level is always present. There are claims that ZFS servers must have huge amounts of RAM, but that is not true. It is a misinterpretation of the desire to have large ARC disk caches. The ARC is very clever and efficient, which means disks will often not be touched at all, provided the ARC size is sufficiently large. In worst case, if the RAM size is very low (say 1GB RAM) there will hardly be any ARC at all, then ZFS always needs to reach for the disks. This means read performance degrades to disk speed.

The second level of disk cache are SSD disks. This level is optional, and is easy to add or remove during live usage, there is no need to shutdown the zpool. There are two different caches, one cache for reads, and one cache for writes.

• The read SSD cache is called L2ARC and is similar to Level 2 CPU cache. The L2ARC will also considerably speed up Deduplication, if the entire Dedup table can be cached in L2ARC. It can take several hours to fully populate the L2ARC (before it has decided which data is "hot", and should be cached). If the L2ARC device is lost, all reads will go out to the disks which slows down performance, but nothing else will happen, no data will be lost.
• The write SSD cache is called ZIL (ZFS Intent Log). ZIL improves synchronous writes and is similar to a journal log. All data is written to the ZIL, but is only read after a crash. Thus, the ZIL data is normally never read. It is important that the ZIL uses a very fast disk, for superior performance a disk consisting of battery backed up RAM, such as the ZeusRAM should be used. Because the ZIL is written to a lot, an SSD disk will eventually be worn out, but a RAM disk will not. If the ZIL device is lost, it is possible to lose the latest writes, therefore the ZIL device should be mirrored. In earlier versions of ZFS, loss of the ZIL device could result in loss of the entire zpool, therefore one should upgrade ZFS if planning to use ZIL. In case there is no separate ZIL device added to the zpool, a part of the zpool will automatically be used as ZIL, thus there is always a ZIL on every zpool.

Capacity[link]

ZFS is a 128-bit file system, so it can address 1.84 × 10¹⁹ times more data than 64-bit systems such as NTFS. The limitations of ZFS are designed to be so large that they would never be encountered. This was assured by surpassing physical rather than theoretical limitations—there simply is not enough usable matter on the planet Earth to support a maximized ZFS filesystem.^{[citation needed]} Some theoretical limits in ZFS are:

• 2⁴⁸ — Number of entries in any individual directory^[38]
• 16 exabytes (2⁶⁴ bytes) — Maximum size of a single file
• 16 exabytes — Maximum size of any attribute
• 256 zettabytes (2⁷⁸ bytes) — Maximum size of any zpool
• 2⁵⁶ — Number of attributes of a file (actually constrained to 2⁴⁸ for the number of files in a ZFS file system)
• 2⁶⁴ — Number of devices in any zpool
• 2⁶⁴ — Number of zpools in a system
• 2⁶⁴ — Number of file systems in a zpool

Copy-on-write transactional model[link]

ZFS uses a copy-on-write transactional object model. All block pointers within the filesystem contain a 256-bit checksum or 256-bit hash (currently a choice between Fletcher-2, Fletcher-4, or SHA-256)^[39] of the target block which is verified when the block is read. Blocks containing active data are never overwritten in place; instead, a new block is allocated, modified data is written to it, then any metadata blocks referencing it are similarly read, reallocated, and written. To reduce the overhead of this process, multiple updates are grouped into transaction groups, and ZIL (intent log) write cache is used when synchronous write semantics are required. The blocks are arranged in a tree, as are their checksums (see Merkle signature scheme).

Snapshots and clones[link]

An advantage of copy-on-write is that when ZFS writes new data, the blocks containing the old data can be retained, allowing a snapshot version of the file system to be maintained. ZFS snapshots are created very quickly, since all the data composing the snapshot is already stored; they are also space efficient, since any unchanged data is shared among the file system and its snapshots.

Writeable snapshots ("clones") can also be created, resulting in two independent file systems that share a set of blocks. As changes are made to any of the clone file systems, new data blocks are created to reflect those changes, but any unchanged blocks continue to be shared, no matter how many clones exist. This is an implementation of the Copy-on-write principle.

Dynamic striping[link]

Dynamic striping across all devices to maximize throughput means that as additional devices are added to the zpool, the stripe width automatically expands to include them; thus all disks in a pool are used, which balances the write load across them.

Variable block sizes[link]

ZFS uses variable-sized blocks of up to 128 kilobytes. The currently available code allows the administrator to tune the maximum block size used as certain workloads do not perform well with large blocks. If data compression (LZJB) is enabled, variable block sizes are used. If a block can be compressed to fit into a smaller block size, the smaller size is used on the disk to use less storage and improve IO throughput (though at the cost of increased CPU use for the compression and decompression operations).

Lightweight filesystem creation[link]

In ZFS, filesystem manipulation within a storage pool is easier than volume manipulation within a traditional filesystem; the time and effort required to create or expand a ZFS filesystem is closer to that of making a new directory than it is to volume manipulation in some other systems.

Cache management[link]

ZFS also uses the ARC, a new method for Read cache management, instead of the traditional Solaris virtual memory page cache. For write caching, ZFS employs the ZFS Intent Log (ZIL). ZFS makes allowances for both of these methods to incorporate separate virtual devices to improve the total IOPS. For read operations it is the "cache" vdev and for write operations it is the "log" vdev.^[40]

Adaptive endianness[link]

Pools and their associated ZFS file systems can be moved between different platform architectures, including systems implementing different byte orders. The ZFS block pointer format stores filesystem metadata in an endian-adaptive way; individual metadata blocks are written with the native byte order of the system writing the block. When reading, if the stored endianness does not match the endianness of the system, the metadata is byte-swapped in memory.

This does not affect the stored data itself; as is usual in POSIX systems, files appear to applications as simple arrays of bytes, so applications creating and reading data remain responsible for doing so in a way independent of the underlying system's endianness.

Deduplication[link]

This section may require copy editing for grammar, style, cohesion, tone, or spelling. You can assist by editing it. (January 2012)

Deduplication capability was added to the ZFS source repository at the end of October 2009.^[41] The OpenSolaris ZFS development packages have been available since December 3, 2009 (build 128).

The effective use of deduplication may require additional hardware. ZFS designers recommend 2 GB of RAM for every 1 TB of storage. For example, at least 32 GB of memory is recommended for 20 TB of storage.^[42] However, some further discussion on the zfs-discuss lists,^[43] and testing on FreeBSD^[44] by individuals affiliated with the project, have shown this value to be significantly higher than the prescribed 2 GB of RAM for every 1 TB of storage. If RAM is lacking, consider adding an SSD as a cache, which will automatically handle the large deduplication tables. This can speed up deduplication performance by a factor of eight or more. Insufficient physical memory or lack of ZFS cache results in virtual memory thrashing, which can either lower performance, or result in complete memory starvation.^[44]

As of today with Solaris 11 Express, deduplication can cause several problems if you are not aware of the limitations involved.^[45]

Encryption[link]

The encryption capability in ZFS^[46] is embedded into the I/O pipeline. During writes a block may be compressed, encrypted, checksummed and then deduplicated in that order. The policy for encryption is set at the dataset level when datasets (file systems or ZVOLs) are created. The wrapping keys provided by the user/administrator can be changed at any time without taking the file system off line. The default behaviour is for the wrapping key to be inherited by any child data sets. The data encryption keys are randomly generated at dataset creation time. Only descendant datasets (snapshots and clones) share data encryption keys.^[47] A command to switch to a new data encryption key for the clone or at any time is provided — this does not re-encrypt already existing data, instead utilising an encrypted master-key mechanism.

Additional capabilities[link]

• Explicit I/O priority with deadline scheduling.
• Claimed globally optimal I/O sorting and aggregation.
• Multiple independent prefetch streams with automatic length and stride detection.
• Parallel, constant-time directory operations.
• End-to-end checksumming, using a kind of "Data Integrity Field", allowing data corruption detection (and recovery if you have redundancy in the pool).
• Transparent filesystem compression. Supports LZJB and gzip.^[48]
• Intelligent scrubbing and resilvering (resyncing).^[49]
• Load and space usage sharing among disks in the pool.^[50]
• Ditto blocks: Configurable data replication per filesystem, with zero, one or two extra copies requested per write for user data, and with that same base number of copies plus one or two for metadata (according to metadata importance).^[51] If the pool has several devices, ZFS tries to replicate over different devices. Ditto blocks are primarily an additional protection against corrupted sectors, not against total disk failure.^[52]
• ZFS design (copy-on-write + superblocks) is safe when using disks with write cache enabled, if they honor the write barriers. This feature provides safety and a performance boost compared with some other filesystems.
• On Solaris, when entire disks are added to a ZFS pool, ZFS automatically enables their write cache. This is not done when ZFS only manages discrete slices of the disk, since it does not know if other slices are managed by non-write-cache safe filesystems, like UFS. The FreeBSD implementation can handle disk flushes for partitions thanks to its GEOM framework, and therefore does not suffer from this limitation
• Per-user and per-group quotas support.^[53]
• Filesystem encryption since Solaris 11 Express^[1]
• Pools can be imported in read-only mode
• It is possible to recover data by rolling back entire transactions at the time of importing the zpool.
• Planned features:
  • The so-called Block Pointer Rewrite functionality is due to be added in the same time frame, paving the way for resizing pools, defragmentation, (re-)applying compression on filesystems and so on.^[54]

Limitations[link]

• Capacity expansion is normally achieved by adding groups of disks as a top-level vdev: simple device, RAID-Z, RAID-Z2, RAID-Z3, or mirrored. Newly written data will dynamically start to use all available vdevs. It is also possible to expand the array by iteratively swapping each drive in the array with a bigger drive and waiting for ZFS to heal itself — the heal time will depend on the amount of stored information, not the disk size. The new free space will not be available until all the disks have been swapped.^{[citation needed]}
• It is currently^[when?] not possible to reduce the number of top-level vdevs in a pool nor otherwise reduce pool capacity.^[55] This functionality was said to be in development already in 2007.^[56] It is not available as of Solaris 10 9/10 (AKA update 9).
• It is not possible to add a disk as a column to a RAID-Z, RAID-Z2, or RAID-Z3 vdev. This feature depends on the block pointer rewrite functionality due to be added soon. One can however create a new RAID-Z vdev and add it to the zpool.^[57]
• Vdevs cannot be nested, so a mirror or RAID-Z top-level vdev can only contain files or disks. Mirrors of mirrors (or other combinations) are not allowed.^{[citation needed]}
• Reconfiguring the number of devices in a top-level vdev requires copying data offline, destroying the pool, and recreating the pool with the new top-level vdev configuration, except for adding extra redundancy to an existing mirror, which can be done at any time or if all top level vdevs are mirrors with sufficient redundancy the zpool split^[58] command can be used to remove a vdev from each top level vdev in the pool, creating a 2nd pool with identical data.
• Resilver (repair) of a crashed disk in a ZFS raid takes a long time. This applies to all types of RAID, in one way or another. This means that future large disks, say 5 TB or 6 TB, can take several days to repair. This means that raidz1 (similar to RAID-5) should be avoided, because repairing a raid puts additional stress on the other disks which might cause them to crash, losing all data in the storage pool if configured as raidz1. Therefore, with large disks one should use raidz2 (allow two disks to crash) or raidz3 (allow three disks to crash). Adam Leventhal explains this problem further. It should be noted, however, that ZFS RAID differs from conventional RAID solutions by only reconstructing live data and metadata when replacing a disk, not the entirety of the disk including blank and garbage blocks, which means that replacing a member disk on a ZFS pool that is only partially full will take proportionately less time compared to conventional RAID.^[59]
• IOPS performance of a ZFS storage pool can suffer if the ZFS raid is not appropriately configured. This applies to all types of RAID, in one way or another. If the zpool consists of only one group of disks configured as, say, eight disks in raidz2 – then the write IOPS performance will be that of a single disk, however, read IOPS will be the sum of eight individual disks. This means, to get high write IOPS performance, the zpool should consist of several vdevs, because one vdev gives the write IOPS of a single disk. However, there are ways to mitigate this IOPS performance problem, for instance add SSDs as L2ARC cache — which can boost IOPS into 100.000s.^[60] In short, a zpool should consist of several groups of vdevs, each vdev consisting of 8–12 disks. It is not recommended to create a zpool with a single large vdev, say 20 disks, because write IOPS performance will be that of a single disk, which also means that resilver time will be very long (possibly weeks with future large drives).

Platforms[link]

Solaris 10[link]

ZFS is part of Sun's own Solaris operating system and is thus available on both SPARC and x86-based systems. Since the code for ZFS is open source, a port to other operating systems and platforms can be produced without Sun's involvement.

Solaris 11[link]

After Oracle's Solaris 11 Express release, the OS/Net consolidation (the main OS code) was made proprietary and closed-source, and further ZFS upgrades and implementations inside Solaris (such as encryption) are not compatible with other non-proprietary implementations which use previous versions of ZFS.

When creating a new ZFS pool, to retain the ability to use access the pool from other non-proprietary Solaris-based distributions, it is recommended to upgrade to Solaris 11 Express from OpenSolaris (snv_134b), and thereby stay at ZFS version 28.

OpenSolaris[link]

OpenSolaris 2008.05 and 2009.06 use ZFS as their default filesystem. There are over a dozen 3rd party distributions, of which nearly a dozen are mentioned here. (OpenIndiana and Illumos are two new distributions not included on the OpenSolaris distribution reference page.)

OpenIndiana[link]

OpenIndiana 148 and 151 use ZFS version 28, as implemented in Illumos.

By upgrading from OpenSolaris snv_134 to both OpenIndiana and Solaris 11 Express, one also has the ability to upgrade and separately boot Solaris 11 Express on the same ZFS pool, but one should not install Solaris 11 Express first because of ZFS incompatibilities introduced by Oracle past ZFS version 28.^[61]

FreeBSD[link]

Paweł Jakub Dawidek ported ZFS to FreeBSD, and it has been part of FreeBSD since version 7.0.^[62] This includes zfsboot, which allows booting FreeBSD directly from a ZFS volume.^[63][64]

FreeBSD's ZFS implementation is fully functional; the only missing features are kernel CIFS server and iSCSI, but at least the latter can be added using externally available packages.^[65] Samba can be used to provide a userspace CIFS server.

FreeBSD 7-stable (where updates to the series of versions 7.x are committed to) uses zpool version 6.

FreeBSD version 8 includes a much-updated implementation of ZFS, and zpool version 13 is supported in FreeBSD release 8.0.^[66] zpool version 14 support was added to the 8-stable branch on January 11, 2010,^[67] and is included in FreeBSD release 8.1. zpool version 15 is supported in release 8.2.^[68] The 8-stable branch gained support for zpool version v28 and zfs version 5 in early June 2011.^[69] These changes were released mid-April 2012 with FreeBSD 8.3.^[70]

FreeBSD 9.0-RELEASE uses ZFS Pool version 28.^[71][72]

FreeNAS[link]

FreeNAS, an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD.

PC-BSD[link]

PC-BSD is a desktop version of FreeBSD, which inherits FreeBSD's ZFS support, similarly to FreeNAS. It also allows installation with disk encryption using geli.

GNU/kFreeBSD[link]

Being based on the FreeBSD kernel, GNU/kFreeBSD has ZFS support from the kernel. However, it depends on the distribution of GNU/kFreeBSD whether the necessary userland tools are available. The only distribution of this system to the date (Debian GNU/kFreeBSD) provides ZFS utilities in the zfsutils package. Additionally, the Debian installer supports installing the operating system under ZFS on the amd64 architecture.

NetBSD[link]

The NetBSD ZFS port was started as a part of the 2007 Google Summer of Code and in August 2009 the code was merged into NetBSD's source tree.^[73]

Mac OS X[link]

The first indication of Apple Inc.'s interest in ZFS was an April 2006 post on the opensolaris.org zfs-discuss mailing list where an Apple employee mentioned being interested in porting ZFS to their Mac OS X operating system.^[74] In the release version of Mac OS X 10.5, ZFS was available in read-only mode from the command line, which lacks the possibility to create zpools or write to them.^[75] Before the 10.5 release, Apple released the "ZFS Beta Seed v1.1", which allowed read-write access and the creation of zpools,^[76] however the installer for the "ZFS Beta Seed v1.1" has been reported to only work on version 10.5.0, and has not been updated for version 10.5.1 and above.^[77] In August 2007, Apple opened a ZFS project on their Mac OS Forge web site. On that site, Apple provided the source code and binaries of their port of ZFS which includes read-write access, but there was no installer available^[78] until a third-party developer created one.^[79] In October 2009, Apple announced a shutdown of the ZFS project on Mac OS Forge. No explanation was given, just the following statement: "The ZFS project has been discontinued. The mailing list and repository will also be removed shortly." Versions of the previously released source and binaries, as well as the wiki, have been preserved and development has been adopted by a group of enthusiasts.^[80][81] Complete ZFS support was once advertised as a feature of Snow Leopard Server (Mac OS X Server 10.6). However, all references to this feature have been silently removed; it is no longer listed on the Snow Leopard Server features page.^[82] Apple has not commented regarding the omission.

The MacZFS project mirrored the public archives of the Apple ZFS before they disappeared,^[80] and has grown into a community-maintained project, producing releases of ZFS for most recent versions of OS X, including Lion (10.7).^[80] The project has an active mailing list. As of January 2012, MacZFS implements zpool version 8, the version from the October 2008 release of Solaris.

In January 2012, the company Ten's Complement LLC released a ZFS implementation for Mac OS X called "Zevo" which includes both GUI and command-line features. As of January 2012, Zevo implements zpool version 28, the same as FreeBSD 9.0. The first "Silver" release of Zevo does not include compression, snapshots, or RAIDZ, although it does enable checksumming. Future releases of Zevo, slated for release later in 2012, will offer additional features. The company was founded by Don Brady, a former Apple engineer who was technical lead on the original HFS+ team and worked on Apple's abandoned internal project to port ZFS.^[83]

Linux[link]

Porting ZFS to Linux is complicated by the fact that the GNU General Public License, which governs the Linux kernel, is incompatible with the Sun CDDL under which ZFS is distributed. According to some developers^[who?] a single derived work of both projects cannot be legally distributed, as it is not possible to simultaneously meet both licenses' requirements.^[84] To include ZFS in the Linux kernel it would have to be cleanly reimplemented, and patents may hamper this.^[85]

Linux FUSE[link]

Another solution to this problem was to port ZFS to Linux's FUSE system so the filesystem runs in userspace instead, where it is not considered a derived work of the kernel. A project to do this was sponsored by Google's Summer of Code program in 2006.^[86] The original ZFS on FUSE project is available here. Development for ZFS on FUSE/Linux now takes place at zfs-fuse.net.

Native ZFS on Linux[link]

A native port of ZFS for Linux is in development. This ZFS on Linux port was produced at the Lawrence Livermore National Laboratory (LLNL) under Contract No. DE-AC52-07NA27344 (Contract 44) between the U.S. Department of Energy (DOE) and Lawrence Livermore National Security, LLC (LLNS) for the operation of LLNL. It has been approved for release under LLNL-CODE-403049. The port is currently in release candidate status for version 0.6.0, which supports mounting filesystems.

Another native port was being worked on by KQ Infotech .^[87][88] This port used the LLNL ZVOL implementation as a starting point. A GA release supporting zpool v28 was released in January 2011.^[89] In mid-2011, KQ Infotech was acquired by another company, and as such their work on ZFS had ceased.^[90] Their code can be found on github.^[91]

The work of KQ Infotech seems to continue as another project ZFS on Linux , with the latest code available also on Github.

Comparisons[link]

List of Operating Systems, Distros and add-ons that support ZFS, the zpool version it supports, and the Solaris build they are based on (if any):

(updated 2011/11/26)

See also[link]

Free software portal

• Btrfs — for Linux
• Comparison of file systems
• Ext4
• HAMMER — a file system with a similar feature set for DragonFly BSD
• LFS — BSD Log Structured Filesystem
• List of file systems
• LVM — Logical Volume Manager (Linux), supports snapshots
• LZJB — data compression algorithm used in ZFS
• NILFS — a Linux filesystem with checksumming (but not scrubbing), also supporting snapshots
• Reiser4
• Sun Open Storage
• Veritas File System and Veritas Volume Manager — similar to ZFS
• Versioning file systems — List of versioning file systems
• Write Anywhere File Layout — a similar file system by NetApp

References[link]

Bibliography[link]

External links[link]

• ZFS Development Community and detailed ZFS Documentation
• ZFS Best Practices Guide
• ZFS The Last word in File Systems
• ZFS Boot Project
• ZFS Encryption Project
• Sun's ZFS Learning Center Presentations on ZFS by Bill Moore
• Comparison of SVM mirroring and ZFS mirroring
• EON ZFS Storage (NAS) distribution
• ZFS on Linux Homepage
• End-to-end Data Integrity for File Systems: A ZFS Case Study
• Storage systems and ZFS in FenixOS