A brief history of OpenZFS and TrueNAS CORE®
In the first part of our series about FreeNAS, we give a historical overview and show you the advantages of the attractive storage operating system.
ZFS
In these articles we try to give a brief introduction to OpenZFS, and to TrueNAS CORE®, the most popular solution built around it – by iXsystems. We aim at giving you some concentrated and easy to grasp information on how OpenZFS and TrueNAS work under the hood, how to build your TrueNAS server to take advantage of OpenZFS potentials, and the most common pitfalls you should avoid. To escape making these articles lengthy, we embed the links to many of the references in the technical terms we use. Feel free to click and explore those links for more expansive articles and descriptions. Let’s begin…
Creation of ZFS
The Zettabyte Filesystem (ZFS) is a scalable file system and logical volume manager that was created in 2001 by Matthew Ahrens and Jeff Bonwick while working for Sun Microsystems to be used in their Solaris operating system. Its core goal was to overcome the limitations of existing filesystems, such as the ability to host a large number of files, the maximum storage capacity, the inability to span multiple disks without additional hardware, etc.
ZFS was designed to be a scalable filesystem with plenty of advanced features like:
- storage pooling
- extensive protection against data corruption
- support for high storage capacities
- support for file size of up to 16 Exabyte
- support for 256 Quadrillion Zettabytes pool size
- efficient in-line data compression
- copy-on-write and snapshots
- continuous integrity checking and automatic repair
- native NFSv4 ACLs
- RAID-Z which provides RAID-like abilities and complements the volume management capabilities of the filesystem.
Berth of OpenZFS and ZFS on Linux
Initially, ZFS was closed-source, but in 2005 Sun Microsystems opened the source of its Solaris OS (which included their ZFS code-base) under a GPL license and the OpenSolaris OS was born.
A few years later Oracle bough Sun and closed the source again. But by that time the source code has already been open for a good while. Developers quickly forked OpenSolaris and the illumos OS was created to replace it. The ZFS portion of the code was successfully ported to multiple open-source Unix-based OS platforms such as FreeBSD and macOS.
In 2013 a group of developers initiated the OpenZFS project to help coordinate development efforts and avoid code fragmentation. Their initiative made ZFS widely available within Unix-like systems especially FreeBSD whose developers maintained the most stable and feature-rich branch of the code, which in turn lead to OpenZFS being referred to as ZFS on FreeBSD (ZoF).
Because they originated from the same code-base, both OpenZFS and the original ZFS are extremely similar. Due to this similarity, OpenZFS is usually referred to as ZFS. Also, the proprietary version of ZFS currently exists only in the Solaris OS. All other platforms are using OpenZFS or ZFS on Linux (ZoL), which is the Linux port of OpenZFS that was first released in 2013. This is also why, within open-source enthusiasts, both ZoF and ZoL are usually referred to as ZFS.
OpenZFS and ZFS on Linux merging
ZoF remained, for a good while, the leading open-source branch of ZFS. But the increasing popularity of Linux and its relatively larger user and developer base eventually lead to ZoL catching up with ZoF. ZoF developers, on the other hand were slower to import newer features into their release because they were focused on stability of the filesystem. This difference in development paces quickly lead to ZoL being the superior branch when it comes to feature-richness. In 2019 ZoL was available for use in many leading Linux distros and in mid 2020 Ubuntu – the most popular Linux distribution – released version 20.04 LTS with support for ZFS as the root filesystem for the OS.
In December 2020, and with the release of OpenZFS V2.0.0, the maintainers of ZoF announced they are merging with ZoL, ending years of distinction between ZoL and ZoF and everywhere else.
This was actually great news because by that time ZoL was the more advanced branch with plenty of features that ZoF was lacking such as Persistent L2ARC and Sequential Resilvering.
FreeNAS
The FreeNAS project was started in 2005 by iXsystems. It was initially based on a micro-distribution of Linux. In 2010 the developers decided to rebase their solution on FreeBSD to benefit from its stability and the more advanced ZFS code within.
In spite of all the cool features of ZFS, it was still quite challenging to use because it was 100% Commandline-based. This led to the majority of its users being either experienced Unix admins or tech savvy enthusiasts. Those with no Unix shell administration skills were left out. It was a very unfortunate situation.
The other negative thing was that ZFS was just a filesystem. To be able to connect a client to a ZFS server to store data on it you still needed to configure and enable some common sharing protocol, such as NFS or SMB, on that server to turn it into a Network Attached Storage (NAS). This was also quite challenging, especially with ZFS being adopted by the much less popular distributions, OpenSolaris and the likes.
FreeNAS was created to bridge this gab and provide a stable ZFS open-source NAS-focused solution with a user-friendly Web GUI. iXsystems used to produce two versions of their popular NAS solution with TrueNAS being the enterprise edition (paid for) and FreeNAS being the open-source (free) version. This meant that both versions used to have separate code-bases and also different installation ISO files.
TrueNAS CORE – Where did FreeNAS go?
In March 2020 iXsystems announced they will be consolidating both FreeNAS and TrueNAS into a single software image and name – TrueNAS.
There are two versions of TrueNAS:
- TrueNAS CORE: free to use with unrestricted feature-set and no capacity limit.
- TrueNAS Enterprise: requires a license to enable advanced features such as high-availability.
So, TrueNAS CORE is the new name for FreeNAS. It the same software that has been tried and tested by hundreds of thousands of users and has won the trust of many enterprise IT professionals. When properly configured, TrueNAS can provide a robust and secure storage backend to many applications, ranging between SOHO and multi-terabyte enterprise pools. With TrueNAS you get the latest iteration of OpenZFS which include the latest file-system features, updates and patches.
TrueNAS supports Windows, macOS and Unix clients and various virtualisation hosts such as XenServer and VMware using the SMB, AFP, NFS, iSCSI, SSH, rsync and FTP/TFTP protocols. Advanced TrueNAS features include full-disk encryption and a plug-in architecture for third-party software.
What does this mean?
It simply means that, for all new comers to ZFS on TrueNAS CORE, you will only hear about and have to deal with one unified and well-maintained variant of ZFS – which is the OpenZFS one. You will also get to enjoy all the latest features, performance and security enhancements of ZFS rather quickly and won’t have to wait for this team or that team of developers to adopt them. Lucky you!
Reasons for TrueNAS CORE
Block and File Sharing
Is what TrueNAS does best. It supports all major operating system with SMB/CIFS (Windows file shares), NFS (Unix file shares) and AFP (Apple File Shares) as well as FTP, iSCSI (block sharing), and WebDAV as methods of sharing data over the network. The iSCSI implementation supports VMware VAAI, Microsoft ODX and Microsoft Windows Server 2008 and 2012 R2 Clustering.
Windows, Mac OS X, many Linux distributions, and PC-BSD® can connect using SMB shares with little or no additional configuration needed on the client side. Most Unix-like operating systems support connecting with NFS out of the box, and free clients are widely available. AFP is primarily used by Mac OSX and is well suited for a network environment that only connects with Macintosh clients. TrueNAS® also supports Time Machine backups.
Data Protection
ZFS is designed around data integrity from top to bottom. RAID-Z, the software RAID that is part of ZFS, offers single parity protection like RAID 5, but without the notorious “write hole” vulnerability, thanks to the copy-on-write architecture of ZFS. The additional levels RAID-Z2 and RAID-Z3 offer double and triple parity protection, respectively. A software mirror option is also available. The TrueNAS’ Pool Manager screen lists each possible parity arrangement based on the number of disks you select when creating a new volume.
Every ZFS filesystem is also verified with checksums from top to bottom to ensure data integrity. If inconsistencies are found, parity blocks can be used to repair corrupt data.
A regular scrub is turned on by default and can be rescheduled or configured from the web interface.
Snapshots
Thanks to ZFS, snapshots of the entire filesystem can be made and saved at any time. As long as a snapshot exists, administrators can restore files, and entire datasets, to where they were when the snapshot was taken.
Snapshots can be taken on a one-off basis or scheduled to run at specific intervals from the web interface. At any time, the entire filesystem can be rolled back to the most recent snapshot. Older snapshots can be cloned and accessed to recover data from that version of the filesystem. From the web interface, users can see how much space a particular snapshot is occupying on the volume and delete, clone, or roll back to individual snapshots as needed.
Replication
ZFS Snapshots are more than just local backups – they can be used to create remote backups as well. Replicating snapshots of the filesystem to a remote ZFS filesystem creates a complete duplicate there. Furthermore, additional snapshots of the same filesystem can be sent incrementally, reducing the size of each backup to the changes that were made between snapshots. In case of catastrophic damage to a local ZFS filesystem (such as disk failure in excess of parity protection or irrecoverable log device failure), any backed-up snapshot can be sent to a new ZFS filesystem, recovering all data up to that backup.
Encryption
TrueNAS is the first and only open-source project to offer encryption on ZFS volumes! A full-volume encryption option is available during volume creation, providing industry standard AES-XTS encryption which can be hardware-accelerated (when the processor has AES-NI capability).
Encrypted volumes can only be read by TrueNAS systems in possession of the master key for that volume. The user can optionally create a passphrase to add extra protection for their system against loss or theft.
Encryption allows for confidence when retiring and recycling hard drives because the drives no longer need to be wiped provided the master keys are obliterated.
Web Interface
TrueNAS’ goal is to simplify complex administrative tasks for its users. Every aspect of a TrueNAS system can be managed from its Web User Interface. A setup Wizard further simplifies configuration at installation time or later in the setup process. Dataset and Volume creation, iSCSI LUN creation, setting permissions on individual shares, or performing software updates – all can be done without missing a critical step or encountering a silent failure.
All services are configurable and many of them have advanced options that can be modified by clicking the Edit > Advanced buttons.
In addition to the Web Interface, the full power of the FreeBSD shell environment is also available from the GUI and over SSH. Ultimately, TrueNAS makes NAS deployment easier than ever but doesn’t get between you and the solution you need.
The TrueNAS GUI. A streamlined management cockpit for your ZFS storage server.
