ZFS Space Accounting
ZFS is based on a concept of pooled storage. Unlike typical file
systems, which are mapped to physical storage, all ZFS file systems in a
pool share the available storage in the pool. So, the available space reported
by utilities such as df might change even when the file system is
inactive, as other file systems in the pool consume or release space. Note
that the maximum file system size can be limited by using quotas. For
information about quotas, see Setting Quotas on ZFS File Systems. Space can be guaranteed to a file system by
using reservations. For information about reservations, see Setting Reservations on ZFS File Systems. This model is very similar
to the NFS model, where multiple directories are mounted from the same file
system (consider /home).
All metadata in ZFS is allocated dynamically. Most other file systems pre-allocate much
of their metadata. As a result, an immediate space cost at file system
creation for this metadata is required. This behavior also means that the total
number of files supported by the file systems is predetermined. Because ZFS allocates
its metadata as it needs it, no initial space cost is required, and
the number of files is limited only by the available space. The output
from the df -g command must be interpreted differently for ZFS than other file
systems. The total files reported is only an estimate based on the amount of
storage that is available in the pool.
ZFS is a transactional file system. Most file system modifications are bundled into
transaction groups and committed to disk asynchronously. Until these modifications are committed to
disk, they are termed pending changes. The amount of space used, available, and
referenced by a file or file system does not consider pending changes. Pending
changes are generally accounted for within a few seconds. Even committing a change
to disk by using fsync(3c) or O_SYNC does not necessarily guarantee that the space
usage information is updated immediately.
Out of Space Behavior
File system snapshots are inexpensive and easy to create in ZFS. Most likely,
snapshots will be common in most ZFS environments. For information about ZFS snapshots,
see Chapter 6, Working With ZFS Snapshots and Clones.
The presence of snapshots can cause some unexpected behavior when you attempt to
free space. Typically, given appropriate permissions, you can remove a file from a
full file system, and this action results in more space becoming available in
the file system. However, if the file to be removed exists in a
snapshot of the file system, then no space is gained from the
file deletion. The blocks used by the file continue to be referenced from
As a result, the file deletion can consume more disk space, because a
new version of the directory needs to be created to reflect the
new state of the namespace. This behavior means that you can get an
unexpected ENOSPC or EDQUOT when attempting to remove a file.