5.2 Using tar to Perform Incremental Dumps

Incremental backup is a special form of GNU tar archive that stores additional metadata so that exact state of the file system can be restored when extracting the archive.

GNU tar currently offers two options for handling incremental backups: --listed-incremental=snapshot-file (-g snapshot-file) and --incremental (-G).

The option --listed-incremental instructs tar to operate on an incremental archive with additional metadata stored in a standalone file, called a snapshot file. The purpose of this file is to help determine which files have been changed, added or deleted since the last backup, so that the next incremental backup will contain only modified files. The name of the snapshot file is given as an argument to the option:

--listed-incremental=file

-g file

Handle incremental backups with snapshot data in file.

To create an incremental backup, you would use --listed-incremental together with --create (see create). For example:

     $ tar --create \
                --file=archive.1.tar \
                --listed-incremental=/var/log/usr.snar \
                /usr

This will create in archive.1.tar an incremental backup of the /usr file system, storing additional metadata in the file /var/log/usr.snar. If this file does not exist, it will be created. The created archive will then be a level 0 backup; please see the next section for more on backup levels.

Otherwise, if the file /var/log/usr.snar exists, it determines which files are modified. In this case only these files will be stored in the archive. Suppose, for example, that after running the above command, you delete file /usr/doc/old and create directory /usr/local/db with the following contents:

     $ ls /usr/local/db
     /usr/local/db/data
     /usr/local/db/index

Some time later you create another incremental backup. You will then see:

     $ tar --create \
                --file=archive.2.tar \
                --listed-incremental=/var/log/usr.snar \
                /usr
     tar: usr/local/db: Directory is new
     usr/local/db/
     usr/local/db/data
     usr/local/db/index

The created archive archive.2.tar will contain only these three members. This archive is called a level 1 backup. Notice that /var/log/usr.snar will be updated with the new data, so if you plan to create more ‘level 1’ backups, it is necessary to create a working copy of the snapshot file before running tar. The above example will then be modified as follows:

     $ cp /var/log/usr.snar /var/log/usr.snar-1
     $ tar --create \
                --file=archive.2.tar \
                --listed-incremental=/var/log/usr.snar-1 \
                /usr

Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file's time stamps during dumping (e.g., with the --atime-preserve=replace option), or if you set the clock backwards.

Metadata stored in snapshot files include device numbers, which, obviously is supposed to be a non-volatile value. However, it turns out that NFS devices have undependable values when an automounter gets in the picture. This can lead to a great deal of spurious redumping in incremental dumps, so it is somewhat useless to compare two NFS devices numbers over time. The solution implemented currently is to considers all NFS devices as being equal when it comes to comparing directories; this is fairly gross, but there does not seem to be a better way to go.

Note that incremental archives use tar extensions and may not be readable by non-GNU versions of the tar program.

To extract from the incremental dumps, use --listed-incremental together with --extract option (see extracting files). In this case, tar does not need to access snapshot file, since all the data necessary for extraction are stored in the archive itself. So, when extracting, you can give whatever argument to --listed-incremental, the usual practice is to use --listed-incremental=/dev/null. Alternatively, you can use --incremental, which needs no arguments. In general, --incremental (-G) can be used as a shortcut for --listed-incremental when listing or extracting incremental backups (for more information, regarding this option, see incremental-op).

When extracting from the incremental backup GNU tar attempts to restore the exact state the file system had when the archive was created. In particular, it will delete those files in the file system that did not exist in their directories when the archive was created. If you have created several levels of incremental files, then in order to restore the exact contents the file system had when the last level was created, you will need to restore from all backups in turn. Continuing our example, to restore the state of /usr file system, one would do¹:

     $ tar --extract \
                --listed-incremental=/dev/null \
                --file archive.1.tar
     $ tar --extract \
                --listed-incremental=/dev/null \
                --file archive.2.tar

To list the contents of an incremental archive, use --list (see list), as usual. To obtain more information about the archive, use --listed-incremental or --incremental combined with two --verbose options²:

     tar --list --incremental --verbose --verbose archive.tar

This command will print, for each directory in the archive, the list of files in that directory at the time the archive was created. This information is put out in a format which is both human-readable and unambiguous for a program: each file name is printed as

     x file

where x is a letter describing the status of the file: ‘Y’ if the file is present in the archive, ‘N’ if the file is not included in the archive, or a ‘D’ if the file is a directory (and is included in the archive). Each such line is terminated by a newline character. The last line is followed by an additional newline to indicate the end of the data.

The option --incremental (-G) gives the same behavior as --listed-incremental when used with --list and --extract options. When used with --create option, it creates an incremental archive without creating snapshot file. Thus, it is impossible to create several levels of incremental backups with --incremental option.