SNEBU

You will need to set up user permissions for any user ID that access the snebu binary. To set up permissions for the root user, run the following commands:

If a user other than root needs to run snebu, then they need to be added to the snebu group.

sudo is needed here to allow snebu-client to read all the files on the host. If a user is only backing up their own directory then sudo isn’t needed, and they can be given more granular permissions to the snebu command — see the man page snebu-permissions(1).

This backs up all mounted disk-based file systems (automatically skips tmpfs, procfs, sysfs, and other "nodev" mount points). The backup name defaults to the host name of the client, however this can be overridden with --name parameter. You can also follow the command with a file or path list. For example:

The first command gives a list of all hosts that have been backed up. With the "-v" flag, it will also give all backup sets that are part of each backed up host. Backup sets are identified by a serial number, which is the time/date that the backup was created, represented in Unix time_t format (i.e., the number of seconds since Jan 1, 1970).

The second command will list the files that are part of the host and backup set, restricting the output to the given file specification.

In the third command, a restore of this backup set is initiated. The client changes to the "/tmp" directory, so everything restored is relative to this directory (or specify -C / to restore to the original location). The --graft parameter is specified to re-write part of the file path — in this case it replaces the directory "/home/bigboss/" with "bigboss-restored". Putting it together the final path file that gets restored is in /tmp/bigobss-restored/BudgetProposal2021.doc.

Notice that the backup above is a "daily" backup — this is the retention schedule that this backup set is assigned to. By default, backups ran on Sunday through Friaday are daily backups, Saturday is a weekly backup, and the first of the month is a monthly backup.

If snebu is installed in a remote backup server called bkupsvr1, and you have the snebu-client script on a local host, you can add the parameters --backup-server bkupsvr1 and --backup-user svc-bosshost1 to the above commands. Make sure to create the service user account svc-bosshost1 on the remote host (or whichever user account name specified by your organizations practices), along with adding the account to the snebu group. Set up ssh key based authentication for unattended backups, and then create the appropriate permissions for this service user on the remote backup server:

Now you can back up to this host:

Note, you may wish to grant more granular permissions such as "backup", "listbackups" and "restore" in the above snebu permissions command. This would prevent the client from deleting backups on the backup server if it were to become compromised. See the snebu-permissions man page for detailed command usage.

If you want more protection, you can reverse the process and have the remote backup server "pull" a backup from the client:

This will access bosshost1 as the service user svs-backup, then sudo to root to pull the data. Make sure to set up ssh key authentication between snebu@bkupsvr1 and svs-backup@bosshost1. If you leave off the --sudo flag, then the user root will be directly accessed via ssh (requiring ssh key authorization to root@bosshost1)

Run the following on the backup server to expire old backups

This expires all daily backups older than 2 weeks, weekly backups older than 6 weeks, and monthly backups older than a year. Expiring a backup only removes the metadata, and takes a short amount of time. A purge permanently remove data from the vault, and can take a bit longer (depending on the number of files that need to be removed).

In the above example, the commands were run under the user ID snebu, which owns the repository and has all permissions. Again, you can grant a specific user permission to run the expire and purge commands to limit the need to access the main user account (see snebu-permissions(1) documentation).

Snebu supports client-side public key encryption. This requires the program tarcrypt to be installed on the client. On the client, run the command tarcrypt genkey -f outputfile, and make sure it has appropriate permissions and ownership

You will be prompted for a passphrase to protect the private key stored in the .key file. Then, on any of the backup command variations, add the parameter --encryption-key /etc/snebu-backup.key:

Note, you can repeate the --encryption-key parameter to encrypt with multiple keys — in this case, the passphrase for any one of the keys can be used to decode the backup upon restoring.

When restoring an encrypted backup, specify snebu-client restore --decrypt along with the other parameters as appropriate. No key file is specified, as all key data is securely stored with the backup. You will be prompted for the private key passphrase for one of the keys when restoring.

snebu-client is the client front end for snebu. Use it to easily back up a local or remote host, to either local a local storage device, or to a remote backup server. Use it with one of the following subcommands.

snebu-client-backup(1), snebu-client-restore(1), snebu-client-listbackups(1), snebu-client-validate(1),

Initiates a system backup. By default, it will back up the local host to a local snebu install. You can also use this command to back up to a remote backup server, back up a remote host to either a local snebu installation, or back up a remote host to another remote backup server, depending on which options are chosen.

tarcrypt(1)

With no arguments specified, "listbackups" will return a list of all systems that are contained in the backup catalog. Otherwise, when specifying the -n parameter, a list of backup sets for that host is returned.

Compares the contents a given backup session identified by "-n" and "-d" parameters, to what is on the client. Use the "listbackups" subcommand to get a list of available backup sessions.

tarcrypt(1)

Restores a given backup session identified by "-n" and "-d" parameters. Use the "listbackups" subcommand to get a list of available backup sessions.

tarcrypt(1)

The sneub-client.conf is used to specify a number of default parameters for snebu-client, such as default include / exclude specifications. The contents are read in and executed as shell script commands by snebu-client, so in addition to specifying parameters it is possible to include standard shell scripting logic. This also means that parameters are specified without a space between the name and value.

If running backups from a backup server, the parameters will by default apply to all clients. To target parameters for specific clients, you can wrap them in a shell scripting conditional clause (if-then-else, or case statement).

To include specified directories:

To exclude /tmp and /var/tmp

To exclude all ".tmp" and ".dbf" files

On a server backing up multiple clients — to exclude all database ".dbf" files only on the database server "erp-database", include the following:

Specifying the --plugin argument when executing a snebu-client backup operation will cause the specified plugin script to be incorporated into the backup process. The plugin script defines at least two shell script functions — -fBpluginpre(), which executes prior to the backup, and pluginpost()\R which executes after the backup has completed.

This allows performing operations such as mounting a file system snapshot prior to a backup and removing the snapshot afterwards, or placing a database in hot backup mode at the beginning of the backup. In the case of backing up a database, often times the backup will need to be completed in multiple stages — place the DB in hot backup mode, gather a list of database files, back them up, take the DB out of hot backup mode, gather a list of files containing transaction logs that were created during the backup, and finally backing up those files.

To facilitate these type of backup steps, the pluginpre function can save the contents of the file include/exclude variables INCLUDE, EXCLUDE, and EXCLUDEMATCH. Then set the variable bkrepeat=1, along with setting any other housekeeping variables used inside the plugin script. Once it has completed the last stage, it can then restore the include/exclude variables, let the rest of the backup process continue, and then set bkrepeat=0 to finish off the backup.

The following is a template that can be used when backing up a server containing a database. In this case, the include/exclude list initially includes files to back up the entire server. So this script does a hot backup of the database first, then adds to the exclude list the dbf files that were backed up initially, and then backs up the rest of the system.

The Since these functions get called at various times from the snebu-client script, the order of the various code fragments can become a bit confusing. Pay attention to the "(Step x)" labels for the actual execution order in each fragment.

The snebu command is a backup tool which manages storing data from backup sessions on disk-based storage, utilizing a simple database for tracking backup sets and meta data. With the exception of administrative sub commands (expire, purge, permissions), it is typically it is called via a front end script (such as the included "snebu-client" shell script). The subcommands are listed below along with the most common options. Details on each command are given in each command’s individual man page.

snebu-newbackup(1), snebu-submitfiles(1), snebu-restore(1), snebu-listbackups(1), snebu-expire(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

The "newbackup" command creates a new backup set, by consuming a tab-delimited list of file names (along with associated meta data) to include in the backup. It then compares this list to the backup catalog database to determine which files are new, and which ones are already contained on the backup media. A list of new / changed files is returned (the snapshot manifest), which can then be passed along to "tar" to generate the input for the "submitfiles" subcommand.

The input manifest contains a list of files to include in this backup set. The format is a delimited list of file names and file metadata, with the following fields: .BP

Field 14 (Link Target) is only present if the file type is "l" (symbolic link).

The fields are tab-delimited. If the "--null" option is specified, then each line is null terminated, with an additional null character delimiting fields 13 and 14. Otherwise if "--not-null" is specified, fields 13 and 14 are tab delimited, and the path names must have special characters escaped.

The input manifest can be created with the GNU find command, with the following print formatting specification (suitable for the "--null" flag):

find [ parameters ] \( -type f -o -type d \ .br -printf "%y\t%#m\t%D\t%i\t%u\t%U\t%g\t%G\t%s\t0\t%C@\t%T@\t%p\0" .br -o -type l -printf "%y\t%#m\t%D\t%i\t%u\t%U\t%g\t%G\t%s\t0\t%C@\t%T@\t%p\0%l\0" .EE

The manifest returned is either a null-delimited list of files (if "--null-output" is specified), or a newline-delimited list of files with special characters escaped (if "--not-null-output is specified). This is the list of files that are required to complete the snapshot (any file that hasn’t changed from previous backups will be referenced from the backup server). This list is suitable for passing into the tar command.

snebu(1), snebu-submitfiles(1), snebu-restore(1), snebu-listbackups(1), snebu-expire(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

The "submitfiles" sub command is called after running snebu newbackup, and is used to submit a tar file containing the files from the snapshot manifest returned by newbackup.

snebu(1), snebu-newbackup(1), snebu-restore(1), snebu-listbackups(1), snebu-expire(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

With no arguments specified, listbackups will return a list of all systems that are contained in the backup catalog. Otherwise, when specifying the -n parameter, a list of backup sets for that host is returned.

snebu(1), snebu-newbackup(1), snebu-submitfiles(1), snebu-restore(1), snebu-expire(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

Generates a tar file containing files from a given backup set. Pipe the output of this command into tar to restore files.

snebu(1), snebu-newbackup(1), snebu-submitfiles(1), snebu-listbackups(1), snebu-expire(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

Removes backup sessions from the snebu backup catalog database. A specific backup session can be purged by providing the -n and -d options, or all backups that are part of a given retention schedule (specified with -r, and optionally from a given host, with the -n option) that are older than a given number of days (-a) are removed.

snebu(1), snebu-newbackup(1), snebu-submitfiles(1), snebu-restore(1), snebu-listbackups(1), snebu-purge(1), snebu-permissions(1), snebu-client(1)

Permanently removes files from disk storage that are no longer referenced by any backups. Run this command after running "snebu expire".

snebu(1), snebu-newbackup(1), snebu-submitfiles(1), snebu-restore(1), snebu-listbackups(1), snebu-expire(1), snebu-permissions(1), snebu-client(1)

Available subcomands that work with permissions are:

backup (covers both newbackup and submitfiles functions)

restore

listbackups

expire

purge

permissions

Note that in the case of functions that aren’t host specific (such as permissions) or affect all hosts (snebu purge, or snebu expire -a …​), users will need to be granted permission to all hosts by specifying -h '*' in order to be granted access to those specific functions).

To grant permissions, this command must be run as the user that snebu is installed under, or the user must be granted access to the permissions sub command

snebu(1), snebu-newbackup(1), snebu-submitfiles(1), snebu-restore(1), snebu-listbackups(1), snebu-expire(1), snebu-purge(1), snebu-client(1)

The tarcrypt command acts as a filter for the tar command. In encryption mode, it will compress and encrypt the data portion of TAR files while leaving header metadata intact. This allows the TAR file to be sent to a backup system that expects the Unix TAR format as input.

The key file that is generated by the genkey function contains an encrypted (passphrase protected) RSA private key, public key, and HMAC key used for verification. The encrypted private key, public key, and a hash of the HMAC key are stored in the TAR file’s global header.

Each individual file is encrypted with a random AES-256 key, which is in turn encrypted using the RSA public key. The file contents is also signed using the HMAC key, and the signature for the file is attached to the file header.

During a decrypt operation, the user is prompted for the passphrase protecting the private key, which is used to decrypt the AES key stored with each file. The passphrase, along with the public key hash, are used to re-create the secret HMAC key in order to validate each file contents as it is begin decrypted.

Multiple key files may be specified by repeating this parameter.

Public Key cryptography splits up a key into two parts — an encryption (public) key, and a decryption (private) key. The public key isn’t considered sensitive — it is designed so that one party can send encrypted data to another party, by using the other party’s public key. The private key, however, needs to be kept confidential by the receiving party. Typically the private key is stored encrypted with a symmetric algorithm (one where the same password is used to both encrypt and decrypt the data) to provide additional security.

In the case of tarcrypt, the sending and receiving party is the same entity, using a third party (the backup server) to store the encrypted data. If the backup is needed, chances are that the keyfile used to encrypt the backup is lost also. Therefore, both the public key (for reference), and an encrypted copy of the private key are stored in the generated encrypted tar file’s global header.

For the purpose of file confidentiality the keyfile isn’t considered sensitive, as the private key used to decrypt the data is stored in encrypted form. However, the keyfile also contains a secret HMAC key used to authenticate the contents of the encrypted tar file. Therefore if authenticity is needed, then the keyfile must be kept confidential (i.e., stored with appropriate file system permissions to be accessed only by the backup process).

The tarcrypt file format is an extension of the PAX TAR format, with custom values in the PAX header. If used in conjunction with a backup tool which expects the TAR format as input, then the backup tool may need some modifications in order to handle the extensions. If the tool stores the tar file intact, and if it doesn’t choke on the custom header fields, then no modifications should be necessary. However if the backup system parses out the TAR file format (i.e., it if uses it as a serialization format), then it would need to be modified to store the encrypted header info along with the rest of the metadata, and re-generate the appropriate global and individual file headers.