NAME¶

btar - tar-compatible archiver with extra features based on tar double-achiving

SYNOPSIS¶

btar [actions] [options] [files...]

Actions: [-cxTlLmh]

Options: [-HNRvXYV] [-b <blocksize>] [-d <file>] [-D <file|->] [-f <file>] [-F <filter>] [-j <n>] [-X <pattern>] [-G <defilter>]

DESCRIPTION¶

btar archives trees of files into a single file, using internally any desired amount of filters (programs that transform data from stdin to stdout, like gzip or ccrypt for example). The filters are applied to big fixed-size blocks of usual tar data, and they will include an index to allow later dearchiving without having to decode the whole archive (uncompressing, unciphering, etc.). This allows nice features like quick extraction of partial contents of the archive, and parallelization of compression, for example.

btar also allows for creation of differential archives, based on the index of a btar archive already created.

It is similar to tar in the sense that it can operate creating a btar archive to standard output, or unpack an archive from standard input (unseekable sources or destinations). And it can also interoperate with tar, in the sense that it can create a btar archive based on tar output, instead of btar archiving the files. As btar does not support as much filesystem features as GNU tar, it may be interesting for users to have the btar features while respecting GNU tar archiving features, as btar can also create a file index based on a GNU tar archive. Additionally, btar archives can be extracted without the help of btar at all, only using tar.

btar can create multiple blocks at once, running their filters in parallel. This means that it can easily use multiprocessor systems at the time of creating the archive. At the current version, this is only possible archiving, not dearchiving.

ACTIONS¶

Running btar with no options will make btar act as a filter, simply splitting the input from stdin into blocks, applying the chosen filters, and joining them with tar to form a btar archive. This allows creating btar archives from any kind of input, and for tar input it can also index it, like in this example: tar cp /home | btar -F gzip

-c

Create a btar archive from filesystem directories, similar to tar -c

Then, files or directories added to the btar command will be recursively traversed for inclusion into the btar archive. -X allows for patterns of exclusion.

-x

Extract btar archive files to the filesystem, similar to tar -x

Then, files or directories added to the btar command will be extracted from the btar archive, based on fnmatch(3) without flags.

The defilters for the file can be speficied with -G or they will be guessed calling the original filter programs with a parameter -d (usual in compressors, ccrypt, etc.).

-T

Extract the internal tar archive of btar. For example, it can be used with GNU tar for detailed listing this way: btar -x | tar tv

The defilters will be called as explained in -x.

-l

Output a simple list of the btar internal index.

The defilters will be called as explained in -x.

-L

Output the internal btar index tar file to stdout. It can be used, for exmaple, to get its detailed listing this way: btar -L | tar tv

-m

Take an input btar from stdin, and rebuild a new btar according to the block size (-b), new filters (-F), or add redundancy block (-R).

The data inside each block is not manipulated, and the index and deleted files data as well. In case the input btar did not have index or deleted files, the new btar will have them, but at zero-length.

For extraction of the input btar, defilters will be called as explained in -x.

OPTIONS¶

-b <blocksize>

By default btar splits the input (its internal tar archive in case of -c) into blocks of 10MiB, but this can be overriden with this parameter, that expects the number of mebibytes per block. For example it can be interesting to use bigger block sizes with the xz compressor, as it uses very big compression windows.

-d <file>

Base the creation of an archive (only using -c) on the index inside the btar file given to the parameter. It can be used several times, to create level1, level2, ... archives, and combined with -d too.

The defilters will be called as explained in -x.

-D <file|->

Base the creation of an archive (only using -c) on the tar-format btar index file given to the parameter. It can be used several times, to create level1, level2, ... archives, and combined with -d too. It accepts reading from stdin too.

This can be useful to make a remote backup, in a command like this: btar -L -f base.btar | ssh remote btar -D - -c /home | btar -T | btar -F xz > diff.btar

This above will take the index from base.btar and use it to create a differential backup in a remote host, while we apply the xz filter locally to get the final differential dense btar archive.

-f <file>

In case of creating an archive (this is either without action, or with -c), this determines the output btar file instead of the default stdout.

In case of reading from an archive (rest of actions), this determines the input btar file instead of the default stdin. In this case it can be specified multiple times, and this is useful in case of extracting a base and its differential archives of different levels.

-F <filter>

For every block created (either filtering or with -c), filter the block calling this given filter program, giving the block to it through stdin, and storing its output into the created btar archive.

It can be specified multiple times, and the filters will pass data one to the other in the order given in the command line.

In case of requiring to pass parameters to the filter, the filter can contain spaces, and the filter command executed will have the arguments splitted by those spaces. For a filter argument that requires spaces in it, the user can write a wrapper script, and then pass the script name to -F.

The word before the first space will be used as the filter name, stored inside the btar archive for each filter, for later automatic defiltering at extraction time, not requiring -G parameters.

-G <defilter> If btar has to extract blocks from a btar archive (either extracting its files, or making a differential archive), specify the commands (analogous to -F) that will recover every block to its original data.

As btar archives store the first word of the filter specified by -F at the time of btar archive creation, btar will call each of those filters with the parameter -d in case of not specifying any defilter with -G. This allows not requiring any -G parameter for usual programs like gzip or ccrypt.

-H

Differential archives keep a list of removed files and directories from the filesystem. Extracting with -H makes btar remove those files and directories; otherwise, the list of removed files and directories is ignored at extraction.

-j <n>

Number of blocks to filter in parallel at the time of creating an archive (either filtering or with -c).

-N

In case of creating an archive by btar filtering, do not create an index of the files seen in standard input.

The index may be useful only if the input comes from GNU tar.

-R

In case of creating a btar archive, add a block that will be the XOR of the rest of the blocks. This adds some redundancy to the archive, that can allow recovering the full archive if some of its contents have been damaged.

-v

Output the file names processed to stderr, in -c and -x.

-V

Add verbose output of the btar internal actions performed. Can be specified multiple times to increase the verbosity.

-X

At the time of creating an archive, exclude files from inclusion into the archive, based on fnmatch(3) without flags.

Ignored at the time of extracting from an archive.

-Y

If btar has been built with librsync support, this indicates that differential btar archives will not include full changed files, but only the changed parts of the files based on the rdiff algorithm.

INTERNAL FORMAT¶

The btar archive format is compatible with tar. Tar will see there block files, named according to the filters used. Defiltering every block, and joining all the blocks, creates back a normal tar archive (that's the operation done by -T).

The btar archive may contain, additionalto the archive blocks, the index file and a list of files deleted (in case of a differential archive).

Therefore, a btar archive can be uncompressed without having the btar program. An archive created with "-F gzip" can be extracted with: (for a in `tar tf file.btar` | grep ^block`; do tar xf file.btar -O $a | gunzip; done) | tar x

SEE ALSO¶

tar(1)

AUTHOR¶

Lluis Batlle i Rossell

NOTES¶

This page describes btar as in version 1.0.0. Other versions may differ.