NAME¶

mbxz - Compress a file in parallel with MPI-Bash

SYNOPSIS¶

mbxz [-k] [-f] [-S .suf] [-C check] [-0 ... -9] [-e] [-M limit] [-q] [-v] [-Q] input_file...

DESCRIPTION¶

mbxz is a parallel version of the standard xz command, designed to compress large files by distributing the work across a set of computers that share a filesystem.

mbxz is a bash script that uses MPI-Bash functions for communication. It should therefore be run like any other MPI program (typically with mpirun or mpiexec).

OPTIONS¶

mbxz accepts many of xz's options:

-k: Don't delete the input file(s).
-f: If the target file already exists, delete it before compressing.
-S .suf: When compressing, use .suf as the suffix for the target file instead of .xz.
-C check: Specify the type of the integrity check.
-0 ... -9: Select a compression preset level. The default is -6.
-e: Use a slower variant of the selected compression preset level (-0 ... -9) to hopefully get a little bit better compression ratio, but with bad luck this can also make it worse. Compressor memory usage increases a little at preset levels -0 ... -3.
-M limit: Set a memory usage limit for compression.
-q: Suppress warnings and notices. Specify this twice to suppress errors, too.
-v: Be verbose. Specifying -v twice will give even more verbose output.
-Q: Don't set the exit status to 2 even if a condition worth a warning was detected. This option doesn't affect the verbosity level, thus both -q and -Q have to be used to not display warnings and to not alter the exit status.

See the xz manual page for further explanations of many of those options.

At least one source file must be specified on the command line.

Apart from supporting fewer options, mbxz differs from xz in that it can only compress, not decompress, files.

EXAMPLES¶

Use 128 processes to compress a large file:

mpirun -np 128 mbxz -v /scratch/me/my_gigantic_file

BUGS¶

I sometimes see segmentation faults when MPI-Bash programs exit. I don't know why.

AUTHOR¶

Scott Pakin, pakin@lanl.gov