MULTITHREADED FILE COMPRESSION WITH ARCHIVER ON LINUX
Fast and reliable archiving:
tar --use-compress-program=pigz -cf base.tar.gz base.sql
Extract everything to the /target/dir directory using the multithreaded archiver unpigz:
cat /path/to/archives/backupname.tar.gz.* | tar --use-compress-program "unpigz" -x -C /target/dirPreamble
Not everyone knows that there are analogs of gzip and gzip2 with SMP support that generate archives fully compatible with standard archivers. These amazing archivers are called pigz, pbzip2, and lbzip2 (the last two are implementations of bzip2; I think it's better to use lbzip2, as this version is mentioned in the tar documentation).
Installation:
yum install pigz pbzip2 lbzip2
Usage:
Packaging:
tar cf /directory | pigz name.tar.gz
Extraction
cat name.tar.gz | unpigz - | tar xf -
Set pigz as the default for tar:
Add the following line to the /etc/bashrc file:
alias tarz='tar --use-compress-program=pigz'
echo "alias tarz='tar --use-compress-program=pigz'" >> /etc/bashrc
or:
alias tarz='tar --use-compress-program=pbzip2'
After that, use the following command for archiving:
tarz -cf <output_file.tar.gz> <directory>