Actions
Task #1904
openreplace xz with different compression tool to compress database dumps (xz is single-threaded, takes too long) - evaluate parallel compression
Status:
New
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
Start date:
Due date:
% Done:
80%
Tags:
URL:
Description
creating database dumps takes way too long with xz, better switch to different compression, or even to a tool that can use multiple cores..
Updated by Jan Iversen over 8 years ago
We could use lbzip2 or pbzip2, they both use the number of cores available (or take a parameter to limit.
In order not to change the output files we could do:
tar cf - paths-to-archive | lbzip2 -9 -j 32 > archive.tar.gz
If the scripts are in SALT, I will be happy to change and test it.
Just saw "tar" has an option to do that
tar -c --use-compress-program=pigz -f tar.file dir_to_xz
Updated by Christian Lohmaier over 8 years ago
- Subject changed from replace xz with different compression tool to compress database dumps (xz is single-threaded, takes too long) to replace xz with different compression tool to compress database dumps (xz is single-threaded, takes too long) - evaluate parallel compression
- Priority changed from Normal to Low
- % Done changed from 0 to 80
changed to use plain gzip in salt - leaving with low prio to maybe test out the parallel compression variants at a later time
Actions