[英]Unzipping bz2 file
I have the following command to open a tbz file: 我有以下命令打开一个tbz文件:
# pricing20150304.tbz
tar xpj -C {tarball} {files_to_unarchive}
The compressed file is about 15 GB and when it is expanded it is about 500GB or so. 压缩文件约为15 GB,展开时约为500 GB左右。 This operation on an ec2-4x-large operation takes roughly 1h40m.
一次ec2-4x大型操作的此操作大约需要1h40m。
Is there a way to optimize this operation? 有没有优化此操作的方法? What would be the fastest way to do the above operation?
进行上述操作最快的方法是什么?
A couple possibilities come to mind. 我想到了几种可能性。 First off, bzip2 is pretty slow, so if you can use a different algorithm you might want to consider doing so.
首先,bzip2相当慢,因此,如果您可以使用其他算法,则可能需要考虑这样做。 Assuming you still want a fairly high ratio, LZHAM and Brotli might be good choices;
假设您仍然想要一个相当高的比率,LZHAM和Brotli可能是不错的选择。 they take longer to compress but are much faster when it comes to decompression, and IIRC both come with multi-threaded decompressors.
它们需要更长的时间进行压缩,但在解压缩时要快得多,而且IIRC都带有多线程解压缩器。 There are lots of choices, and they all have different trade-offs between compression speed, decompression speed, and ratio.
有很多选择,并且它们在压缩速度,解压缩速度和比率之间都有不同的权衡 。
If a different algorithm isn't an option, you might want to consider using pbzip2 instead of bzip2. 如果无法选择其他算法,则可能要考虑使用pbzip2而不是bzip2。 Something like
pbzip2 -dc infile.tar.bz2 | tar x
类似于
pbzip2 -dc infile.tar.bz2 | tar x
pbzip2 -dc infile.tar.bz2 | tar x
. pbzip2 -dc infile.tar.bz2 | tar x
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.