使用LZ4解壓縮byte []

Question

我正在使用LZ4來壓縮和解壓縮字符串。我嘗試了以下方法

public class CompressionDemo {

    public static byte[] compressLZ4(LZ4Factory factory, String data) throws IOException {
        final int decompressedLength = data.getBytes().length;
        LZ4Compressor compressor = factory.fastCompressor();
        int maxCompressedLength = compressor.maxCompressedLength(decompressedLength);
        byte[] compressed = new byte[maxCompressedLength];
        compressor.compress(data.getBytes(), 0, decompressedLength, compressed, 0, maxCompressedLength);
        return compressed;

    }

    public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
        LZ4FastDecompressor decompressor = factory.fastDecompressor();
        byte[] restored = new byte[data.length];
        decompressor.decompress(data,0,restored, 0,data.length);
        return new String(restored);
    }

    public static void main(String[] args) throws IOException, DataFormatException {
        String string = "kjshfhshfashfhsakjfhksjafhkjsafhkjashfkjhfjkfhhjdshfhhjdfhdsjkfhdshfdskjfhksjdfhskjdhfkjsdhfk";
        LZ4Factory factory = LZ4Factory.fastestInstance();
        byte[] arr = compressLZ4(factory, string);
        System.out.println(arr.length);
        System.out.println(deCompressLZ4(factory, arr) + "decom");
    }
}

它給予了以下的激勵

線程“main”中的異常net.jpountz.lz4.LZ4Exception：解碼輸入緩沖區的偏移量92時出錯

這里的問題是只有當我傳遞實際的String byte []長度時，解壓縮才有效

public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
        LZ4FastDecompressor decompressor = factory.fastDecompressor();
        byte[] restored = new byte[data.length];
        decompressor.decompress(data,0,restored, 0,"kjshfhshfashfhsakjfhksjafhkjsafhkjashfkjhfjkfhhjdshfhhjdfhdsjkfhdshfdskjfhksjdfhskjdhfkjsdhfk".getBytes().length);
        return new String(restored);
    }

它期望實際的字符串byte []大小。 有人可以幫我弄這個嗎

Answer 1

由於壓縮和解壓縮可能發生在不同的機器上，或者機器默認字符編碼不是Unicode格式之一，因此也應該指示編碼。

對於其余部分，它使用實際的壓縮和解壓縮長度，並且更好地以普通格式存儲未壓縮數據的大小，因此可以在解壓縮之前提取它。

public static byte[] compressLZ4(LZ4Factory factory, String data) throws IOException {
    byte[] decompressed = data.getBytes(StandardCharsets.UTF_8).length;
    LZ4Compressor compressor = factory.fastCompressor();
    int maxCompressedLength = compressor.maxCompressedLength(decompressed.length);
    byte[] compressed = new byte[4 + maxCompressedLength];
    int compressedSize = compressor.compress(decompressed, 0, decompressed.length,
                                             compressed, 4, maxCompressedLength);
    ByteBuffer.wrap(compressed).putInt(decompressed.length);
    return Arrays.copyOf(compressed, 0, 4 + compressedSize);
}

public static String deCompressLZ4(LZ4Factory factory, byte[] data) throws IOException {
    LZ4FastDecompressor decompressor = factory.fastDecompressor();
    int decrompressedLength = ByteBuffer.wrap(data).getInt();
    byte[] restored = new byte[decrompressedLength];
    decompressor.decompress(data, 4, restored, 0, decrompressedLength);
    return new String(restored, StandardCharsets.UTF_8);
}

應該告訴我， String不適合二進制數據，壓縮/解壓縮僅用於文本處理。 （字符串包含UTF-16雙字節字符形式的Unicode文本。轉換為二進制數據總是涉及二進制數據編碼的轉換。這會降低內存，速度和可能的數據損壞。）

Answer 2

我剛剛在Android上面臨同樣的錯誤並根據以下問題解決了它： https ： //github.com/lz4/lz4-java/issues/68

簡而言之，請確保您使用相同的工廠進行兩種操作（壓縮+解壓縮）並使用Arrays.copyOf（），如下所示：

  byte[] compress(final byte[] data) {
     LZ4Factory lz4Factory = LZ4Factory.safeInstance();
     LZ4Compressor fastCompressor = lz4Factory.fastCompressor();
     int maxCompressedLength = fastCompressor.maxCompressedLength(data.length);
     byte[] comp = new byte[maxCompressedLength];
     int compressedLength = fastCompressor.compress(data, 0, data.length, comp, 0, maxCompressedLength);
     return Arrays.copyOf(comp, compressedLength);
}

  byte[] decompress(final byte[] compressed) {
     LZ4Factory lz4Factory = LZ4Factory.safeInstance();
     LZ4SafeDecompressor decompressor = lz4Factory.safeDecompressor();
     byte[] decomp = new byte[compressed.length * 4];//you might need to allocate more
     decomp = decompressor.decompress(Arrays.copyOf(compressed, compressed.length), decomp.length);
     return decomp;

希望這會有所幫助。

Answer 3

恢復的byte []長度是小的，你不應該使用壓縮的data.length，而應該使用data [] .length * 3或者3。

Answer 4

我像這樣恢復：

public static byte[] decompress( byte[] finalCompressedArray,String ... extInfo) {
    int len = finalCompressedArray.length * 3;
    int i = 5;
    while (i > 0) {
        try {
            return decompress(finalCompressedArray, len);
        } catch (Exception e) {
            len = len * 2;
            i--;
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("decompress Error: extInfo ={} ", extInfo, e);
            }

        }

    }

    throw new ItemException(1, "decompress error");
}

/**
 * 解壓一個數組
 *
 * @param finalCompressedArray 壓縮后的數據
 * @param length               原始數據長度, 精確的長度,不能大,也不能小。
 * @return
 */
private static byte[] decompress(byte[] finalCompressedArray, int length) {
    byte[] desc = new byte[length ];
    int decompressLen = decompressor.decompress(finalCompressedArray, desc);

    byte[] result = new byte[decompressLen];
    System.arraycopy(desc,0,result,0,decompressLen);
    return result;
}