[英]Reading an LZ4 compressed text file (mozlz4) in WebExtensions (JavaScript, Firefox)
I'm porting a Firefox Add-on SDK extension to WebExtensions. 我正在向WebExtensions移植Firefox Add-on SDK扩展。 Previously I could access the browser's search engines, but now I can't, so a helpful user suggested I try reading the search.json.mozlz4 file, which has every installed engine.
以前我可以访问浏览器的搜索引擎,但现在我不能,所以一个有用的用户建议我尝试阅读search.json.mozlz4文件,其中包含每个已安装的引擎。 However, this file is json with LZ4 compression, and it's in Mozilla's own LZ4 format , with a custom magic number, 'mozLz40\\0'.
但是,这个文件是带有LZ4压缩的json,它采用Mozilla 自己的LZ4格式 ,带有自定义幻数“mozLz40 \\ 0”。
Before, one could use this to read a text file that uses LZ4 compression, including a mozlz4 file: 之前,可以使用它来读取使用LZ4压缩的文本文件,包括mozlz4文件:
let bytes = OS.File.read(path, { compression: "lz4" });
let content = new TextDecoder().decode(bytes);
(although I couldn't find documentation about the "compression" field, it works) (虽然我找不到关于“压缩”字段的文档,但它有效)
Now, using WebExtensions, the best I could come up with to read a file is 现在,使用WebExtensions,我能想出的最好的文件是读取文件
var reader = new FileReader();
reader.readAsText(file);
reader.onload = function(ev) {
let content = ev.target.result;
};
This does not handle compression in any way. 这不会以任何方式处理压缩。 This library handles LZ4
, but it is for node.js so I can't use that. 这个库处理LZ4
,但是它适用于node.js,所以我不能使用它。 [edit: it works standalone too]. [编辑:它也是独立的]。 However, even if I remove the custom magic number processing I can't get it to decompress the file, while this Python code, in comparison, works as expected:
但是,即使我删除了自定义幻数处理,我也无法解压缩该文件,而相比之下,这个Python代码按预期工作:
import lz4
file_obj = open("search.json.mozlz4", "rb")
if file_obj.read(8) != b"mozLz40\0":
raise InvalidHeader("Invalid magic number")
print(lz4.block.decompress(file_obj.read()))
How can I do this in JS? 我怎么能在JS中这样做?
After much trial and error, I was finally able to read and decode the search.json.mozlz4 file in a WebExtension. 经过多次试验和错误,我终于能够在WebExtension中读取和解码search.json.mozlz4文件。 You can use the node-lz4 library , though you'll only need one function -
uncompress
(aliased as decodeBlock
for external access) - so I renamed it to decodeLz4Block
and included it here with slight changes: 你可以使用node-lz4库 ,虽然你只需要一个函数 -
uncompress
(别名为decodeBlock
用于外部访问) - 所以我将它重命名为decodeLz4Block
并将其包含在这里稍作修改:
// This method's code was taken from node-lz4 by Pierre Curto. MIT license.
// CHANGES: Added ; to all lines. Reformated one-liners. Removed n = eIdx. Fixed eIdx skipping end bytes if sIdx != 0.
function decodeLz4Block(input, output, sIdx, eIdx)
{
sIdx = sIdx || 0;
eIdx = eIdx || input.length;
// Process each sequence in the incoming data
for (var i = sIdx, j = 0; i < eIdx;)
{
var token = input[i++];
// Literals
var literals_length = (token >> 4);
if (literals_length > 0) {
// length of literals
var l = literals_length + 240;
while (l === 255) {
l = input[i++];
literals_length += l;
}
// Copy the literals
var end = i + literals_length;
while (i < end) {
output[j++] = input[i++];
}
// End of buffer?
if (i === eIdx) {
return j;
}
}
// Match copy
// 2 bytes offset (little endian)
var offset = input[i++] | (input[i++] << 8);
// 0 is an invalid offset value
if (offset === 0 || offset > j) {
return -(i-2);
}
// length of match copy
var match_length = (token & 0xf);
var l = match_length + 240;
while (l === 255) {
l = input[i++];
match_length += l;
}
// Copy the match
var pos = j - offset; // position of the match copy in the current output
var end = j + match_length + 4; // minmatch = 4
while (j < end) {
output[j++] = output[pos++];
}
}
return j;
}
Then declare this function that receives a File object (not a path) and callbacks for success/error: 然后声明这个接收File对象(不是路径)的函数和成功/错误的回调:
function readMozlz4File(file, onRead, onError)
{
let reader = new FileReader();
reader.onload = function() {
let input = new Uint8Array(reader.result);
let output;
let uncompressedSize = input.length*3; // size estimate for uncompressed data!
// Decode whole file.
do {
output = new Uint8Array(uncompressedSize);
uncompressedSize = decodeLz4Block(input, output, 8+4); // skip 8 byte magic number + 4 byte data size field
// if there's more data than our output estimate, create a bigger output array and retry (at most one retry)
} while (uncompressedSize > output.length);
output = output.slice(0, uncompressedSize); // remove excess bytes
let decodedText = new TextDecoder().decode(output);
onRead(decodedText);
};
if (onError) {
reader.onerror = onError;
}
reader.readAsArrayBuffer(file); // read as bytes
};
Then you can add an HTML button to your add-on settings page that lets the user search and select search.json.mozlz4 (in WebExtensions you can't simply open any file in the filesystem without user intervention): 然后,您可以在附加设置页面上添加一个HTML按钮,让用户搜索并选择search.json.mozlz4(在WebExtensions中,您无法在没有用户干预的情况下打开文件系统中的任何文件):
<input name="selectMozlz4FileButton" type="file" accept=".json.mozlz4">
To respond to the user selecting the file, use something like this, which calls the method we previously declared (here I don't use the error callback, but you can): 要响应用户选择文件,请使用类似这样的方法,调用我们之前声明的方法(这里我不使用错误回调,但你可以):
let button = document.getElementsByName("selectMozlz4FileButton")[0];
button.onchange = function onButtonPress(ev) {
let file = ev.target.files[0];
readMozlz4File(file, function(text){
console.log(text);
});
};
I hope this helps someone. 我希望这可以帮助别人。 I sure spent a lot of time working this simple thing out.
我确实花了很多时间来处理这个简单的事情。 :)
:)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.