节点：尝试流式传输 Excel 文件并将缓冲区传递给“xlsx”库

Question

I'm using the node.js package 'xlsx' to read an excel file. Reading a file from the file system synchronously works fine, but streaming is a bit tricky. ( The file is served remotely and I can only receive it as a stream. ) Local streamed files produce the same problem, so it's easy to reproduce.

I've followed the examples here: https://www.npmjs.com/package/xlsx but get "end of data reached" or "unsupported file" depending on approach.

const XLSX = require('xlsx');
const fs = require('fs');

const stream = fs.createReadStream('sample.xlsx');


// This function returns Error: Unsupported file 48

documentedExample = function(){
  var arr = new Array();

  stream.on('data', function( arraybuffer ){
    var data = new Uint8Array(arraybuffer);
    for(var i = 0; i != data.length; ++i) arr[i] = String.fromCharCode(data[i]);
  });
  stream.on('end', function(){
    var bstr = arr.join("");
    var workbook = XLSX.read(bstr, {type:"binary"});

  });
}


// This function returns Error: End of data reached (data length = 75589, asked index = 77632). Corrupted zip ?

alternateExample = function(){
  var bufferArray = [];
  stream.on('data', function( thisChunk ){
    bufferArray.push( thisChunk );
  });
  stream.on('end', function(){
    var excelDataBuffer = bufferArray.join("");
    excelDataBuffer = excelDataBuffer.toString();
    var workbook = XLSX.read(excelDataBuffer, {type:"binary"});
  });
}

What would be the correct way to reassemble an xlsx binary for local use?

Answer 1

Current library do not support streams for reading

From their docs :

Note on Streaming Read

The most common and interesting formats (XLS, XLSX/M, XLSB, ODS) are ultimately ZIP or CFB containers of files. Neither format puts the directory structure at the beginning of the file: ZIP files place the Central Directory records at the end of the logical file, while CFB files can place the FAT structure anywhere in the file! As a result, to properly handle these formats, a streaming function would have to buffer the entire file before commencing. That belies the expectations of streaming, so we do not provide any streaming read API. If you really want to stream, there are node modules like concat-stream that will do the buffering for you.

There are something for writing in streams in xlsx package:

The streaming write functions are available in the XLSX.stream object. They take the same arguments as the normal write functions but return a readable stream. They are only exposed in node.

XLSX.stream.to_csv is the streaming version of XLSX.utils.sheet_to_csv.

XLSX.stream.to_html is the streaming version of XLSX.utils.sheet_to_html.

I have wrote to support asking do Pro version has it and received this answer:

The zip format, used in XLSX, prevents true stream reading (you have to read to the end of the file in order to find out where all of the subfiles are). I wouldn't use the extremist language of https://github.com/thejoshwolfe/yauzl#no-streaming-unzip-api but I agree with the core message:

Any library that offers a streaming unzip API [is] either dishonest or nonconformant (usually the latter).

We offer event-based reading, which skips building a full workbook object. You receive row objects and that minimizes memory pressure.

On the write side, we offer stream-based XLSX writing as well as SpreadsheetML and other formats.

In my case I had to rewrote everything =(