Node.js-仅出于获取文件长度的目的将文件流保存到磁盘是否可以接受？

Question

I have multiple Node.js servers running on my backend. One is an API server, which can accept image files from a 3rd party. The image files are streamed to the stdin of an ImageMagick process, and then the stdout of the ImageMagick process is streamed to a Node.js TCP server where the file is ultimately saved locally. The TCP server then needs to send a response to the API server after the file is successfully saved, so I need some way for the TCP server to know when it has the entire file (ie I can't simply close the socket on the API server after the file is sent).

One solution I could use is to save the stdout of the ImageMagick process to a temporary file on the API server, so I can get the length of the full file before I send it, and embed it in the beginning of the stream. Writing to disk throws a bit of a knot in the system though.

Is it acceptable to write a temp file to disk for the purpose of getting the length of the file, or is there a better / more efficient way to solve this problem?

Thanks

Answer 1

I do not know if this is relevant. But ImageMagick miff: format is a streaming format. So you can write the dimensions of the image to a text file while streaming it out again via miff:.

convert rose: miff:- | convert - -format "%wx%h" -write info:tmp.txt miff:-

Here I use convert rose: miff:- to simulate your input stream.

Then I pipe it to the next convert, which reads the input stream, writes the WxH information to a tmp.txt text file, which you could access subsequently. The second convert also creates a miff:- output stream.

You could use NetPBM format in place of miff, since it also is a streaming format.

Answer 2

In case anybody else with a similar problem stumbles upon this, I found an alternative solution, which avoids the need to specify the file size altogether.

Create the TCP server using the {allowHalfOpen: true} option. The client can send the file to the TCP server, and then simply call the "end" method on the socket to signify that no more data will be written by the client (the socket is still readable by the client). With the "allowHalfOpen" option set on the server, the server can simply listen for the "end" event (which signifies that all of the data has been received), with the writable side of the server still open (allowing the server to send a response back to the client).

Note that the "allowHalfOpen" option defaults to false. If this option isn't set, the server automatically closes the socket when the writable side of the client is closed.

eg

SERVER

const fs = require('fs');
const net = require('net');
const server = net.createServer({allowHalfOpen: true});

server.on('connection', (socket) => {
    const writeStream = fs.createWriteStream('myFile.png');

    socket.on('data', (chunk) => {
        writeStream.write(chunk);
    });

    socket.on('end', () => {
        writeStream.end();
        socket.end('File Written To Disk'); // send response and close socket
    });
});

server.listen(8000);

CLIENT

const fs = require('fs');
const net = require('net');

const socket = net.connect(8000);

fs.createReadStream('myFile.png').pipe(socket);

socket.on('data', (chunk) => {
    // response from TCP server
    console.log(chunk.toString()); // File Written To Disk
});

socket.on('end', () => {
    // server closed socket
});