通过FTP从无服务器AWS Lambda发送二进制图像

Question

I'm building an AWS Lambda API, using the serverless stack, that will receive an image as a binary string then store that image on GitHub and send it over FTP to a static app server. I am NOT storing the image in S3 at all as I am required not to.

I have already figured out how to save the image to GitHub (by converting binary to base64), the issue is sending the binary image data over FTP to another server to hold statically. The static image server is pre-existing and I cannot use S3. I am using the ftp npm package to send the image. The image server indeed receives the image BUT NOT in the correct format and the image is just non-displayable junk data. I saw an example on how to do this on the client side by encoding the image in an Uint16Array and passing it to a new Blob() object, but sadly, NODE.JS DOES NOT SUPPORT BLOBS!

Using ftp npm module:

    async sendImage(image) {
        try {
            const credentials = await getFtpCredentials(this.platform);
            console.log('Sending files via FTP client');
            return new Promise((resolve, reject) => {
                let ftpClient = new ftp();
                ftpClient.on('ready', () => {
                    console.log(`Putting ${image.path} onto server`);
                    // Set transfer type to binary
                    ftpClient.binary(err => {
                        if(err)
                            reject(err);
                    });
                    // image.content is binary string data
                    ftpClient.put(image.content, image.path, (err) => {
                        if (err) {
                            reject(err);
                        }
                        console.log('Closing FTP connection');
                        ftpClient.end();
                        resolve(`FTP PUT for ${image.path} successful!`);
                    });
                });
                console.log('Connecting to FTP server');
                ftpClient.connect(credentials);
            });
        } catch(err) {
            console.log('Failed to load image onto FTP server');
            throw err;
        }
    }

This procedure sends the binary data to the server, but the data is un-readable by any browser or image viewer. Do I need to use another FTP package? Or am I just not encoding this right?? I've spent days googleing the answer to this seemingly common task and it's driving me up the wall! Can anyone instruct me on how to send binary image data from a node.js lambda function over FTP to another server so that the image is encoded properly and works when viewed? ANY help is very much appreciated!

Answer 1

So, it turns out that the ftp node module that I was using was the issue all along. It will corrupt any binary image transferred over FTP. I submitted a ticket to the their GitHub repo, but they haven't made a commit in 4 years so I don't expect a timely fix.

To solve the problem, I used the Basic FTP package instead:

const ftp = require('basic-ftp');

async sendImage(image) {
    try {
        const { host, user, password } = await getFtpCredentials(this.platform);
        console.log('Received FTP credentials');
        console.log('Creating FTP client');
        const client = new ftp.Client();
        client.ftp.verbose = true;
        console.log('Logging in to remote server via FTP');
        await client.access({
            host, user, password, secure: false
        });
        console.log('Creating readable stream from binary data');
        const buffer = new Buffer(image.content, 'binary');
        const imgStream = new Readable();
        imgStream._read = () => {};
        imgStream.push(buffer);
        imgStream.push(null);
        console.log('Uploading readable stream to server via FTP');
        const result = await client.upload(imgStream, image.path);
        client.close();
        return result
    } catch(err) {
        console.log('Failed to load image onto FTP server');
        throw err;
    }
}

Note that this package requires data to be transferred via a Readable stream. Thanks to this SO answer, I learned how to convert a binary string to a Readable stream. All works fine.

Happy coding!