[英]Writing file in /tmp in a Firebase Function does not work
I am writing a Firebase function that exposes an API endpoint using express
.我正在编写一个使用
express
公开 API 端点的 Firebase 函数。 When the endpoint is called, it needs to download an image from an external API and use that image to make a second API call.调用端点时,它需要从外部 API 下载图像并使用该图像进行第二次 API 调用。 The second API call needs the image to be passed as a
readableStream
.第二个 API 调用需要将图像作为
readableStream
传递。 Specifically, I am calling the pinFileToIPFS
endpoint of the Pinata API .具体来说,我正在调用Pinata API的
pinFileToIPFS
端点。
My Firebase function is using axios
to download the image and fs
to write the image to /tmp
.我的 Firebase 功能是使用
axios
下载图像并使用fs
将图像写入/tmp
。 Then I am using fs
to read the image, convert it to a readableStream
and send it to Pinata.然后我使用
fs
读取图像,将其转换为readableStream
并将其发送给 Pinata。
A stripped-down version of my code looks like this:我的代码的精简版本如下所示:
const functions = require("firebase-functions");
const express = require("express");
const axios = require("axios");
const fs = require('fs-extra')
require("dotenv").config();
const key = process.env.REACT_APP_PINATA_KEY;
const secret = process.env.REACT_APP_PINATA_SECRET;
const pinataSDK = require('@pinata/sdk');
const pinata = pinataSDK(key, secret);
const app = express();
const downloadFile = async (fileUrl, downloadFilePath) => {
try {
const response = await axios({
method: 'GET',
url: fileUrl,
responseType: 'stream',
});
// pipe the result stream into a file on disc
response.data.pipe(fs.createWriteStream(downloadFilePath, {flags:'w'}))
// return a promise and resolve when download finishes
return new Promise((resolve, reject) => {
response.data.on('end', () => {
resolve()
})
response.data.on('error', () => {
reject()
})
})
} catch (err) {
console.log('Failed to download image')
console.log(err);
throw new Error(err);
}
};
app.post('/pinata/pinFileToIPFS', cors(), async (req, res) => {
const id = req.query.id;
var url = '<URL of API endpoint to download the image>';
await fs.ensureDir('/tmp');
if (fs.existsSync('/tmp')) {
console.log('Folder: /tmp exists!')
} else {
console.log('Folder: /tmp does not exist!')
}
var filename = '/tmp/image-'+id+'.png';
downloadFile(url, filename);
if (fs.existsSync(filename)) {
console.log('File: ' + filename + ' exists!')
} else {
console.log('File: ' + filename + ' does not exist!')
}
var image = fs.createReadStream(filename);
const options = {
pinataOptions: {cidVersion: 1}
};
pinata.pinFileToIPFS(image, options).then((result) => {
console.log(JSON.stringify(result));
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Headers", "Authorization, Origin, X-Requested-With, Accept");
res.status(200).json(JSON.stringify(result));
res.send();
}).catch((err) => {
console.log('Failed to pin file');
console.log(err);
res.status(500).json(JSON.stringify(err));
res.send();
});
});
exports.api = functions.https.onRequest(app);
Interestingly, my debug messages tell me that the /tmp
folder exists, but the file of my downloaded file does not exist in the file system.有趣的是,我的调试消息告诉我
/tmp
文件夹存在,但我下载的文件的文件在文件系统中不存在。 [Error: ENOENT: no such file or directory, open '/tmp/image-314502.png']
. [Error: ENOENT: no such file or directory, open '/tmp/image-314502.png']
。 Note that the image can be accessed correctly when I manually access the URL of the image.请注意,当我手动访问图像的 URL 时,可以正确访问图像。
I've tried to download and save the file using many ways but none of them work.我尝试使用多种方法下载和保存文件,但没有一种方法有效。 Also, based on what I've read, Firebase Functions allow to write and read temp files from
/tmp
.此外,根据我所读到的内容,Firebase 函数允许从
/tmp
写入和读取临时文件。
Any advice will be appreciated.任何建议将被认真考虑。 Note that I am very new to NodeJS and to Firebase, so please excuse my basic code.
请注意,我对 NodeJS 和 Firebase 非常陌生,所以请原谅我的基本代码。
Many thanks!非常感谢!
I was not able to see you are initializing the directory as seen in this post :我看不到您正在初始化目录,如本文所示:
const bucket = gcs.bucket(object.bucket);
const filePath = object.name;
const fileName = filePath.split('/').pop();
const thumbFileName = 'thumb_' + fileName;
const workingDir = join(tmpdir(), `${object.name.split('/')[0]}/`);//new
const tmpFilePath = join(workingDir, fileName);
const tmpThumbPath = join(workingDir, thumbFileName);
await fs.ensureDir(workingDir);
Also, please consider that if you are using two functions, the /tmp
directory would not be shared as each one has its own.另外,请考虑如果您使用两个函数,
/tmp
目录将不会共享,因为每个函数都有自己的。 Here is an explanation from Doug Stevenson :这是Doug Stevenson 的解释:
Cloud Functions only allows one function to run at a time in a particular server instance.
Cloud Functions 仅允许在特定服务器实例中一次运行一个函数。 Functions running in parallel run on different server instances, which have different /tmp spaces.
并行运行的函数在具有不同 /tmp 空间的不同服务器实例上运行。 Each function invocation runs in complete isolation from each other.
每个函数调用彼此完全隔离运行。 You should always clean up files you write in /tmp so they don't accumulate and cause a server instance to run out of memory over time.
您应该始终清理您在 /tmp 中写入的文件,以免它们累积并导致服务器实例随着时间的推移耗尽内存。
I would suggest using Google Cloud Storage extended with Cloud Functions to achieve your goal.我建议使用 通过 Cloud Functions 扩展的 Google Cloud Storage来实现您的目标。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.