[英]Store files in mongodb with Nodejs
I was saving my files on the FS of my server and now I want to save them in the mongodb.(for easier backup and stuff).I want to store files like 4-5Mb maximum and I tried save them with mongoose with Buffer type.I successfully saved them and retrieved them but I noticed a significant slow performance when i save and retrieve files like 4 or 5Mb. 我将文件保存在服务器的FS上,现在我想将它们保存在mongodb中。(为了更容易备份和填充)。我想存储最大4-5Mb的文件,我尝试使用带有缓冲类型的mongoose保存它们。我成功保存并检索了它们但是当我保存并检索4或5Mb等文件时,我注意到性能显着降低。
My schema: 我的架构:
let fileSchema = new Schema({
name: {type: String, required: true},
_announcement: {type: Schema.Types.ObjectId, ref: 'Announcements'},
data: Buffer,
contentType: String
});
How I retrieve them from the expressjs server: 我如何从expressjs服务器检索它们:
let name = encodeURIComponent(file.name);
res.writeHead(200, {
'Content-Type': file.contentType,
'Content-Disposition': 'attachment;filename*=UTF-8\'\'' + name
});
res.write(new Buffer(file.data));
My question is should I use some zlib compress functions like 'deflate' to compress buffer before saving them in the mongodb and then uncompress the binary before sending them to the client? 我的问题是我应该使用一些zlib压缩函数,如'deflate'来压缩缓冲区,然后再将它们保存在mongodb中,然后在将它们发送到客户端之前解压缩二进制文件? Would this make the whole proccess faster?Am I missing something?
这会让整个过程变得更快吗?我错过了什么吗?
It seems that you are trying to save a really big amount of information with mongoDb. 您似乎正在尝试使用mongoDb保存大量信息。
I can think in 3 diferent options for your case 我可以为您的案例考虑3种不同的选择
Instead of storing a file in a single document, GridFS divides the file into parts, or chunks [1], and stores each chunk as a separate document.
GridFS不是将文件存储在单个文档中,而是将文件分成多个部分或块[1],并将每个块存储为单独的文档。 By default, GridFS uses a default chunk size of 255 kB;
默认情况下,GridFS使用默认的块大小255 kB; that is, GridFS divides a file into chunks of 255 kB with the exception of the last chunk.
也就是说,GridFS将文件分成255 kB的块,但最后一个块除外。
And next they say in what situations you may use this way of storing information: 接下来他们说在什么情况下你可以用这种方式存储信息:
In some situations, storing large files may be more efficient in a MongoDB database than on a system-level filesystem.
在某些情况下,在MongoDB数据库中存储大文件可能比在系统级文件系统上更高效。
- If your filesystem limits the number of files in a directory, you can use GridFS to store as many files as needed.
如果文件系统限制目录中的文件数,则可以使用GridFS根据需要存储任意数量的文件。
- When you want to access information from portions of large files without having to load whole files into memory, you can use GridFS to recall sections of files without reading the entire file into memory.
如果要从大型文件的各个部分访问信息而无需将整个文件加载到内存中,可以使用GridFS调用文件的各个部分,而无需将整个文件读入内存。
- When you want to keep your files and metadata automatically synced and deployed across a number of systems and facilities, you can use GridFS.
如果要保持文件和元数据在多个系统和设施中自动同步和部署,可以使用GridFS。 When using geographically distributed replica sets, MongoDB can distribute files and their metadata automatically to a number of mongod instances and facilities.
使用地理位置分散的副本集时,MongoDB可以自动将文件及其元数据分发到许多mongod实例和工具。
Hope it was useful :) 希望它有用:)
I will suggest you to use GridFS
it's faster and very easy to use. 我建议你使用
GridFS
它更快,更容易使用。
For more info please check this url: https://docs.mongodb.com/manual/core/gridfs/ . 有关详细信息,请查看此URL: https : //docs.mongodb.com/manual/core/gridfs/ 。
If you have any question about GridFS
let me know. 如果您对
GridFS
有任何疑问, GridFS
告诉我。
If you absolutely feel that you must store the images in your Database and not in filesystem or other cloud services, I wont comment on that. 如果您绝对认为必须将图像存储在数据库中而不是存储在文件系统或其他云服务中,我不会对此发表评论。
With respect to your specific question, GridFS is a respectable option which people use in production as well and has served its purpose quite well. 关于您的具体问题,GridFS是人们在生产中使用的一个值得尊敬的选择,并且很好地实现了它的目的。 I personally used it couple of years back but my use case changed therefore moved to another medium.
几年前我个人使用它,但我的用例改变了,因此转移到另一种媒介。 (Please check the SO link where people are discussing its performance)
(请查看人们讨论其性能的SO链接)
What is of concern is the fact that you have 4mb images, unless you are serving images with huge dependency on quality and big resolution - that should not happen. 令人担忧的是,你有4mb的图像,除非你提供的图像对质量和大分辨率有很大的依赖性 - 这不应该发生。 Please compress your images before storing them, do it on the frontend or backend (your choice), if you compress them on frontend itself then it will reduce the transmission time of packets.
请在存储之前压缩图像,在前端或后端(您的选择)上进行压缩,如果您在前端压缩它们,那么它将减少数据包的传输时间。
Discussion on scale of GridFS 关于GridFS规模的讨论
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.