简体繁体 English

临时存储组成较大文件的不连续数据的最有效方法

[英]most efficient way to temporarily store discontinuous data composing a larger file

原文 2017-02-14 17:14:51 4 2 java/ file/ p2p

We are emulating a p2p network in java. 我们正在用Java模拟p2p网络。 So we divide the file into chunks (with checksums) so that the individual chunks can be recompiled into the original file once we have all the parts. 因此，我们将文件分为多个块（带有校验和），以便一旦我们拥有所有部分，就可以将各个块重新编译为原始文件。 What is the best way to store the individual parts while they are being downloaded? 在下载各个部分时，最好的存储方式是什么？

I was thinking of just storing each chunk as a separate file...but if there are 20000 chunks, it would create as many files. 我当时只是想将每个块存储为一个单独的文件...但是，如果有20000个块，它将创建尽可能多的文件。 is this the best way? 这是最好的方法吗？

Thanks 谢谢

2 个解决方案

Either keep chunks in memory or in files. 要么将块保留在内存中，要么保留在文件中。 There is no much to discuss here about. 这里没有什么要讨论的。 Findd the perfect ratio between chunks count and the actual size of it, to suit your needs. 找到了块数与实际大小之间的理想比例，以满足您的需求。

Files sounds more reasonable as data would not be totally lost in case of application crash and continue of download would be possible. 文件听起来更合理，因为在应用程序崩溃的情况下数据不会完全丢失，并且可以继续下载。

I would write to memory until you reach some threshold, at which point you dump your memory to disk, and keep reading into memory. 我会写内存，直到达到某个阈值为止，这时将内存转储到磁盘，并继续读入内存。 When the file transfer completes, you can take what is currently stored in memory, and concatenate it with what may have been stored on disk. 文件传输完成后，您可以获取当前存储在内存中的内容，并将其与磁盘上可能存储的内容连接起来。