C ++ Tweetnacl散列文件而不将整个文件读到内存

Question

I'm using tweetnacl to generate sha512 hashes of strings and file. For strings it works quite well but i have no idea how to do it with files.

The signature of the function ist

extern "C" int crypto_hash(u8 *out, const u8 *m, u64 n);

where u8 is of type unsigned char and u64 is of unsigend long long. For string a can use it like that

string s("Hello");
unsigned char h[64];

crypto_hash(h, (unsigned char *)s.c_str(), s.size());

This works great for a string and small files but if i want to create a hash for a big file, it is not viable and uses to much memory. I searching for a solution to read the file byte by byte and pass it as unsigend char pointer to that function. Has anyone a idea how to achieve that?

PS Sorry for the poor English. pss I use tweetnacl because of the small size and i need only the hashing function.

Answer 1

Probably the easiest way is to use a memory-mapped file . This lets you open a file and map it into virtual memory, then you can treat the file on disk as if it is in memory, and the OS will load pages as required.

So in your case, open your file and use mmap() to map it into memory. Then you can pass the pointer into your crypto_hash() function and let the OS do the work.

Note that there are caveats to do with how large the file is wrt virtual memory.

For various platforms:

• Boost Interprocess
• macOS and mmap
• Linux and mmap
• Windows .NET MemoryMappedFile

Answer 2

I'd suggest you to use a different implementation, one which you can incrementally feed in chunks.

This one for example . As the licence is bsd and the code is C with no dependencies, you can copy/paste only the 3 functions that you need without bringing a whole lib (no matter how small) into your project.

The life-cycle goes like:

• sha256_init(&ctx)

• repeatedly read blocks from file and feed them into sha256_update(&ctx, buff, buffLen)

• when EOF, get your digest using sha256_final(&ctx, digestHere)