如何根据请求散列和下载图像？

Question

Situation

What I want to achieve, is download an image to a buffer, hash it, then save it.

I can not write to disk before hashing.

Code (inside a Promise)

const filename = this.getPathFromHash(hash);
if (!fs.existsSync(path.dirname(filename))) fs.mkdirSync(path.dirname(filename));
fs.writeFile(filename, body, 'binary', (e) => {
  if (e) reject(e);
  parser.MediumVersion.create({
    hash, url, type, size, width, height,
  }).then(resolve).catch(reject);
});

Result

file corruption, but it created the files :)

Edit: IrFanView says that it's JPEG with an incorrect extension, then it just says not a jpeg file, starts with (two random bytes here)

Edit2: What is this?? Four 0xfd bytes? What do they mean? Otherwise the image header looks OK.

Edit3: Removing those bytes do not solve the problem :/ Maybe encoding is the culprit? I've tried changing it to UTF-8 , no luck so far.

Edit4: Quick comparison! 1st one downloaded using my program, second one, Chrome.

Directory structure

I know, I know, SHA-256 hashes are 64 characters long. For the sake of simplicity, I'm going to use 8 characters.

Given the hash 0123abcd , create the directory 01 (if it doesn't exist ( slice(0,2) ), then put the file with name 23abcd ( slice(2) ) in it. I've seen this structure in lots of caches.

Example:

01/
  23abcd
  45abcd
ab/
  cdef01
  010101

Answer 1

As per https://github.com/request/request#requestoptions-callback :

if you expect binary data, you should set encoding: null .

So for me that would be request(url, { encoding: null }, (e, resp, body) => {...}) .

In that case, the image is perfectly fine and readable :)