在 Dart/Flutter 本地计算 Dropbox 的内容 Hash

Question

I'm trying to calculate the content_hash of a Dropbox file, following their algorithm, Dropbox docs here , however, I cannot get the overall hash (result of Step 4) correct.

I can produce the correct hashes on each block, so I suspect the issue starts with the conversion to the concantenated binary string (Step 3).

Per Dropbox's docs, there are 4 steps involved:

1. Split the file into blocks of 4 MB (4,194,304 or 4 * 1024 * 1024 bytes). The last block (if any) may be smaller than 4 MB.
2. Compute the hash of each block using SHA-256.
3. Concatenate the hash of all blocks in the binary format to form a single binary string.
4. Compute the hash of the concatenated string using SHA-256. Output the resulting hash in hexadecimal format.

They provide an example, with the resulting hash values, which I can only replicate up to step 2 (unfortunately, they do not provide an example of the single binary string for Step 3)

My code is as follows:

Future<void> getDropContentHash() async {

  String getHash(List<int> bytes) => sha256.convert(bytes).toString();

  String toBinary(String string) {
    /// converting string to binary string, following this SO Answer
    /// https://stackoverflow.com/a/25906926/12132021
    return string.codeUnits.map((x) => x.toRadixString(2).padLeft(8, '0')).join();
  }

  /// Nasa Milky Way Example:
  final response = await http.get(Uri.parse('https://www.dropbox.com/static/images/developers/milky-way-nasa.jpg'));
  final bytes = response.bodyBytes;

  /// Step 1: 
  /// Split the file into blocks of 4 MB (4,194,304 or 4 * 1024 * 1024 bytes). The last block (if any) may be smaller than 4 MB.

  final chunks = bytes.slices(4 * 1024 * 1024);
  print(chunks.length); // 3, as expected in the Nasa example

  /// Step 2: 
  /// Compute the hash of each block using SHA-256.
  final chunkHashes = <String>[];
  for(final chunk in chunks) {
    final blockHash = getHash(chunk);
    chunkHashes.add(blockHash);
  }
  print(chunkHashes);

  /*
  Per the docs, the chunkHashes are correct:
  [
    2a846fa617c3361fc117e1c5c1e1838c336b6a5cef982c1a2d9bdf68f2f1992a,
    c68469027410ea393eba6551b9fa1e26db775f00eae70a0c3c129a0011a39cf9,
    7376192de020925ce6c5ef5a8a0405e931b0a9a8c75517aacd9ca24a8a56818b
  ]
  */

  /// Step 3: 
  /// Concatenate the hash of all blocks in the binary format to form a single binary string.
  /// I suspect the issue with the `toBinary` method, converting to a single binary string:
  final concatenatedString = chunkHashes.map((chunkHash) => toBinary(chunkHash)).join();
  print(concatenatedString);

  /// Step 4: 
  /// Compute the hash of the concatenated string using SHA-256. Output the resulting hash in hexadecimal format.
  final contentHash = getHash(concatenatedString.codeUnits);
  print(contentHash);

  /*
  Does NOT yield the correct value of:
  485291fa0ee50c016982abbfa943957bcd231aae0492ccbaa22c58e3997b35e0

  Instead, it yields:
  0f63574f6c7cf29d1735e8f5ec4ef63abb3bc5c1b8618c6455a2c554bc844396
  */

}

Can anyone help me get the correct overall hash of 485291fa0ee50c016982abbfa943957bcd231aae0492ccbaa22c58e3997b35e0 ?

Answer 1

I figured it out. I did not need to convert to binary string. I just needed to continue working with bytes, and only convert to a string after the overall hash (step 4)

The resulting code (simplified & refactored) to calculate the Dropbox content hash locally:

  Future<void> hashNasaImage() async {
    final response = await http.get(Uri.parse('https://www.dropbox.com/static/images/developers/milky-way-nasa.jpg'));
    final bytes = response.bodyBytes;
    final contentHash = getDropContentHash(bytes);
    print(contentHash); 
    // 485291fa0ee50c016982abbfa943957bcd231aae0492ccbaa22c58e3997b35e0
  }

  String getDropContentHash(List<int> bytes) {
    List<int> getHash(List<int> bytes) => sha256.convert(bytes).bytes;    
    final chunks = bytes.slices(4 * 1024 * 1024);
    final chunkHashes = chunks.fold(<int>[], (buffer, chunk) => buffer..addAll(getHash(chunk)));
    return Digest(getHash(chunkHashes)).toString();
  }