[英]AWS S3 - Etag Sha256 instead of Md5
I want to use Sha256 for the checksum of my objects.我想使用 Sha256 作为我的对象的校验和。 But it looks like, that amazon uses md5 in the ETag.
但看起来,亚马逊在 ETag 中使用了 md5。
Is there any workaround?有什么解决方法吗?
This is possible as of 2022-02-25.从 2022-02-25 开始,这是可能的。 S3 now features a Checksum Retrieval function
GetObjectAttributes
: S3 现在具有校验和检索 function
GetObjectAttributes
功能:
New – Additional Checksum Algorithms for Amazon S3 | 新 – 用于 Amazon S3 的附加校验和算法 | AWS News Blog
AWS 新闻博客
Checksum Retrieval – The new
GetObjectAttributes
function returns the checksum for the object and (if applicable) for each part.校验和检索——新的
GetObjectAttributes
function 返回 object 和(如果适用)每个部分的校验和。
This function supports SHA-1, SHA-256, CRC-32, and CRC-32C for checking the integrity of the transmission.此 function 支持 SHA-1、SHA-256、CRC-32 和 CRC-32C,用于检查传输的完整性。
I'm so glad that they now have alternatives to the sad choice of MD5, which is not optimal for anything in particular and was broken for other purposes long ago.我很高兴他们现在有了 MD5 的可悲选择的替代方案,MD5 对于任何特定的东西都不是最佳选择,并且很久以前就因为其他目的而被破坏了。 See also related discussion of quirks with their MD5 approach at How to get the md5sum of a file on Amazon's S3 .
另请参阅How to get the md5sum of a file on Amazon's S3 中关于MD5 方法的怪癖的相关讨论。
[And while I'm discussing hashes for various purposes, note that a good one for hash-table lookups and other situations which have some basic randomness and security propertiees is HighwayHash: Fast strong hash functions: SipHash/HighwayHash ] [虽然我正在讨论用于各种目的的哈希,但请注意,对于哈希表查找和其他具有一些基本随机性和安全属性的情况,一个很好的哈希是HighwayHash: Fast strong hash functions: SipHash/HighwayHash ]
Unfortunately, there's no direct way to make S3 use SHA256 for ETag.不幸的是,没有直接的方法可以让 S3 将 SHA256 用于 ETag。 You could use S3 metadata as a workaround.
您可以使用 S3 元数据作为解决方法。 For this, you can calculate the SHA256 checksum yourself and use user defined S3 object metadata to set it for each upload.
为此,您可以自己计算 SHA256 校验和,并使用用户定义的 S3 对象元数据为每次上传设置它。 User defined metadata is just a set of key-value pairs you can assign to your object.
用户定义的元数据只是一组可以分配给对象的键值对。 You'll have to set the checksum when you
PUT
your object and compare it on GET
/ HEAD
object.你必须在你设置校验和
PUT
你的对象,并比较在GET
/ HEAD
对象。
More information is available in the S3 documentation: S3 文档中提供了更多信息:
Please refer: How to calculate SHA-256 checksum of S3 file content请参考: 如何计算 S3 文件内容的 SHA-256 校验和
It can be achieved by following steps in Java:可以通过Java中的以下步骤来实现:
InputStream inputStream = amazonS3.getObject(bucket, file).getObjectContent();
InputStream inputStream = amazonS3.getObject(bucket, file).getObjectContent();
Use MessageDigest and DigestInputStream classes for the SHA-256 hash对 SHA-256 哈希使用 MessageDigest 和 DigestInputStream 类
public static String getHash(InputStream inputStream, String algorithm) { try { MessageDigest messageDigest = MessageDigest.getInstance(algorithm); DigestInputStream digestInputStream = new DigestInputStream(inputStream, messageDigest); byte[] buffer = new byte[4096]; int count = 0; while (digestInputStream.read(buffer) > -1) { count++; } log.info("total read: " + count); MessageDigest digest = digestInputStream.getMessageDigest(); digestInputStream.close(); byte[] md5 = digest.digest(); StringBuilder sb = new StringBuilder(); for (byte b: md5) { sb.append(String.format("%02X", b)); } return sb.toString().toLowerCase(); } catch (Exception e) { log.error(e); } return null; }
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.