如何将 S3 存储桶中的数据文件导入 Sagemaker 笔记本？

Question

我有想要为我的 model 培训导入的 npz 文件。 下面是我尝试过的代码。

import s3fs
fs = s3fs.S3FileSystem()

# To List 5 files in your accessible bucket
#fs.ls('s3://input_data/train_npz/')[:5]

# open it directly
with fs.open(f's3://input_data/train_npz/0.npz') as f:
    display(Image.open(f))

 --------------------------------------------------------------------------- AttributeError Traceback (most recent call

last) in 7 8 # 直接打开 ----> 9 with fs.open(f's3://input_data/train_npz/0.npz') as f: 10 display(Image.open(f))

/opt/conda/lib/python3.6/site-packages/fsspec/spec.py in open(self, path, mode, block_size, cache_options, **kwargs) 980 autocommit=ac, 981 cache_options=cache_options, --> 982 **kwargs, 983 ) 984 如果不是 ac 并且 "r" 不在模式下：

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in _open(self, path, mode, block_size, acl, version_id, fill_cache, cache_type, autocommit, requester_pays, **kwargs) 543 cache_type= cache_type, 544 autocommit=autocommit, --> 545 requester_pays=requester_pays, 546) 547

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in init (self, s3, path, mode, block_size, acl, version_id, fill_cache, s3_additional_kwargs, autocommit, cache_type, requester_pays) 1822 self. version_id = self.details.get("VersionId") 1823
极好的（）。 init (-> 1824 s3, path, mode, block_size, autocommit=autocommit, cache_type=cache_type 1825) 1826 self.s3 = self.fs # 兼容性

/opt/conda/lib/python3.6/site-packages/fsspec/spec.py in init (self, fs, path, mode, block_size, autocommit, cache_type, cache_options, **kwargs) 1302 if mode == "rb “：1303
if not hasattr(self, "details"): -> 1304 self.details = fs.info(path) 1305 self.size = self.details["size"] 1306 self.cache = caches[cache_type](

/opt/conda/lib/python3.6/site-packages/fsspec/asyn.py in wrapper(*args, **kwargs) 86 def wrapper(*args, **kwargs): 87 self = obj 或 args[0 ] ---> 88 return sync(self.loop, func, *args, **kwargs) 89 90 return wrapper

/opt/conda/lib/python3.6/site-packages/fsspec/asyn.py in sync(loop, func, timeout, *args, **kwargs) 67 raise FSTimeoutError 68 if isinstance(result[0], BaseException) : ---> 69 提升结果[0] 70 返回结果[0] 71

/opt/conda/lib/python3.6/site-packages/fsspec/asyn.py 在 _runner（事件，coro，结果，超时）23 coro = asyncio.wait_for（coro，timeout=timeout）24 尝试：--- > 25 结果 [0] = 等待 coro 26 例外为 ex：27 结果 [0] = ex

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in _info(self, path, bucket, key, refresh, version_id) 1062 else: 1063 try: -> 1064 out = await self._simple_info (路径) 1065 除了 PermissionError: 1066

如果权限不足以扫描前缀

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in _simple_info(self, path) 982 Delimiter="/", 983 MaxKeys=1, --> 984 **self.req_kw, 985 ) 986 # 此方法可以返回 object 的信息块，如果它

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in _call_s3(self, method, *akwarglist, **kwargs) 235 236 async def _call_s3(self, method, *akwarglist, **kwargs ): --> 237 await self.set_session() 238 s3 = await self.get_s3(kwargs.get("Bucket")) 239 method = getattr(s3, method)

/opt/conda/lib/python3.6/site-packages/s3fs/core.py in set_session(self, refresh, kwargs) 376 377 conf = AioConfig(**config_kwargs) --> 378 self.session = aiobotocore.AioSession (**self.kwargs) 379 380 用于 (config_kwargs, self.kwargs, init_kwargs, client_kwargs) 中的参数：

AttributeError：模块“aiobotocore”没有属性“AioSession”

谁能让我知道我在哪里犯了错误或怎么做？

Answer 1

s3fs需要凭据，请参见此处，我认为您必须使用我链接的文档中列出的方法之一提供存储桶的凭据。 但是，还有许多其他方法可以将s3文件读入 sagemaker 笔记本，例如，看看这里。