仅将 s3 分区文件之一添加到 AWS Glue

Question

I'm having slight issues when it comes to running a crawler through my s3 buckets. My folders have data that was dumped from redshift that was sliced into many different files. These files naming convention go as the following:

dump_0000_part_00.gz, dump_0001_part_01.gz ....

However when my crawler fetches the metadata in this folder, it instead makes a few hundred tables, thinking each one of these sliced files is its own table. Is there a way to tell the crawler to group all these sliced files into ONE catalog table?

Answer 1

配置爬虫（或编辑现有爬虫）时，在Output部分下，展开Grouping behavior for S3 data (optional)并选择Create a single schema for each S3 path