简体   繁体   English

AWS Glue和跨区域访问

[英]AWS Glue and Cross Region Access

I have a few questions about AWS Glue that I was hoping someone could answer for me. 我对AWS Glue有一些疑问,希望有人可以为我解答。

So the scenario is we have an AWS Account set up in the us-east-1 region and we use AWS EMR along with Glue as the hive metastore (through emr configurations). 因此,场景是我们在us-east-1区域中设置了一个AWS账户,并使用AWS EMR和Glue作为配置单元元存储(通过emr配置)。 This has been working fine for the past year or so and no hiccups, the Glue Catalog has a bunch of DB's and a bunch of tables. 在过去一年左右的时间里,这一直很好,并且没有打h,Glue目录中有很多DB和很多表。 Now what we want to do is migrate our entire project into another region (same account, different region, eg us-west-1). 现在,我们要做的是将整个项目迁移到另一个区域(相同帐户,不同区域,例如us-west-1)。 So I've moved everything else over, ec2 instances, sg's, etc.. Problem is I don't see the tables from the us-east-1 glue metastore in the us-west-1. 因此,我将所有其他内容移到了ec2实例,sg等上。问题是我看不到us-west-1中us-east-1胶水元存储中的表。 Now for my questions. 现在我的问题。

1.) Are AWS Glue Catalogs region specific? 1.)AWS Glue目录是否特定于区域? Meaning, if i create a db/table in glue in one region, does it stay in that region or is there something I can do to access it across all regions? 意思是,如果我在一个区域中用胶水创建了一个数据库/表,它是否留在该区域中?或者我可以做些什么来在所有区域中访问它? Via Console & Programmatically. 通过控制台和以编程方式。

2.) If there is an ability to share a single glue metastore across regions, how does that work with EMR, do I have to specify a region attribute along with the other glue settings in the EMR configurations page? 2.)如果能够跨区域共享单个胶水元存储,那么它如何与EMR一起使用,我是否必须在EMR配置页面中指定区域属性以及其他胶水设置?

3.) If #2 is possible, are there any charges for accessing glue catalogs across regions (cross-region access) 3.)如果可以使用#2,则跨区域访问胶水目录是否收费(跨区域访问)

3.) Otherwise if #2 is not possible, do I have to manually go copy the db/tables from one region and import them into a glue data catalog in another region? 3.)否则如果#2不可能,我是否必须手动从一个区域复制db /表并将其导入另一区域的胶水数据目录中? Is this the right way to go about doing this? 这是执行此操作的正确方法吗?

Just for anyone who sees this in the future. 只适用于将来看到这种情况的任何人。 I ended up moving tables over using a java program I wrote that uses the AWS Java Glue API's to read the tables from one region and create them in another. 我最终使用我编写的Java程序移动了表,该程序使用AWS Java Glue API从一个区域读取表,然后在另一个区域创建它们。 And then the same code deletes the tables from the original region. 然后相同的代码从原始区域中删除表。 Didn't really see any other way. 真的没有其他办法。 We had about 1000+ tables to move over so manually copying them over wasn't really an option. 我们大约有1000多个表要移动,因此手动复制它们并不是一个真正的选择。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM