简体   繁体   English

如何将数据从 Redshift 卸载到 S3?

[英]How to unload data from Redshift to S3?

I've been trying to unload some data from Redshift to the S3 bucket.我一直在尝试将一些数据从 Redshift 卸载到 S3 存储桶。 Except I've been getting the following error:除了我收到以下错误:

Amazon Invalid operation: cannot drop active portal;亚马逊无效操作:无法删除活动门户; [SQL State=XX000, DB Errorcode=500310] [SQL 状态=XX000,数据库错误代码=500310]

To note the Redshift and S3 are in 2 different buckets so I specified the region within the unload.要注意 Redshift 和 S3 位于 2 个不同的存储桶中,因此我指定了卸载中的区域。

I've tried looking it up, it mentions that I can't drop the table.我试过查找它,它提到我不能放下桌子。

So I created a "test" table that I am 100% certain is not connected to any view or other but I still get the same error.所以我创建了一个“测试”表,我 100% 确定它没有连接到任何视图或其他视图,但我仍然得到同样的错误。

I tried also troubleshooting through AWSQuickSolutions: Redshift Table Can't Be Dropped or Drop Table Hangs except I do not have access to svv_table_info .我还尝试通过AWSQuickSolutions 进行故障排除:Redshift Table Can't Be Dropped 或 Drop Table Hangs除非我无权访问svv_table_info

Is there any way to do the unload?有什么办法可以卸载吗?

reporting @nofinator reply since it worked for me and may be helpful for someone else too:报告@nofinator 回复,因为它对我有用,并且可能对其他人也有帮助:

By running the UNLOAD from a different app, in my case the Query Editor in the Redshift Console, resolved the exception Amazon Invalid operation: cannot drop active portal通过从不同的应用程序运行 UNLOAD,在我的例子中,Redshift 控制台中的查询编辑器解决了异常Amazon Invalid operation: cannot drop active portal

OK, so I ran into the same problem.好的,所以我遇到了同样的问题。 None of the known solutions applied to my issue because:没有任何已知的解决方案适用于我的问题,因为:

1) The cluster was on the same region as the S3 bucket I created 1) 集群与我创建的 S3 存储桶位于同一区域

2) I tried running the UNLOAD command via python, cli, and redshift with the same results. 2) 我尝试通过 python、cli 和 redshift 运行 UNLOAD 命令,结果相同。

3) I tried adding a bucket policy for the redshift role 3) 我尝试为 redshift 角色添加存储桶策略

4) I tried running the unload command using for arns (the redshift role and the s3 role) 4)我尝试使用 for arns(redshift 角色和 s3 角色)运行卸载命令

Finally, I got it to work.最后,我让它工作了。 What changed?发生了什么变化? I was now directly on my company's network instead of using VPN which is when I experienced this issue.我现在直接在我公司的网络上,而不是在我遇到这个问题时使用 VPN。

What finally worked for me:最终对我有用的是:

UNLOAD ('SELECT * FROM table where EXTRACT(YEAR FROM order_datetime_utc )=2019')   
              to 's3://bucket/' 
              REGION 'us-east-1'
                iam_role 'arn:aws:iam::0000000000:role/RedshiftCopyUnload,arn:aws:iam::0000000000:role/S3Access'
              parallel off
              delimiter as '\t' 
gzip;

I had the same issue yesterday.我昨天有同样的问题。 At least in my incident, there was a sev-2 in Data-Warehouse.至少在我的事件中,数据仓库中有一个 sev-2。 I use Amazon DWP that's used by Amazon retail, so not sure if this is the same service used by AWS.我使用亚马逊零售使用的 Amazon DWP,所以不确定这是否与 AWS 使用的服务相同。

tldr; tldr; It's possible this is a server issue within the provider of the redshift.这可能是红移提供商内部的服务器问题。

I had this issue after I modified IAM permissions on the role.在我修改了角色的 IAM 权限后,我遇到了这个问题。 Unfortunately I had to reboot the cluster and then my UNLOAD command worked.不幸的是,我不得不重新启动集群,然后我的 UNLOAD 命令才起作用。

I don't know what exactly caused this error but I was getting it after a failed unload.我不知道究竟是什么导致了这个错误,但我在卸载失败后得到它。

I disconnect then reconnect my sql client and got it to work.我断开连接然后重新连接我的 sql 客户端并让它工作。 not super convenient but better than restarting the cluster.不是超级方便,但比重新启动集群更好。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM