Populate external schema table in Redshift from S3 bucket file

Question

I am new to AWS and trying to figure out how to populate a table within an external schema, residing in Amazon Redshift. I used Amazon Glue to create a table from a .csv file that sits in a S3 bucket. I can query the newly created table via Amazon Athena.

Here is where I am stuck because my task is to take the data and populate a table living in an RedShift external schema. I tried created a Job within Glue, but had no luck.

This is where I am stuck. Am I supposed to first create an empty destination table that mirrors the table that I can query using Athena?

Thank you to anyone in advance who might be able to assist!!!

Answer 1

Redshift Spectrum and Athena both use the Glue data catalog for external tables. When you create a new Redshift external schema that points at your existing Glue catalog the tables it contains will immediately exist in Redshift.

-- Create the Redshift Spectrum schema
CREATE EXTERNAL SCHEMA IF NOT EXISTS my_redshift_schema
FROM DATA CATALOG DATABASE 'my_glue_database'
IAM_ROLE 'arn:aws:iam:::role/MyIAMRole'
;
-- Review the schema info
SELECT * 
FROM svv_external_schemas 
WHERE schemaname = 'my_redshift_schema'
;
-- Review the tables in the schema
SELECT * 
FROM svv_external_tables 
WHERE schemaname = 'my_redshift_schema'
;
-- Confirm that the table returns data
SELECT * 
FROM my_redshift_schema.my_external_table LIMIT 10
;