Amazon Athena table creation issue

Question

I have a table in Athena which has S3 as a source.

One of the table columns "Col1" has a datatype of string.

Col1
2019-11-21T12:26:13.422Z
2019-11-21T17:12:49.812Z
2019-11-21T17:51:38.299Z

I want this column to be converted to a data type of timestamp .

I ran the below query:

SELECT
  from_iso8601_timestamp(col1) AS ABC
FROM "testtest"."layer2_keywords" 

Then I get a column with timestamp datatype:

col1

    col1
1   2019-11-12 10:11:08.017 UTC
2   2019-11-21 09:19:58.937 UTC
3   2019-11-22 09:23:47.786 UTC

I want to create a new table with this table output.

I tried below query:

CREATE TABLE NEW_TABLE AS
SLECT
  from_iso8601_timestamp(col1) AS ABC
FROM layer2_keywords

But this command is not creating a new table.

It throws error something like below:

NOT_SUPPORTED: Unsupported Hive type: timestamp with time zone. You may need to manually clean the data at location 's3://athena-query-results-layer2/XXXXXXXXXX/' before retrying. Athena will not delete data in your account.

Can any one guide me how do I get a new table from the select query in AWS Athena for a string to timestamp conversion?

I have to rely on Athena as neither Glue crawler nor Glue ETL is transforming the string to timestamp transformation.

Answer 1

I got the answer.

I used CREATE TABLE AS SELECT (CTAS).

See: Use the Results of an Amazon Athena Query in Another Query

But is there a method like a GROK expression that can be used even before data gets saved in S3? If yes please can anyone provide a step by step procedure to write a GROK in Glue for this particular issue?

string to timestamp

String : 2019-11-21T12:26:13.422Z

Answer 2

You could use CAST() function as follow:

CREATE TABLE NEW_TABLE AS
SLECT
  CAST(from_iso8601_timestamp(col1) AS TIMESTAMP) AS ABC
FROM layer2_keywords

I hope it helps you.