I have Athena and Athena Iceberg tables partitioned by multiple columns. I want to create a logic in Python script to break data if have more than 100 total partitions to insert the data without errors.
SHOW PARTITIONS table_name
lists all the partitions but I need the count for them. I need it to be generic to get the count for any table.
I tried select count(*) from information_schema.__internal_partitions__ WHERE table_schema = 'db_name' AND table_name = 'table_name'
but it doesn't work.
Any workarounds?
This works for Athena Engine V2:
SELECT count(*) from "table_name$partitions"
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.