spark hive 获取外部分区文件位置

Question

I'm trying to get the external table file location of a partition calculated at run time. Simply alter table drop wouldn't work since it's external. The closest I can get is

 spark.sql(s"describe $tableName partition ($partitionBy=$partitionValue)")

But this would fail when partitionValue is of type timestamp and directly toString before calling the above function. Is there a way to use the same function where spark.write.saveastable() use to create the file path? or is there a way to get the location of the data by partition at runtime?

Answer 1

Try something

show table extended FROM $tableSchema like $tableName partition ($partitionBy=$partitionValue)

See more here: https://spark.apache.org/docs/3.0.0/sql-ref-syntax-aux-show-table.html