drop column from a partition in hive external table

Question

I have a hive external table with 3 partition columns (A,B,C) and now I want to drop B and C columns from the partition.Is it possible to do so? I have tried with Alter table tab_name drop column col_name; --- but it throws an error stating partitioned columns cannot be dropped.

Answer 1

To drop partition columns the table should be recreated . The steps are:

1. Drop table, dropping external table will not drop data files.
2. Reorganize data folders to reflect new partition structure. Partitions are folders on physical level, hierarchically organized. If you delete upper level partition, then all sub-folders should be moved to the upper level and so on. if you are deleting two upper partition columns and only one is left then it should be only one level subfolders under the table location.
3. Create table with new partitioning schema on top of old location.
4. Run MSCK repair table . It will create partition metadata for all found partitions folders.

If all of these steps seem too complex or too difficult to do, then simply create new table and load data :

1. Create new table with new partitioning schema.
2. Load data into new table.
3. drop old table and rename new one

Like this:

set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
insert overwrite table new_table partition(C)
select --list columns without deleted 
from old_table;

And finally, after dropping old table, you can rename new one using ALTER TABLE table_name RENAME TO new_table_name .