Hive - insert into table partition throwing error

Question

I am trying to create a partitioned table in Hive on spark and load it with data available in other table in Hive. I am getting following error while loading the data:

Error: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.Table.ValidationFailureSemanticException: Partition spec {cardsuit=, cardcolor=, cardSuit=SPA, cardColor=BLA} contains non-partition columns;

following are the commands used to execute the task:-

create table if not exists hive_tutorial.hive_table(color string, suit string,value string) comment 'first hive table' row format delimited fields terminated by '|' stored as TEXTFILE;

LOAD DATA LOCAL INPATH 'file:///E:/Kapil/software-study/Big_Data_NoSql/hive/deckofcards.txt' OVERWRITE INTO TABLE hive_table; --data is correctly populated(52 rows)

SET hive.exec.dynamic.partition = true;

SET hive.exec.dynamic.partition.mode = nonstrict;

create table if not exists hive_tutorial.hive_table_partitioned(color string, suit string,value int) comment 'first hive table' partitioned by (cardSuit string,cardColor string) row format delimited fields terminated by '|' stored as TEXTFILE;

INSERT INTO TABLE hive_table_partitioned PARTITION (cardSuit,cardColor) select color,suit,value,substr(suit, 1, 3) as cardSuit,substr(color, 1, 3) as cardColor from hive_table;

--alternatively i tried
INSERT OVERWRITE  TABLE hive_table_partitioned PARTITION (cardSuit,cardColor) select color,suit,value,substr(suit, 1, 3) as cardSuit,substr(color, 1, 3) as cardColor from hive_table;

sample of data:-

BLACK|SPADE|2

BLACK|SPADE|3

BLACK|SPADE|4

BLACK|SPADE|5

BLACK|SPADE|6

BLACK|SPADE|7

BLACK|SPADE|8

BLACK|SPADE|9

I am using spark 2.2.0 and java version 1.8.0_31.

I have checked and tried answers given in similar thread but could not solve my problem:- SemanticException Partition spec {col=null} contains non-partition columns

Am I missing something here?

Answer 1

After carefully reading the scripts above while creating tables ,value column is type of int in partitioned table where as string in original. Error was misleading !!!