Updating hive table with sqoop from mysql table

Question

I have already a hive table called roles. I need to update this table with info coming up from mysql. So, I have used this script think that it will add and update new data on my hive table:`

sqoop import --connect jdbc:mysql://nn01.itversity.com/retail_export --username retail_dba --password itversity \ --table roles --split-by id_emp --check-column id_emp --last-value 5 --incremental append \ --target-dir /user/ingenieroandresangel/hive/roles --hive-import --hive-database poc --hive-table roles

Unfortunately, that only insert the new data but I can't update the record that already exits. before you ask a couple of statements:

• the table doesn't have a PK

• if i dont specify --last-value as a parameter I will get duplicated records for those who already exist.

How could I figure it out without applying a truncate table or recreate the table using a PK? exist the way?

thanks guys.

Answer 1

Hive does not operate update queries. You have to drop/truncate old table and reload again.