如何在Sqoop中为增量数据指定多列？

Question

I am using following query to fetch incremental data in sqoop-

bin/sqoop job --create JOB_NAME -- import  --connect jdbc:oracle:thin:/system@HOST:PORT:ORACLE_SERVICE --username USERNAME --password-file /PASSWORD_FILE.txt --fields-terminated-by ',' --enclosed-by '"'  --table SCHEMA.TABLE_NAME --target-dir /TARGET_DIR -m 2 --incremental append --check-column NVL(UPDATE_DATE,INSERT_DATE) --last-value '2019-01-01 00:00:00.000'  --split-by PRIMARY_KEY --direct

It throwing error for Multiple columns in --check-columns parameters.

Is there any approcach to specify multi columns in --check-column parameter?

I want to fetch data , if UPDATE_DATE field contains null value then it should fetch the data on the basis of INSERT_DATE column.

I want to extract transaction records from a table which is being updated daily , and if the records is inserted first time then there is no value in UPDATED_DATE column. That's why I need to compare both columns while fetching data from table.

Any help regarding this would be highly appreciated.

Answer 1

As per my understanding it doesn't look like it's possible to have 2 check columns when doing incremental imports, so the only way we can manage to get this done is with 2 separate imports:

1. Incremental import with the Insert date as check column for first time records
2. Incremental import with the updated column as check column for UPDATED records