创建Sqoop Hive导入作业

Question

I successfully did an ad hoc import from an oracle table into Hive table using Sqoop. Now I want to create and save a Sqoop job with the exact parameters I used for the ad hoc import.

But if I say:

sqoop job --create myjob --import \\ --connect jdbc:mysql://host.port/foo \\ --username myUserName \\ --password-file file:///myfile.pwd \\ --table mytable \\ --create-hive-table \\ --hive-import \\ --hive-database my_hive_db \\ --hive-table my_hive_table \\ --fields-terminated-by '|' \\ --hive-drop-import-delims \\ --m 1

I get an ERROR:

19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Error parsing arguments for job: 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --import 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: --connect 19/03/05 14:56:41 ERROR tool.BaseSqoopTool: Unrecognized argument: jdbc:mysql:@host:port/foo If I delete all the \\ it still doesn't work.

How do I define a Sqoop job with all the necessary parameters for Hive import?

Answer 1

Seems to be a bash escape issue. There should be no space just after the backslack "\\" characters at the end of the line. You can see the extra spaces with:

cat -A sqoop_script.sh 

Answer 2

Not sure if the example you provide is wrong, but I was always using a slightly different syntax where import was not a flag inside a job.

From the documentation , these 2 syntaxes seem familiar to me:

sqoop import (generic-args) (import-args)
sqoop-import (generic-args) (import-args)

Answer 3

You are missing space between -- and import .

Try with below syntax:

sqoop job --create myjob  -- import \
--connect jdbc:mysql://host.port/foo \
--username myUserName \
--password-file file:///myfile.pwd \
--table mytable \
--create-hive-table \
--hive-import \
--hive-database my_hive_db \
--hive-table my_hive_table \
--fields-terminated-by '|' \
--hive-drop-import-delims \
--m 1