Sqoop-将所有表从mysql导入到配置单元

Question

I have three tables in my mysql db:

parent_table with two join tables: foo , bar , where parent_table has many foo 's and bar 's and foo , bar belong to parent_table .

How can I use sqoop, or an alternative method to import these tables into hive to be queried.

Answer 1

Here is the example:

sqoop import-all-tables \
  --connect "jdbc:mysql://quickstart.cloudera:3306/retail_db" \
  --username=retail_dba \
  --password=cloudera \
  --warehouse-dir=/user/hive/warehouse/retail_stage.db

retail_db in --connect clause is mysql database retail_dba in --username clause is mysql user who have access to read tables under retail_db mysql database cloudera in --password clause is password for mysql user retail_dba /user/hive/warehouse/retail_stage.db in --warehouse-dir is hadoop directory (in this case it is hive database, one can have any valid hadoop directory). Above script will create directory for each of the mysql table under /user/hive/warehouse/retail_stage.db.

You can run this script as is in Cloudera Quickstart VM.

Answer 2

You can start by taking a look into Sqoop User Guide that describes how to use Sqoop or more use case oriented book Apache Sqoop Cookbook . Both sources should be able to help you understand what needs to be done for this case.