需要有关将数据从Mysql导入到HDFS的帮助

Question

I'm trying to import the data from MySQL DB to HDFS using SQOOP. The SQOOP import statement executed fine but when I looked at the HDFS file, some of the fields are showing as \\ character at end of the field (because the MYSQL table field is having a character , between two words but same field). Can you help me how to import this field to HDFS as same as Mysql table field?.

Example: MySQL table field (name) is having value as (sami, ramesh) and when I imported it to HDFS using SQOOP, I see this field as sami/ ramesh (different fields like 2 fields). I wanted to have it as sami, ramesh (as a single field like mySQL field) in HDFS.

Is there any way to do that?. Thanks a lot in advance for your help.

Answer 1

Without seeing the import statement that you tried, I am not 100% sure about why is that happening... But perhaps the reason is that you are using the default delimiter for fields, which is precisely a comma, so if you want to keep it, you should use a different delimiter, like for example:

--fields-terminated-by \t

Another option would be to use --enclosed-by '\\"' , and it would look like "sami, ramesh".

Hope that helps! If it doesn't, please post your import statement to see where the problem is.

Answer 2

Typical sqoop cannot resolve your issue. You need to use query (instead of table) and replace ',' while importing the data.

--query <statement>

eg

--query select replace(name, ',', ' '), .... from ....