How to copy a file fron windows os to cloudera ? I also want to import the .csv file in to the hive, how can i do that?

Question

How to copy a file from windows OS to cloudera ? I also want to import the .csv file in to the hive, how can i do that ? Please help me to get it done.

Answer 1

To get the file into Linux you will need to either ftp it to the CentOS CLoudera box or mount a file share and copy it from there.

Once on the CentOS/Cloudera box you can open the web browser and log into Cloudera Manager and Hue which is the user interface to Hive/Pig etc. There are icons across the top with the third from the left being Beeswax/Hive. Click on that icon and start from there.

Your other choice once you have the file on the Cloudera box is to use the Hadoop fs commands found HERE specifically copyFromLocal

To load your file into Hive you can do one of two things. Load it into a single large staging field and then parse it into fields you want using offsets like so:

LOAD DATA INPATH '/user/xyz/Inbound/files/target.csv' INTO TABLE 'myTable'

This loads the entire record into a single field. Then you would do the following once loaded.

INSERT OVERWRITE TABLE myTable SELECT
regexp_extract(col_value, '^(?:([^,]*)\,?)(1)', 1) New_Field_name1
regexp_extract(col_value, '^(?:([^,]*)\,?)(5)', 1) New_Field_name2
......
FROM myTable;

The syntax looks challenging but it is not bad, you will just have to read up on HiveQL Hue also allows you to load a CSV into the individual fields but you have to add the field names yourself. I have never done that before though.