从目录中读取文件以创建 ZIP hadoop

Question

I'm looking for Hadoop examples, something more complex than the wordcount example.

What I want to do It's read the files in a directory in Hadoop and get a zip, so I have thought to collect al the files in the map class and create the zip file in the reduce class.

Can anyone give me a link to a tutorial or example than can help me to built it?

I don't want anyone to do this for me, I'm asking for a link with better examples than the wordaccount.

I almost get it, if you need it: https://github.com/flopezluis/testing-hadoop

Answer 1

If your objective is to to normalize the structured data in records, coming in from several inputs and then process it. Based on it, i think you really need to look at this article which helped me in past. It included How To Normalize Data Using Hadoop/MapReduce and provide Java based source code as below:

• Step 1: Extract the column value pairs from the original data.
• Step 2: Extract column-value Pairs Not In Master ID File
• Step 3: Calculate the Maximum ID for Each Column in the Master File
• Step 4: Calculate a New ID for the Unmatched Values
• Step 5: Merge the New Ids with the Existing Master IDs
• Step 6: Replace the Values in the Original Data with IDs

There is another examples about Method for Reading and Writing General Record Structures using new Writable and InputFormat classes in JAVA. Have a look here .