应该何时使用MapReduce而不是Pig / Hive？

Question

I am not having a problem understand the fact the hive and pig makes the job of a programmer easier. But are there any limitation where one cannot use them and have to rely on map reduce?

When asked this question in an interview, what should be the response like?

Answer 1

As Chirag points out with MR you get more low level control, and thus more potential for optimization. I'd also like to add:

1. Pig and Hive are more for scripts, and thus more volatile and harder to debug. Setting up proper logging and monitoring in MR allows for more robust programs.

2. You don't have to stick to Java MR to do MR, frameworks like Scalding and Cascading streamline a huge amount while still giving you the flexibility to drop down into the lower levels to do optimizations. In fact Scalding is basically the most concise framework you can get, more concise than Pig and Hive - mainly by virtue of it being in Scala.

Answer 2

With MapReduce we have more control, we can do something to increase performance.

Also the skill of the team should be consideration ( like what if they are good only in Java )

Not sure if everything can be described in hive or pig ( like unstructured data )

Below link should be useful.

http://blog.mortardata.com/post/60274287605/pig-vs-mapreduce