Read HDFS files in eclipse

Question

I have installed hadoop plugin in eclipse.my mapreduce program has iterations and i want to read a file in hdfs in each iteration. I read the file directly from hdfs or i copy the file into local and then read it? which of them is more efficient? Thanks.

Answer 1

It depends entirely what you want to do with it; if you will be reusing the file contents, you might as well just copy it to local to save the repeat lookups to HDFS, but if you just need to read it once, there's no harm in direct from HDFS as copying to local will involve reading it from there anyhow.