installing HDFS for use with SHARK without YARN

Question

I'm trying to install Apache Shark. One of the requirement is to have HDFS installed. I don't want to use YARN or MESOS. I just want HDFS. My question is: Does this mean I can only install hadoop distribution prior to 2.x? If so, which one?

or can I use Hadoop 2.4 and somehow disable YARN? I'm not really sure how to do this. All the tutorial I can find seem to use YARN. does anyone have any suggestion on how to use HDFS only with 2.x? which hadoop distribution do I need if just want to install HDFS?

Answer 1

To use shark you need to install :

• HDFS to store data
• Hive
• Spark for in-memory usage

Yarn is not mandatory