Install Apache Spot

Question

I need to install Apache Spot on Ubuntu.

http://spot.incubator.apache.org/doc/#installation

I have already did the setup of a single node cluster of Hadoop following this guide:

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html

And i already have installed Hive, Kafka and Spark following the guides written in Apache documentation. The main problem is that I'm not able to configure the file '/spot-setup/spot.conf' properly because when i run the command:

./hdfs_setup.sh

the Terminal displays:

sudo: unknown user: hdfs
sudo: unable to initialize policy plugin
./hdfs_setup.sh:line 48:hdfs:command not found
./hdfs_setup.sh:line 52:hdfs:command not found
./hdfs_setup.sh: line 62:impala-shell:command not found

my /etc/hosts file is:

127.0.0.1 localhost
127.0.1.1 osboxes
127.0.0.2 node03
127.0.0.3 node04
127.0.0.4 node16

Which values should I write in the rows of spot.conf? Thank you very much.

Answer 1

Script output implies that hadoop is not properly configured in that node. Instead of installing and configuring the dependencies individually you can try cloudera quickstart vm which packages all the dependencies required for Apache Spot.