Cloudera - Start services - NameNode not formatted

Question

I have Cloudera cluster on Miscrosoft azure. I shut down the cluster every weekend by following the following steps:

1) Turn of Cloudera Management Service from Cloudera Manager

2) Stop all services from Cloudera Manager

3) Go to the Name node and run the command:

service cloudera-scm-server stop

4) Turn the VMs on the cluster off.

When I try to start this cluster I following steps:

1) Start the VMs on the cluster

2) the service "cloudera-scm-server" is already started on startup.

3) Turn on Cloudera management service from CM

4) Turn on all the services.

This is where it fails and gives me the errors that it wasn't able to start the name node because it wasn't formatted.

If I do go and format the name node from CM, all the services start but it wipes off all the contents of HDFS (at least the Name node entries, not sure about the data).

What am I doing wrong here and what's the best way to turn a Cloudera cluster on/off ?

Answer 1

If you deployed the Cloudera EDH cluster using the Azure Marketplace offering or the template in GitHub ( https://github.com/Azure/azure-quickstart-templates/tree/master/cloudera-on-centos ), the following procedure has proved working for shutting down the cluster -

1. In Cloudera Manager, stop cluster services
2. In Cloudera Manager, stop Cloudera management service
3. In Azure portal, stop the VMs

Reverse the order to re-start the cluster.