EMR集群在AWS控制台UI上不可见

Question

I created a Cluster using the code :

> StepFactory stepFactory = new StepFactory();

            StepConfig enableDebugging = new StepConfig().withName("Enable Debugging")
                    .withActionOnFailure("TERMINATE_JOB_FLOW").withHadoopJarStep(stepFactory.newEnableDebuggingStep());


            Application spark = new Application().withName("Spark");

            RunJobFlowRequest createClusterParameters = new RunJobFlowRequest().withName("CreateDatamart")
                                            .withReleaseLabel("emr-5.5.0")
                                            .withSteps(enableDebugging)
                                            .withApplications(spark)
                                            .withLogUri("s3://logs/")
                                            .withServiceRole("EMR_DefaultRole")
                                            .withJobFlowRole("EMR_EC2_DefaultRole")
                                            .withInstances(new JobFlowInstancesConfig()
                                                    .withEc2KeyName("keypair")
                                                    .withInstanceCount(3)
                                                    .withKeepJobFlowAliveWhenNoSteps(false)
                                                    .withMasterInstanceType("m3.xlarge")
                                                    .withSlaveInstanceType("m3.xlarge"));

            RunJobFlowResult createCluster = emr.runJobFlow(createClusterParameters);

The cluster gets created. The steps attached to it are also running. But the cluster is not visible in AWS EMR UI. I can see the details under the Events Tab in EMR. Since it is coming under the Events tab, there is not point that it might be created in some other region. (Although i have checked that too) In the EC2 console i can see the containers created for the EMR.

In the other-case if i create a cluster directly from UI it is visible.

Is there some error with the code?

Answer 1

If you are adding IAM user visibility to a new cluster, call RunJobFlow and set VisibleToAllUsers to true, otherwise IAM users cannot view the cluster.

So you'll just need to add the following line before creating your cluster:

createClusterParameters.setVisibleToAllUsers(true);

Note: Is it similar if you are creating an AWS Lambda with the Python SDK :

response1 = emr.run_job_flow(
    Name=CLUSTER_NAME, # more properties [...]
    VisibleToAllUsers=True
)