使用Java SDK从AWS EMR集群获取主公共DNS值

Question

I need to obtain the master public DNS value via the Java SDK. The only information that I'll have at the start of the application is the ClusterName which is static.

Thus far I've been able to pull out all the other information that I need excluding this and this, unfortunately is vital for the application to be a success.

This is the code that I'm currently working with:

List<ClusterSummary> summaries = clusters.getClusters();
        for (ClusterSummary cs: summaries) {
            if (cs.getName().equals("test") && WHITELIST.contains(cs.getStatus().getState())) {
                ListInstancesResult instances = emr.listInstances(new ListInstancesRequest().withClusterId(cs.getId()));
                clusterHostName = instances.getInstances().get(0).toString();
                jobFlowId = cs.getId();
            }
        }

I've removed the get for PublicIpAddress as wanted the full toString for testing. I should be clear in that this method does give me the DNS that I need but I have no way of differentiating between them.

If my EMR has 4 machines, I don't know which position in the list that Instance will be. For my basic trial I've only got two machines, 1 master and a worker. .get(0) has returned both the values for master and the worker on successive runs.

The information that I'm able to obtain from these is below - my only option that I can see at the moment is to use the 'ReadyDateTime' as an identifier as the master 'should' always be ready first, but this feels hacky and I was hoping on a cleaner solution.

    {Id: id,
Ec2InstanceId: id,
PublicDnsName: ec2-54--143.compute-1.amazonaws.com,
PublicIpAddress: 54..143,
PrivateDnsName: ip-10--158.ec2.internal,
PrivateIpAddress: 10..158,
Status: {State: RUNNING,StateChangeReason: {},
Timeline: {CreationDateTime: Tue Feb 21 09:18:08 GMT 2017,
ReadyDateTime: Tue Feb 21 09:25:11 GMT 2017,}},
InstanceGroupId: id,
EbsVolumes: []}

{Id: id,
Ec2InstanceId: id,
PublicDnsName: ec2-54--33.compute-1.amazonaws.com,
PublicIpAddress: 54..33,
PrivateDnsName: ip-10--95.ec2.internal,
PrivateIpAddress: 10..95,
Status: {State: RUNNING,StateChangeReason: {},
Timeline: {CreationDateTime: Tue Feb 21 09:18:08 GMT 2017,
ReadyDateTime: Tue Feb 21 09:22:48 GMT 2017,}},
InstanceGroupId: id
EbsVolumes: []}

Answer 1

Don't use ListInstances. Instead, use DescribeCluster, which returns as one of the fields MasterPublicDnsName.

Answer 2

To expand on what was mentioned by Jonathon:

        AmazonEC2Client ec2 = new AmazonEC2Client(cred);
    DescribeInstancesResult describeInstancesResult = ec2.describeInstances(new DescribeInstancesRequest().withInstanceIds(clusterInstanceIds));
    List<Reservation> reservations = describeInstancesResult.getReservations();
    for (Reservation res : reservations) {
        for (GroupIdentifier group : res.getGroups()) {
            if (group.getGroupName().equals("ElasticMapReduce-master")) { // yaaaaaaaaah, Wahay!
                masterDNS = res.getInstances().get(0).getPublicDnsName();
            }
        }
    }

Answer 3

 AWSCredentials credentials_profile = null;
 credentials_profile = new 
 DefaultAWSCredentialsProviderChain().getCredentials();

 AmazonElasticMapReduceClient emr = new 
 AmazonElasticMapReduceClient(credentials_profile);
    Region euWest1 = Region.getRegion(Regions.US_EAST_1);
    emr.setRegion(euWest1);
 DescribeClusterFunction fun = new DescribeClusterFunction(emr);
    DescribeClusterResult res = fun.apply(new 
 DescribeClusterRequest().withClusterId(clusterId));
    String publicDNSName =res.getCluster().getMasterPublicDnsName();

Below is the working code to get the public DNS name.