簡體   English   中英

通過AWS Java SDK設置EMR

[英]Setup EMR via AWS Java SDK

下面是我的代碼來設置EMR集群:

public void createCluster(){
        AWSCredentials credentials = new BasicAWSCredentials("xxx","xxx");
        AmazonElasticMapReduceClient emr = new AmazonElasticMapReduceClient(credentials);

        StepFactory stepFactory = new StepFactory();

        StepConfig enableDebugging = new StepConfig()
            .withName("Enable Debugging")
            .withActionOnFailure("TERMINATE_JOB_FLOW")
            .withHadoopJarStep(stepFactory.newEnableDebuggingStep());

        StepConfig installHive = new StepConfig()
            .withName("Install Hive")
            .withActionOnFailure("TERMINATE_JOB_FLOW")
            .withHadoopJarStep(stepFactory.newInstallHiveStep());

       /* StepConfig hiveScript = new StepConfig().withName("Hive Script")
            .withActionOnFailure("TERMINATE_JOB_FLOW")
            .withHadoopJarStep(stepFactory.newRunHiveScriptStep("s3://mywordcountbuckett/binary/WordCount.jar"));
            */

        RunJobFlowRequest request = new RunJobFlowRequest()
            .withName("Hive Interactive")
            .withAmiVersion("3.8")
            .withSteps(enableDebugging, installHive)

            .withLogUri("s3://mywordcountbuckett/")
            .withServiceRole("EMR_DefaultRole")
            .withJobFlowRole("EMR_EC2_DefaultRole")
            .withInstances(new JobFlowInstancesConfig()
                .withEc2KeyName("mykeypair")
                .withInstanceCount(5)
                .withKeepJobFlowAliveWhenNoSteps(true)
                .withMasterInstanceType("m3.xlarge")
                .withSlaveInstanceType("m1.large"));

        RunJobFlowResult result = emr.runJobFlow(request);
        System.out.println("This is result: " + result.toString());

println打印出以下內容:

This is result: {JobFlowId: j-10BX80MGWF3XQ}

這是否意味着我已經成功創建了集群? 但是,當我嘗試從aws.amazon.com查看時,沒有創建任何集群。

在EC2中也沒有創建任何內容。

這是地區問題。

放置此代碼,一切都將相應地工作:

emr.setRegion(Region.getRegion(Regions.US_WEST_2));

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM