AWS EMRクラスタのステップのステータスを取得するにはどうすればよいですか？

Question

EMRクラスタを作成してスパークジョブを実行しようとしています。

私のジョブが完了すると、EMRクラスターが起動して実行されているので、ステップが完了したことを知ることができるように、スレッドを開始する必要があります。

Exception in thread "main" com.amazonaws.services.elasticmapreduce.model.InvalidRequestException: Step id 'null' is not valid. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: InvalidRequestException; Request ID: 98e1b43e-a440-11e7-920b-158124595c35) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1588) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1258) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1030) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:742) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:716) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) 
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) 

がどのように私は、EMRクラスタ配置内のステップのステータスを取得することができます：次のエラーを取得

while(true) { 
       DescribeClusterRequest desc = new DescribeClusterRequest() 
       .withClusterId(jobFlowId); 
       DescribeClusterResult clusterResult = emrClient.describeCluster(desc); 
       com.amazonaws.services.elasticmapreduce.model.Cluster cluster = clusterResult.getCluster(); 

       DescribeStepRequest d = new DescribeStepRequest().withClusterId(jobFlowId); 
       DescribeStepResult r = emrClient.describeStep(d); 

       Step ss = r.getStep(); 

       String status = ss.getStatus().getState(); 

       System.out.printf("Status: %s\n", status); 
       if(status.equals(ClusterState.TERMINATED.toString()) || status.equals(ClusterState.TERMINATED_WITH_ERRORS.toString())) { 
       System.out.println("Terminated"); 
        break; 
       } 
       try { 
       TimeUnit.SECONDS.sleep(30); 
       } catch (InterruptedException e) { 
       e.printStackTrace(); 
       } 
       // maybe other handle 
      } 

：

{ 
    AmazonElasticMapReduceClient emr = configureEMRClient(); 

    StepFactory stepFactory = new StepFactory(); 

    StepConfig enableDebugging = new StepConfig() 
     .withName("Enable Debugging") 
     .withActionOnFailure("TERMINATE_JOB_FLOW") 
     .withHadoopJarStep(stepFactory.newEnableDebuggingStep()); 

    HadoopJarStepConfig runExampleConfig = new HadoopJarStepConfig() 
     .withJar("command-runner.jar") 
     .withArgs("spark-submit") 
     .withArgs("--deploy-mode","cluster") 
     .withArgs("--master","yarn") 
     .withArgs("--class", extractorMainClass) 
     .withArgs("--num-executors","3") 
     .withArgs("--driver-memory","8g") 
     .withArgs(resourceExtractorJar) 
     .withArgs("st") 
     .withArgs("ap"); 

    StepConfig customExampleStep = new StepConfig() 
      .withName("Example Step") 
      .withActionOnFailure("TERMINATE_JOB_FLOW") 
      .withHadoopJarStep(runExampleConfig); 

    Application spark = new Application().withName("Spark"); 

    RunJobFlowRequest request = new RunJobFlowRequest() 
      .withName("Test") 
      .withReleaseLabel("emr-5.5.0") 
      .withSteps(enableDebugging, customExampleStep) 
      .withApplications(spark) 
      .withServiceRole("EMR_DefaultRole") 
      .withJobFlowRole("EMR_EC2_DefaultRole") 
      .withInstances(new JobFlowInstancesConfig() 
        .withEc2KeyName("keypair") 
        .withInstanceCount(2) 
        .withKeepJobFlowAliveWhenNoSteps(false) 
        .withMasterInstanceType("m3.xlarge") 
        .withSlaveInstanceType("m3.xlarge"));  

    RunJobFlowResult result = emr.runJobFlow(request); 

} 

は、ステップのステータスを取得するには？私は考え出したので、

Answer 1

、質問に答える：

ListStepsResult steps = emrClient.listSteps(new ListStepsRequest().withClusterId(jobFlowId)); 
    StepSummary step = steps.getSteps().get(0); 
    System.out.println(step.getStatus().getState());