[英]Flink 1.5.1 on yarn
我正在使用flink 1.5.1,一個工作集群
我用CLI提交了一份作業,然后取消了作業,這是我不能理解的嚴重錯誤
:~/flink-1.5.1/bin$ ./flink run -d -m yarn-cluster -ynm lz_test_alone1 -ytm 4096 -yjm 1024 -c Demo /home/test/flink1.5_demo-1.1-SNAPSHOT.jar
2018-08-10 15:05:28,390 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:28,390 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:28,937 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:28,937 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:29,324 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Cluster specification: ClusterSpecification{masterMemoryMB=1024, taskManagerMemoryMB=4096, numberTaskManagers=1, slotsPerTaskManager=1}
2018-08-10 15:05:29,818 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2018-08-10 15:05:29,824 WARN org.apache.flink.yarn.AbstractYarnClusterDescriptor - The configuration directory ('/home/flink-1.5.1/conf') contains both LOG4J and Logback configuration files. Please delete or rename one of them.
2018-08-10 15:05:31,087 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Submitting application master application_1526888270443_0071
2018-08-10 15:05:31,121 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1526888270443_0071
2018-08-10 15:05:31,121 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Waiting for the cluster to be allocated
2018-08-10 15:05:31,126 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Deploying cluster, current state ACCEPTED
2018-08-10 15:05:35,644 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - YARN application has been deployed successfully.
2018-08-10 15:05:35,644 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - The Flink YARN client has been started in detached mode. In order to stop Flink on YARN, use the following command or a YARN web interface to stop it:
yarn application -kill application_1526888270443_0071
Please also note that the temporary files of the YARN session in the home directory will not be removed.
Job has been submitted with JobID 4c4f9564adc94cd38d91d46660ade0d9
:~/flink-1.5.1/bin$ ./flink cancel 4c4f9564adc94cd38d91d46660ade0d9
2018-08-10 15:05:55,484 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
2018-08-10 15:05:55,484 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - Found Yarn properties file under /tmp/.yarn-properties-sloth.
Cancelling job 4c4f9564adc94cd38d91d46660ade0d9.
2018-08-10 15:05:55,837 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 3
2018-08-10 15:05:55,837 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - YARN properties set default parallelism to 3
YARN properties set default parallelism to 3
2018-08-10 15:05:56,026 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:56,026 INFO org.apache.flink.yarn.cli.FlinkYarnSessionCli - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2018-08-10 15:05:56,116 INFO org.apache.flink.yarn.AbstractYarnClusterDescriptor - Found application JobManager host name 'cluster23.org' and port '57813' from supplied application id 'application_1526888270443_0067'
2018-08-10 15:05:56,707 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
當我使用'flink run'返回我的jobID和applicationId(application_1526888270443_0071)時;
然后我使用'flink cancel'並且日志顯示另一個applicationId(application_1526888270443_0067),所以它出錯了。
有沒有人可以幫助我?
./flink cancel
用於紗線工作需要-該規則應為:
./flink cancel 4c4f9564adc94cd38d91d46660ade0d9 -yid application_1526888270443_0071
這有效
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.