[英]Python Version of WordCount is failing on Flink
On CentOS (CentOS 8.0.1905 (64bit)) I tried to run Python(3.6.8) Version of WordCount program on Flink(1.9) as described here .在CentOS(CentOS的8.0.1905(64位)),我试图描述对弗林克(1.9)运行Python(3.6.8)字计数的版本计划在这里。 I got error as below.
我得到如下错误。 The same environment works fine with Java version of WordCount program.
相同的环境适用于 Java 版本的 WordCount 程序。 What is that I am missing here?
我在这里缺少什么? Any help is greatly appreciated.
任何帮助是极大的赞赏。 Error
错误
$ ./bin/flink run -py examples/python/table/batch/word_count.py
Starting execution of program
org.apache.flink.client.program.OptimizerPlanEnvironment$ProgramAbortException
at org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:273)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:205)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1010)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1083)
at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1083)
I think the information you provided does not show root cause.我认为您提供的信息没有显示根本原因。 Generally speaking,we have two ways to run the example in Flink 1.9, one is the answer 1 provided earlier, which is to install the PyFlink from source, and the other is to download the binary package.
一般来说,我们在Flink 1.9中运行示例有两种方式,一种是前面提供的答案1,从源码安装PyFlink,另一种是下载二进制包。 as follows:
如下:
Check the Python python --version It's better to use Python 2.7.6+ for Flink 1.9.检查Python python --version Flink 1.9最好使用Python 2.7.6+。
Download curl -O http://mirrors.gigenet.com/apache/flink/flink-1.9.1/flink-1.9.1-bin-scala_2.11.tgz tar -zxvf flink-1.9.1-bin-scala_2.11.tgz下载 curl -O http://mirrors.gigenet.com/apache/flink/flink-1.9.1/flink-1.9.1-bin-scala_2.11.tgz tar -zxvf flink-1.9.1-bin-scala_2。 11.tgz
Config(Optional) vi flink-1.9.1/conf/flink-conf.yaml -> rest.port: 4000配置(可选) vi flink-1.9.1/conf/flink-conf.yaml -> rest.port: 4000
Start cluster cd flink-1.9.1;启动集群cd flink-1.9.1; bin/start-cluster.sh
bin/start-cluster.sh
Run word count example ./bin/flink run -py examples/python/table/batch/word_count.py运行字数统计示例 ./bin/flink run -py examples/python/table/batch/word_count.py
More detail can be found here: https://enjoyment.cool/2020/01/19/Three-Min-Series-Run-the-Example-of-WordCount-in-PyFlink-1-9/#more更多细节可以在这里找到: https : //enjoyment.cool/2020/01/19/Three-Min-Series-Run-the-Example-of-WordCount-in-PyFlink-1-9/#more
Hoping it is helpful to you!希望对你有帮助!
Thanks to Jincheng Sun for your encouragement and support感谢孙晋诚的鼓励和支持
I moved $FLINK_HOME/opt/python/__MACOSX to one level above.我将 $FLINK_HOME/opt/python/__MACOSX 移到上一层。 Program ran without any errors.
程序运行没有任何错误。 As I am running on Linux, I do not need __MACOSX.
因为我在 Linux 上运行,所以我不需要 __MACOSX。 I will simply delete it
我会简单地删除它
在运行测试之前,您应该使用 pip 构建源代码并安装 PyFlink,详细信息可以在这里找到: https ://ci.apache.org/projects/flink/flink-docs-release-1.9/flinkDev/building 。 html#build-pyflink 。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.