[英]How to integrate a python bolt to a java topology for Apache Storm?
我试图将一个简单的python螺栓集成到使用Apache Storm和Storm Crawler SDK创建的已配置风暴拓扑中。 我正在按照此处提供的说明进行操作
但是我不断出错:
java.lang.Exception: Shell Process Exception: Traceback (most recent call last):
File "D:\<PATH>\storm.py", line 217, in run
tup = readTuple()
File "D:\<PATH>\storm.py", line 74, in readTuple
cmd = readCommand()
File "D:\<PATH>\storm.py", line 67, in readCommand
msg = readMsg()
File "D:\<PATH>\storm.py", line 42, in readMsg
return json_decode(msg[0:-1])
File "D:\<PATH>\storm.py", line 30, in <lambda>
json_decode = lambda x: json.loads(x)
File "C:\Users\akumar\AppData\Local\Continuum\Anaconda2\lib\json\__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "C:\Users\akumar\AppData\Local\Continuum\Anaconda2\lib\json\decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\akumar\AppData\Local\Continuum\Anaconda2\lib\json\decoder.py", line 382, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded
at org.apache.storm.task.ShellBolt.handleError(ShellBolt.java:227) [storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt.access$1100(ShellBolt.java:72) [storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:348) [storm-core-1.2.1.jar:1.2.1]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_171]
96398 [Thread-40] ERROR o.a.s.t.ShellBolt - Halting process: ShellBolt died. Command: [python, D:/<PATH>/ClassifyBolt.py], ProcessInfo pid:12708, name:classify exitCode:0, errorString:
java.lang.RuntimeException: org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
at org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) ~[storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:330) [storm-core-1.2.1.jar:1.2.1]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_171]
96399 [Thread-40] ERROR o.a.s.d.executor -
java.lang.RuntimeException: org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
at org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) ~[storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:330) [storm-core-1.2.1.jar:1.2.1]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_171]
我试图通过Storm-Crawler网站在示例爬网程序示例提供程序中添加创建的螺栓。 在我看来,似乎python bolt并没有从拓扑中的先前组件获取流。
有人可以帮忙吗?
这可能是由于您的bolt的python进程将一些日志输出打印到了stdout。 默认情况下,多语言Storm螺栓使用进程的stdin / stdout与Java进程进行通信。
使用pystorm作为Python螺栓基础结构,将允许您使用不同的流进行此通信,例如UNIX命名管道或套接字,并释放stdout进行常规日志记录。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.