[英]Convert Storm - wordCount topology to use a Kafka Spout
I'm new to Storm and Kafka, and I could manage to install both on a local virtual machine after some time. 我是Storm和Kafka的新手,一段时间后,我可以将它们都安装在本地虚拟机上。 I currently have a working wordCount topology taking sentences from a dropBox text file:
我目前有一个工作的wordCount拓扑,它从dropBox文本文件中提取句子:
public void nextTuple() {
final String APP_KEY = "XXXX";
final String APP_SECRET = "XXXX";
DbxAppInfo appInfo = new DbxAppInfo(APP_KEY, APP_SECRET);
DbxRequestConfig config = new DbxRequestConfig("StormTopology/1.0", Locale.getDefault().toString());
String accessToken = "XXXXXXXX";
DbxClient client = new DbxClient(config, accessToken);
String sentence="";
try {FileOutputStream outputStream = new FileOutputStream("fromSpout.txt");
try {
//client.delete("/*.txt");
DbxEntry.File downloadedFile = client.getFile("/spout.txt", null,outputStream);
sentence= readFile("fromSpout.txt");
if (sentence==null || sentence == "" || sentence == " " || sentence == "/t") {
Utils.sleep(1000);
return;
}
}
catch (DbxException ex) { }
catch (IOException ex) { }
//return 1;
finally {
outputStream.close();
}
}
catch (FileNotFoundException ex){}
catch (IOException ex) {}
if (sentence.length()<2) { Utils.sleep(10000); return; }
try { client.delete("/spout.txt");}
catch (DbxException ex) { }
_collector.emit(new Values(sentence));
Utils.sleep(1000);
Now I want to upgrade my spout to use text from Kafka in order to submit to my next bolt in the topology. 现在,我想升级喷嘴以使用来自Kafka的文本,以便提交到拓扑中的下一个螺栓。 I tried to follow many articles and codes in git without any success.
我试图按照git中的许多文章和代码进行操作,但没有成功。 For example: this kafka spout .
例如: 这个kafka壶嘴 。 Could anyone please help and give me some direction in order to implement the new spout.java file?
有人可以帮忙给我一些指导,以实现新的spout.java文件吗? Thank you!
谢谢!
From storm 0.9.2 release, there is an external storm-kafka package which can do this. 从storm 0.9.2版本开始,有一个外部storm-kafka软件包可以执行此操作。 Actually this package is contributed back to storm community from storm-kafka-0.8-plus .
实际上,此软件包是从storm-kafka-0.8-plus贡献回风暴社区的。 And there is a test project showing its usage.
并且有一个测试项目显示了其用法。
In details, first add dependency to your maven (or lein/gradle): 详细来说,首先将依赖添加到您的Maven(或lein / gradle)中:
groupId: org.apache.storm
artifactId: storm-kafka
version: 0.9.2-incubating
Then create topology and spout like this: 然后创建拓扑和喷口,如下所示:
import storm.kafka
TridentTopology topology = new TridentTopology();
BrokerHosts zk = new ZkHosts("localhost");
TridentKafkaConfig spoutConf = new TridentKafkaConfig(zk, "test-topic");
spoutConf.scheme = new SchemeAsMultiScheme(new StringScheme());
OpaqueTridentKafkaSpout spout = new OpaqueTridentKafkaSpout(spoutConf);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.