Apache Storm拓扑部署

Question

I am a novice in Apache Storm. I have created a storm project in java in intellij and it successfully creates a local cluster and submits the topology to it and runs locally. I want to run this storm project on amazon EC2. I have followed the link https://github.com/nathanmarz/storm-deploy/wiki . Following the link successfully launched 2 intances of Supervisor, 1 instance of Zookeeper and 1 instance of Nimbus. Now I want to run my topology on server. This is my first project in storm so please answer my following question:

• Will I have to launch another instance to run storm project?
• Do I have to create a jar of the storm java project and scp it to server to run it as they have mentioned in their docs storm jar path/to/allmycode.jar org.me.MyTopology arg1 arg2 arg3 ?

Answer 1

Yes, you need to create a jar file that contains your spouts/bolts classes (and an entry point class containing a main method if you want to submit via bin/storm jar command).

Manually uploading the jar file is not required . If you submit from an IDE or via bin/storm jar command, the jar is uploaded automatically.

Answer 2

To answer your second question. This is the way that I've solved the problem up to this point. Manually scp'd the jar to the Amazon instance and uploaded through the command line client.

There seems to be an open issue in creating a REST API to upload that way. I don't really the status on that issue though.

Here is another stackoverflow question dealing with remote submission, but I have not tried using this technique.

Answer 3

first of all you must compile your topology in cluster mode, then deliver it to nimbus(does not matter how, via scp or another one, i have used WinScp), then run it with command bin/storm jar path/to/jar mainClass.java args so, 1) answer to the first question. no, you have to run your topology on nimbus, not another machine. 2) answer to the second one. yes, you must deliver your jar to nimbus(does not matter how), and run it as written in instruction(documentation)