Create counter in Storm Bolt

Question

I tried to create a counter and count how many times I received the tuple in the bolt. But after doing this, :

public class CounterBolt extends BaseRichBolt {
    OutputCollector outputCollector;
    int count;

    @Override
    public void prepare(Map<String, Object> topoConf, TopologyContext context, OutputCollector collector) {
        outputCollector = collector;
    }

    @Override
    public void execute(Tuple tuple) {
        if (tuple.getSourceStreamId().equals("GotResult")) {
            count++;
        } else
            System.out.println(count); //check count
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {}

I realise that count will become 0 because of new instance of CounterBolt is created every time in Storm topology.

An approach that I could think of is using an external storage to do this maybe using a database or message broker like Redis to store the counter. My Java knowledge is still not advanced level so is there any proper way to do this?

Answer 1

The right way to have metrics from your topology is to create Metrics consumer, which collects your programmatically defined stats from all Spouts and Bolts of your topology. Take look at the description in the docs . Yes, you need something to store your +1 data, like Redis, MySQL etc. If you are not an experienced programmer take look at the great projects that use Storm like StromCrawler . This is a cool example of how to create own metrics consumer for storing them in Mysql