Kafka from Confluent or Cloudera?

Question

My question is easy but complicated, in my current company we are evaluating the use of kafka with confluent or kafka with cloudera.

I have found themes of the confluent version where you are supposed to go with the last one, Ksql and the administration interface. In cloudera you go one more version back, ksql does not seem to have and does not offer any administration interface to this day, I think that in the next version it brings a console for the streams but I do not know how it is so even if it is.

On the other hand the prices at least according to my calculations, confluent is very expensive while in cloudera you pay for kafka but also for all the other tools.

In addition to evaluating the 2 products, I would like to know based on the experiences of the community which are the virtues and weaknesses of each one in order to have a small comparison.

Any input is welcome.

Regards.

Answer 1

@cricket_007 Hello,

Yes, it was said that the answer is more complicated than it seems. From an "opensource" point of view and as a technical profile that I am, I don't need any of the 2, neither cloudera nor confluent since both are based on an apache suite tool, and all new features that they develop is released to the apache suite, KSQL, Kstreams, kafka connect, including The Schema Registry and Kafka REST Proxy are confluent projects but are Apache licensed.

The dilemma is as soon as you want to "industrialize" it in a big company, here as it usually happens in every big company you need a support or some facilities so to speak, that a partner gives you, in this way a platform like cloudera from an "opensource" point of view is also not necessary, since it is still a hadoop with a series of ecosystem tools that you could do a fork of them and extend and maintain them, as even impala himself. In this sense, kafka within Cloudera is the same, it is another component of the suite that you can install.

Both platforms offer the most important things, but it seems clear that confluent, founded by creators and commiters of the tool start with an advantage, the Schema Registry and Kafka REST Proxy are confluent projects, Replicator confluent seems more powerful than the mirrorMaker, the Control Center administrative console, security with JASS, Kerberos and more.

In the previous aspects, today cloudera is far behind, since beyond the parcel that they offer you to install in version 6.3.X there is no more and there it is still another component of cloudera, opensource, in fact in Sections refer to the Apache Kafka website directly, so it is understood the support leaves much to be desired.....

Fans of cloudera will tell me the following: Wait for Cloudera DataFlow, ok yes, it looks wonderful, but because it is not an alpha version, there is nothing, and when you try to download something to try, in the case of kafka You are given the parcel of cloudera. Nifi as maximum integration for everything would give us a lot of versatility, but I insist, until March we have nothing.

My debate or decision would be obvious, confluent is my choice, but the price is sometimes prohibitive and clear, and in the company we already have a cloudera that can manage a kafka, not with such versatility guarantee and security options, and especially support than confluent but It can also be worth it.

So know the opinion of someone who has been lucky to see the 2 platforms in action.

Thank you.