通过 SOCKS Proxy 连接到 Kafka

Question

I have a Kafka cluster running on AWS. I want to connect to the cluster with the standard kafka-console-consumer from my application server. The application server has access to the internet via a SOCKS-Proxy. No authentication is required

How do I tell the Kafka client to connect through the proxy?

I tried many things, including adding the standard JVM Proxy parameters to the kaka-run-class.sh script. But I still get a connection refused error.

exec $JAVA -DsocksProxyHost=proxy.host.name -DsocksProxyPort=1080 $KAFKA_HEAP_OPTS $KAFKA_JVM_PERFORMANCE_OPTS $KAFKA_GC_LOG_OPTS $KAFKA_JMX_OPTS $KAFKA_LOG4J_OPTS -cp $CLASSPATH $KAFKA_OPTS "$@"

Edit: It turns out, Kafka uses the java.nio library. It's socket implementation is not compatible with proxies. So the jvm parameters I used have no effect on the Kafka client.

Answer 1

As mentioned in comment and edit, the asker already found the solution/root cause:

It turns out, Kafka uses the java.nio library. It's socket implementation is not compatible with proxies. So the jvm parameters I used have no effect on the Kafka client.