NoSuchMethodError in Elasticsearch5.x flink connector

Question

I'm facing some problem while running my flink program, here is the error:

java.lang.NoSuchMethodError: io.netty.buffer.CompositeByteBuf.addComponents(ZLjava/lang/Iterable;)Lio/netty/buffer/CompositeByteBuf;
    at org.elasticsearch.transport.netty4.Netty4Utils.toByteBuf(Netty4Utils.java:78)
    at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:449)
    at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:91)
    at org.elasticsearch.transport.TcpTransport$ScheduledPing.doRunInLifecycle(TcpTransport.java:261)
    at org.elasticsearch.common.util.concurrent.AbstractLifecycleRunnable.doRun(AbstractLifecycleRunnable.java:67)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527)
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)

i looked on SO and find someone had the same problem, and the suggestion was to shade netty and i included it into my pom:

Here is what i added:

<build>
                <plugins>
                    <!-- disable the exclusion rules -->
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-shade-plugin</artifactId>
                        <version>2.4.1</version>
                        <executions>
                            <execution>
                                <phase>package</phase>
                                <goals>
                                    <goal>shade</goal>
                                </goals>
                                <configuration>
                                    <artifactSet>
                                        <excludes combine.self="override">

                                            <exclude>io.netty:netty-all</exclude>
                                            <exclude>io.netty:netty</exclude>

                                        </excludes>
                                    </artifactSet>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

But the problem is still persisting. If someone have an idea, please share with me, thanks.

Answer 1

Probably a version problem. A library inside another library that you add overrides yet another library. i mean: you have 2 dependency in pom:a and b.dependency a has a different version of b so it overrides library b that you added to pom.