在apache flink中使用DataSet的collect()函数的问题

Question

I am trying to calculate the AdamicAdar index of relations in a Social Media following graph. I set up my Edges, Vertices, Dataset and Graph, using apache flink-gelly lirbrarie. Here is my code:

    import org.apache.flink.api.java.ExecutionEnvironment;
    import org.apache.flink.api.java.operators.DataSource;
    import org.apache.flink.api.java.tuple.Tuple2;
    import org.apache.flink.graph.Graph;
    import org.apache.flink.graph.library.similarity.AdamicAdar;
    import org.apache.flink.types.NullValue;
    import org.apache.flink.types.StringValue;
    
    import java.util.List;
    
    public class MyMain {
    
        public static void main(String[] args) {
    
            ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
            DataSource> edgeDataSet = env.
                    readCsvFile(String.valueOf(MyMain.class.getResource("dataset/edges.csv"))).
                    types(StringValue.class, StringValue.class);
            Graph graph = Graph.fromTuple2DataSet(edgeDataSet, env);
    
            List list = null;
            try {
                list = graph.run(new AdamicAdar()).collect();
            } catch (Exception e) {
                e.printStackTrace();
            }
    
            System.out.println(list.get(0));
        }
    }

and here is the error I get:

    Exception in thread "main" java.lang.NoSuchMethodError: org.apache.flink.configuration.ConfigUtils.decodeListFromConfig(Lorg/apache/flink/configuration/ReadableConfig;Lorg/apache/flink/configuration/ConfigOption;Lorg/apache/flink/util/function/FunctionWithException;)Ljava/util/List;
        at org.apache.flink.client.cli.ExecutionConfigAccessor.getJars(ExecutionConfigAccessor.java:75)
        at org.apache.flink.client.deployment.executors.PipelineExecutorUtils.getJobGraph(PipelineExecutorUtils.java:61)
        at org.apache.flink.client.deployment.executors.LocalExecutor.getJobGraph(LocalExecutor.java:98)
        at org.apache.flink.client.deployment.executors.LocalExecutor.execute(LocalExecutor.java:79)
        at org.apache.flink.api.java.ExecutionEnvironment.executeAsync(ExecutionEnvironment.java:962)
        at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:878)
        at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:862)
        at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
        at MyMain.main(MyMain.java:23)
    
    Process finished with exit code 1

Also this is a part of edges.csv file I use:

    5 122
    5 156
    5 158
    5 169
    5 180
    5 187
    5 204
    5 213
    5 235
    5 315
    5 316
    6 89
    6 95
    6 147
    6 219
    6 319
    7 22

in which 5 316 means vertex number five is connected to vertex number 216 and this defines an edge.

And here is my pom.xml file pom.xml

Answer 1

A NoSuchMethodError usually means there's a version mismatch in your dependencies. Looking at your pom.xml you've got what appears to be a mixture of different versions of your Flink related libraries.

As an aside, I also noticed that your .csv is not comma separated. You may want to simplify the program and verify that edgeDataset is actually a 2-Tuple and not a single value.