网络中最大的连接组件

Question

I have a big text file (20 GB) file that contains source-target nodes and threshold.If threshold > 0 is connected otherwise not connected.I want to add the connected nodes in array or array list(which one is suitable for large amount of data?) and to find the giant connected component.I think BFS algorithm is a solution for the shortest path.

Txt file

100 101 -0.3434
100 102  1.0023
100 103  1.100
103 104  0.210
...

My code:

String line = null;
HashMap<Integer,ArrayList<Node>> arr = new HashMap<Integer,ArrayList<Node>>();
BufferedReader reader = new BufferedReader(new FileReader("C:/Users/UserPC/Desktop/output.txt"));
        while((line = reader.readLine()) != null){
            String[] spl = line.split("\\s+");
            //System.out.println(spl[0]+","+spl[1]);
            int source = Integer.parseInt(spl[0]);
            int target = Integer.parseInt(spl[1]);


            arr.computeIfAbsent( source, k -> new ArrayList<>()).add(new Node(target));

        } 

        reader.close();

Answer 1

20 GB will take too much time if you don't implement them correctly btw you can consider a graph data structure and later you apply minimum spanningtree algorithm by adjusting weight so it only find nodes who have weight less or greater than a certain value.

Steps: Step 1: Make one dimensional array of bags which contains a source node, destination and weight/threshold.

    step 1:
    private Bag<Integer>[] array = (Bag<Integer>[]) new Bag[V];
    for all indexes:
    array[i] = new Bag<Integer>();

Create one class which says that your components are connected and what is their threshold: call is connected.

int firstNode = readIn via scanner.
int secondNode = readIn.
Int thresHold = read;
Connected connected = new Conncted(firstNode,secondNode,threshold);



    add all these connected component into array of bag so you have a graph then use minimumspanning tree or anyother algorithm, there are many.