How can I optimize the intersections between lists with two elements and generate a list of lists without duplicates in python?

Question

I need help with my loop. In my script I have two huge lists (~87.000 integers each) taken from an input file.

Check this example with a few numbers:

We have two lists:

nga = [1, 3, 5, 34, 12]

ngb = [3, 4, 6, 6, 5]

The order of these two lists matters because each position is related with the same position in the other list, so 1 in nga is related with 3 in ngb , 3 with 4 , etc...

So what I want is this output:

listoflists = [[1, 3, 4], [5, 6, 12, 34]]

What I have so far is this loop:

listoflists = []

for p in range(0, len(nga)):
    z = [nga[p], ngb[p]]
    for a, b in zip(nga, ngb):
        if a in z:
            z.append(b)
        else:
            pass
        if b in z:
            z.append(a)
        else:
            pass
    listoflists.append(z)

The problem appears when I used the whole lists, because it crashed and give me a Segmentation fault error. So, what can I do?

Thanks in advance.

Answer 1

I solved my problem with this beautiful function:

net = []
for a, b in zip(nga, ngb):
    net.append([a, b])

def nets_super_gen(net):
    not_con = list(net)
    netn = list(not_con[0])
    not_con.remove(not_con[0])
    new_net = []
    while len(netn) != len(new_net):
        new_net = list(netn)
        for z in net:
            if z[0] in netn and z[1] not in netn:
                netn.append(z[1])
                not_con.remove(z)
            elif z[0] not in netn and z[1] in netn:
                netn.append(z[0])
                not_con.remove(z)
            try:
                if z[0] in netn and z[1] in netn:
                    not_con.remove(z)
            except ValueError:
                pass
    return(netn, not_con)

list_of_lists, not_con = nets_super_gen(net)