目录比较

Question

I have a question in relation to comparing directories. I have requested a transfer of data from one company server, I received this data in 8 portable hard drives as it was not possible due to large volume to put it on one. each of them contains about 1TB data. Now, I want to make sure that all the files that were on that company server were fully transferred, and if there are any files missing I want to be able to detect it and ask for them. The issue is that the only thing I received from the company is one txt file inn which there is detailed directory structure saved in a tree format. In principle great I could just look one by one through it but due to large amount of data that is just not achievable. I can generate the same directory list out of every single one of the 8 drives that I received. but how can I ten compare this one file into those 8 files? I tried different python comparison codes to parse through line by line but it does not work as this compares them string by string(line by line) but the are not in string format, they are in tree style format. Anyone has any suggestions how to do it?is there a way to convert the file of a tree format into a string format to then run it in Python program and compare? Or should I request (not sure if thats possible) another file with directories saved in different structure format than tree? if yes how this should be generated?

I tried to parse it through lists in python

Answer 1

Your task consists of two subtasks:

1. Prepare data;
2. Compare trees.

Task 2 can be solved by this Tree class:

from collections import defaultdict


class Tree:
    def __init__(self):
        self._children = defaultdict(Tree)

    def __len__(self):
        return len(self._children)

    def add(self, value: str):
        value = value.removeprefix('/').removesuffix('/')
        if value == '':
            return
        first, rest = self._get_values(value)
        self._children[first].add(rest)

    def remove(self, value: str):
        value = value.removeprefix('/').removesuffix('/')
        if value == '':
            return
        first, rest = self._get_values(value)
        self._children[first].remove(rest)
        if len(self._children[first]) == 0:
            del self._children[first]

    def _get_values(self, value):
        values = value.split('/', 1)
        first, rest = values[0], ''
        if len(values) == 2:
            rest = values[1]
        return first, rest

    def __iter__(self):
        if len(self._children) == 0:
            yield ''
            return

        for key, child in self._children.items():
            for value in child:
                if value != '':
                    yield key + '/' + value
                else:
                    yield key


def main():
    tree = Tree()

    tree.add('a/b/c')
    tree.add('a/b/d')
    tree.add('b/b/e')
    tree.add('b/c/f')
    tree.add('c/b/e')

    # duplicates are okay
    tree.add('b/c/f')

    # leading and trailing slashes are okay
    tree.add('b/c/f/')
    tree.add('/b/c/f')

    print('Before removing:', [x for x in tree])

    tree.remove('a/b/c')
    tree.remove('a/b/d')
    tree.remove('b/b/e')

    # it's okay to remove non-existent values
    tree.remove('this/path/does/not/exist')

    # it will not remove root-level b folder, because it's not empty
    tree.remove('b')

    print('After removing:', [x for x in tree])


if __name__ == '__main__':
    main()

Output:

Before removing: ['a/b/c', 'a/b/d', 'b/b/e', 'b/c/f', 'c/b/e']
After removing: ['b/c/f', 'c/b/e']

So, your algorithm is as follows:

1. Build tree of file paths on company server (let's call it A-tree );
2. Build trees of file paths on portable hard drives (let's call them B-trees );
3. Remove from the A-tree file paths that exist in B-trees ;
4. Print content of the A-tree .

Now, all you need - is to build these trees, which is task 1 from the answer beginning. And how it will be - depends on the data, that you have in your .txt file.