从 .txt 文件创建 DataFrame

Question

I need some wisdom here!

I'm trying to create a script that takes two (2).txt files with the same format, appends one to the other and then creates a DataFrame from that resulting file, so I can manipulate it.

The files are inventory results, but they are a little bit messy.

From these file I only needed the rows of the products, nothing more, to do so I'm using:

listados = ["analisis_diferencias.txt","no_contadas.txt"]

def unir_listados(listados):
    with open("df_final.txt","w+") as merge:
        for item in listados:
            with open(item) as readable:
                for line in readable:
                    if line[4] in ["1","2","3","4","5","6","7","8","9"]:
                        merge.write(line)
    

The result is a new.txt that looks perfect, since it only uses the lines where there is a product code.

But I just can't make it to a normal DataFrame, or any other structure that has columns.

The farthest I've been is to create a sigle column df using pd.read_table, with no idea on how to separate every single row on columns.

I tried replacing the whitespaces with ";"so I could later delete the empty columns it would generate but then I got a huge list of one row and more than 6k columns...

Also tried to replace them with "\t", but nothing.

The pd.read_csv method isn't working either:

a = pd.read_csv("df_final.txt", header=None, encoding="latin-1")

ParserError: Expected 18 fields in line 3, saw 19. Error could possibly be due to quotes being ignored when a multi-char delimiter is used.

I've seen a solution online that instead of creating a new.txt it creates a new df value by value when parsing every line of the original.txt

But I understand there should be a simpler method once you have the data displayed as I have it right now.

Thanks in advance for any help you can provide.

Ps: BTW when appending the lines to my new.txt, if I used str([1,2,3,4,5,6,7,8,9]) it would select every single row, since it detected the empty string char "" was in the array. Any idea on this?

EDIT:

I added some rows of the final.txt, as requested.

68.17.28   D-AA SPLIT HAIER TUNDRA AS-18 ]          0          1             1              562,00      562,000
42.50.10   Z-CAMARA INSTANT. FUJI INSTAX ]          1          3             2              111,80       55,900
54.15.88   Z-CAMARA INSTANT. FUJI INSTAX ]          2          2             0                0,00       59,900
67.05.04   A-CAMARA INSTANT. FUJI INSTAX ]          1          1             0                0,00       54,500
72.29.13   C-CAMARA INSTANT. FUJI INSTAX ]          1          1             0                0,00      121,950
21.08.75   D-MEMORIA MICRO SD ULTRA SANDI]          7          7             0                0,00       15,699
21.09.35   B-MEMORIA MICRO SD ULTRA SANDI]         16         16             0                0,00        3,616
21.09.70   D-MEMORIA MICRO SD ULTRA SANDI]         11         23            12               56,18        4,682
21.11.33   D-MEMORIA MICRO SD ULTRA SANDI]          4          4             0                0,00        7,830
23.36.92   A-MICROSD SAMSUNG EVO 32GB(MB-]          9          9             0                0,00        6,811

Answer 1

Without a sample of the text file, it is hard to know for sure. But could you try:

pd.read_table("df_final.txt", sep='\s+', header=None, encoding="latin-1")

This looks to separate the txt file columns based on whitespace.

Answer 2

how about use delimiter \s{2,} , in D-AA SPLIT HAIER TUNDRA AS-18 ] only have 1 space.

df = pd.read_csv(file, sep='\s{2,}',header=None, engine='python')

---

another way:

# read file with only one column
obj = pd.read_csv(file, sep='\n',header=None)[0]

def handle_row(row):
    row_list = re.split(r'\s+', row)
    # the first 2 columns
    prt1 = ' '.join(row_list[:-5]).split(' ', maxsplit=1)
    # the last 5 columns
    prt2 = row_list[-5:]
    return (prt1 + prt2)
df = pd.DataFrame(obj.map(handle_row).tolist())


print(df)

             0                                1   2   3   4       5        6
    0  68.17.28  D-AA SPLIT HAIER TUNDRA AS-18 ]   0   1   1  562,00  562,000
    1  42.50.10  Z-CAMARA INSTANT. FUJI INSTAX ]   1   3   2  111,80   55,900
    2  54.15.88  Z-CAMARA INSTANT. FUJI INSTAX ]   2   2   0    0,00   59,900
    3  67.05.04  A-CAMARA INSTANT. FUJI INSTAX ]   1   1   0    0,00   54,500
    4  72.29.13  C-CAMARA INSTANT. FUJI INSTAX ]   1   1   0    0,00  121,950
    5  21.08.75  D-MEMORIA MICRO SD ULTRA SANDI]   7   7   0    0,00   15,699
    6  21.09.35  B-MEMORIA MICRO SD ULTRA SANDI]  16  16   0    0,00    3,616
    7  21.09.70  D-MEMORIA MICRO SD ULTRA SANDI]  11  23  12   56,18    4,682
    8  21.11.33  D-MEMORIA MICRO SD ULTRA SANDI]   4   4   0    0,00    7,830
    9  23.36.92  A-MICROSD SAMSUNG EVO 32GB(MB-]   9   9   0    0,00    6,811