How do I extract data from multiple text files to Excel using Python? (One file's data per sheet)

Question

So far for my code to read from text files and export to Excel I have:

import glob

data = {}
for infile in glob.glob("*.txt"):
    with open(infile) as inf:
        data[infile] = [l[:-1] for l in inf] 

with open("summary.xls", "w") as outf:
    outf.write("\t".join(data.keys()) + "\n")
    for sublst in zip(*data.values()):
        outf.write("\t".join(sublst) + "\n")

The goal with this was to reach all of the text files in a specific folder.

However, when I run it, Excel gives me an error saying,

"File cannot be opened because: Invalid at the top level of the document. Line 1, Position 1. outputgooderr.txt outputbaderr.txt. fixed_inv.txt

Note : outputgooderr.txt, outputbaderr.txt.,fixed_inv.txt are the names of the text files I wish to export to Excel, one file per sheet.

When I only have one file for the program to read, it is able to extract the data. Unfortunately, this is not what I would like since I have multiple files.

Please let me know of any ways I can combat this. I am very much so a beginner in programming in general and would appreciate any advice! Thank you.

Answer 1

If you're not opposed to having the outputted excel file as a .xlsx rather than .xls, I'd recommend making use of some of the features of Pandas . In particular pandas.read_csv() and DataFrame.to_excel()

I've provided a fully reproducible example of how you might go about doing this. Please note that I create 2 .txt files in the first 3 lines for the test.

import pandas as pd
import numpy as np
import glob

# Creating a dataframe and saving as test_1.txt/test_2.txt in current directory
# feel free to remove the next 3 lines if yo want to test in your directory
df = pd.DataFrame(np.random.randn(10, 3), columns=list('ABC'))
df.to_csv('test_1.txt', index=False)
df.to_csv('test_2.txt', index=False)

txt_list = [] # empty list
sheet_list = [] # empty list

# a for loop through filenames matching a specified pattern (.txt) in the current directory
for infile in glob.glob("*.txt"): 
    outfile = infile.replace('.txt', '') #removing '.txt' for excel sheet names
    sheet_list.append(outfile) #appending for excel sheet name to sheet_list
    txt_list.append(infile) #appending for '...txt' to txtt_list

writer = pd.ExcelWriter('summary.xlsx', engine='xlsxwriter')

# a for loop through all elements in txt_list
for i in range(0, len(txt_list)):
    df = pd.read_csv('%s' % (txt_list[i])) #reading element from txt_list at index = i 
    df.to_excel(writer, sheet_name='%s' % (sheet_list[i]), index=False) #reading element from sheet_list at index = i 

writer.save()

Output example: