写入.txt和.xlsx格式时输出文件大小的差异

Question

I have tried saving 800 JSON responses to .txt file and also Excel file.

For txt, I am using:

for activity_id in activity_ids:
        activity_details = requests.get(url, params=activity_id).text
        with open('test.txt','a') as f:
            f.write(activity_details + '\n')

For JSON:

def df_to_excel(df, filename):
    writer = pandas.ExcelWriter(filename)
    df.to_excel(writer, 'Sheet1')
    writer.save()

for activity_id in activity_ids:
    activity_details = requests.get(url, params=activity_id).json()  
    df = json_normalize(j)
    df_to_excel(df, 'test.xlsx')

Why is there a huge difference in output file size: 6.5 Mb for Excel file and 30 Mb for txt? If anything, I would expect Excel file to be larger. Is there something I can do to shrink the txt output file?

Answer 1

Excel documents (.xlsx) are zip files containing xml files. The size difference that you're seeing is a result of the compression from the zip process.

Answer 2

The text file is being opened in append mode. If you haven't cleared it somewhere at the beginning of your code, it will keep accumulating a lot of excess text on each run of code. Additionally, you are writing responses for each activity id in text file, whereas it seems like you are overwriting sheet 1 in excel, so the excel only stores the last activity id info.

Edit: And yes, as @Michael stated, the excel files do store compressed data, and have smaller sizes than a plain text file.