将大数据写入.csv文件时，经常打开和关闭文件更好吗？

Question

I am writing a program with a while loop, which would write giant amount of data into a csv file. There maybe more than 1 million rows.

Considering running time, memory usage, debugging and so on, what is the better option between the two:

1. open a CSV file, keep it open and write line by line, until the 1 million all written

2. Open a file, write about 100 lines, close(), open again, write about 100 lines, ......

I guess I just want to know would it take more memories if we're to keep the file open all the time? And which one will take longer?

I can't run the code to compare because I'm using a VPN for the code, and testing through testing would cost too much $$ for me. So just some rules of thumb would be enough for this thing.

Answer 1

I believe the write will immediately write to the disk, so there isn't any benefit that I can see from closing and reopening the file. The file isn't stored in memory when it's opened, you just get essentially a pointer to the file, and then load or write a portion of it at a time.

Edit

To be more explicit, no, opening a large file will not use a large amount of memory. Similarly writing a large amount of data will not use a large amount of memory as long as you don't hold the data in memory after it has been written to the file.