Using os.listdir() to write file names in TXT but uncompleted

Question

I am trying to write all the file names, including their paths, in a folder to a text file. what I am doing is below:

import os

BASEDIR = r"C:\project 111"

aa = os.listdir(BASEDIR)

text_file = open(os.path.join(BASEDIR, 'all files.txt'), "w")

for b in aa:
    c = os.path.join(BASEDIR, b)
    text_file.write(c)
    text_file.write('\n')

list_open = open(os.path.join(BASEDIR, 'all files.txt'))
read_list = list_open.read()
line_in_list = read_list.split('\n')
number_of_files = sum(1 for line in open(os.path.join(BASEDIR, 'all files.txt')))

print number_of_files

the problem is that, it doesn't write all the wanted files to the text file. the actual number of files is more than 500 however it only writes 379 lines. and the last line in the text file is uncompleted.

what's the cause of the problem, and how can I correct it? thanks.

Answer 1

After opening a file with the open function, you should always close it when you're done, otherwise the behavior can be unpredictable.

text_file.close()
list_open.close()

Update

A better way to go about doing this is to use context managers (ie the with-as statement). Then you don't have to worry about closing them at all; the context manager does it for you as soon as it goes out of scope!

with open(os.path.join(BASEDIR, 'all files.txt'), "w") as text_file:
    for b in aa:
        c = os.path.join(BASEDIR, b)
        text_file.write(c)
        text_file.write('\n')

with open(os.path.join(BASEDIR, 'all files.txt')) as list_open:
    read_list = list_open.read()

line_in_list = read_list.split('\n')

with open(os.path.join(BASEDIR, 'all files.txt') as allfiles:
    number_of_files = sum(1 for line in allfiles))

print number_of_files

Here's a nice introduction to context managers: https://www.inkling.com/read/learning-python-mark-lutz-4th/chapter-33/withas-context-managers

Answer 2

In Python, a better recommend solution is to use with statement to open/write a file with statement will auto call close() when executing exist the block :

with  open(os.path.join(BASEDIR, 'all files.txt'), "w") as f:
    for b in aa:
        c = os.path.join(BASEDIR, b) 
        f.write(c + "\n")

for a quick fix is adding close() call to file object:

for b in aa:
    c = os.path.join(BASEDIR, b)
    text_file.write(c)
    text_file.write('\n')

// pls add this line
text_file.close()

list_open = open(os.path.join(BASEDIR, 'all files.txt'))

Answer 3

using context managers ( with keyword)

for reference:

• https://docs.python.org/2/library/contextlib.html#contextlib.closing
• http://effbot.org/zone/python-with-statement.htm

and of course the official doc: https://docs.python.org/2/tutorial/inputoutput.html

It is good practice to use the with keyword when dealing with file objects. This has the advantage that the file is properly closed after its suite finishes, even if an exception is raised on the way. It is also much shorter than writing equivalent try - finally blocks:

>>> with open('workfile', 'r') as f:
...     read_data = f.read()
>>> f.closed
True

the problem is not close()

The problem isn't that close() magically adds the data to the file. The problem is that most operating systems are intelligent and have different levels/layers of buffers (or the problem is that not everyone reads the docs or goes to a CS course). In order for your data to reach the disk, you need to flush the buffers. This is done with file.flush() . close() calls flush() .

Using the context manager ( with open... ) hides that complexity; so it is safer for most use cases to stick with that construct.