How can I iterate over a list of .txt files using numpy?

Question

I'm trying to iterate over a list of .txt files in Python. I would like to load each file individually, create an array, find the maximum value in a certain column of each array, and append it to an empty list. Each file has three columns and no headers or anything apart from numbers.

My problem is starting the iteration. I've received error messages such as "No such file or directory", then displays the name of the first .txt file in my list.

I used os.listdir() to display each file in the directory that I'm working with. I assigned this to the variable filenamelist , which I'm trying to iterate over.

Here is one of my attempts to iterate:

for f in filenamelist:
    x, y, z = np.array(f)
    currentlist.append(max(z))

I expect it to make an array of each file, find the maximum value of the third column (which I have assigned to z) and then append that to an empty list, then move onto the next file.

Edit: Here is the code that I have wrote so far:

import os
import numpy as np
from glob import glob

path = 'C://Users//chand//06072019'
filenamelist = os.listdir(path)
currentlist = []
for f in filenamelist:
    file_array = np.fromfile(f, sep=",")
    z_column = file_array[:,2]
    max_z = z_column.max()
    currentlist.append(max_z)

Edit 2: Here is a snippet of one file that I'm trying to extract a value from:

0,           0.996,    0.031719
5.00E-08,    0.996,    0.018125
0.0000001,   0.996,    0.028125
1.50E-07,    0.996,    0.024063
0.0000002,   0.996,    0.023906
2.50E-07,    0.996,    0.02375
0.0000003,   0.996,    0.026406

Each column is of length 1000. I'm trying to extract the maximum value of the third column and append it to an empty list.

Answer 1

The main issue is that np.array(filename) does not load the file for you. Depending on the format of your file, something like np.loadtxt() will do the trick (see the docs ).

Edit: As others have mentioned, there is another issue with your implementation. os.listdir() returns a list of file names , but you need file paths . You could use os.path.join() to get the path that you need.

Below is an example of how you might do what you want, but it really depends on the file format. In this example I'm assuming a CSV (comma separated) file.

Example input file:

1,2,3
4,5,6

Example code:

path = 'C://Users//chand//06072019'
filenames = os.listdir(path)
currentlist = []

for f in filenames:
    # get the full path of the filename
    filepath = os.path.join(path, f)
    # load the file
    file_array = np.loadtxt(filepath, delimiter=',')
    # get the whole third column
    z_column = file_array[:,2]
    # get the max of that column
    max_z = z_column.max()
    # add the max to our list
    currentlist.append(max_z)