Reading multiple csv files from a directory in python

Question

I'm having trouble reading multiple csv files from a directory.

The code works perfectly when running for the first file, but however I get an error on the second file. Below are the relevant sections of code. (some however is unnecessary for the question, I just left it in so it makes more sense):

def get_data(filename):
    '''function to read the data form the input csv file to use in the analysis'''
    with open(filename, 'r') as f:
        reader = csv.reader(f,delimiter=',')                      
        #returns all the data from the csv file in list form
        return list(reader)   

path='C:\Users\AClayton\Desktop\AW189 Data' 
for infile in glob.glob(os.path.join(path, '*.csv')):

    # infile stores the complete path of the file
    print "Current File Being Processed is:  " + infile 
    #use split to seperate the path and name of the file
    (PATH, FILENAME) = os.path.split(infile)
    print " PATH is " + PATH
    print " FILENAME is " + FILENAME
    all_data=[]
    #adds the data from the csv file to a blank list so it can be operated on
    all_data.extend(get_data(infile))
    #Create array so numerical operations may be performed
    arrdata=np.array(all_data)
    current_tracks=establish_current_tacks(arrdata,nb)
    rel_values=relative_track(current_tracks,nb)
    avg_rel_track=averaged_rel_track(navg, rel_values, nb)
    avg_rel_track_nan=avg_rel_track_nan(avg_rel_track)
    sd_rel_track_sum=sd_of_rel_track(navg, rel_values, nb)
    sd_index=sd_index(sd_rel_track_sum)

And this is my error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
C:\Users\AClayton\AppData\Local\Enthought\Canopy\App\appdata\canopy-1.0.3.1262.win-x86_64\lib\site-packages\IPython\utils\py3compat.pyc in execfile(fname, glob, loc)
    174             else:
    175                 filename = fname
--> 176             exec compile(scripttext, filename, 'exec') in glob, loc
    177     else:
    178         def execfile(fname, *where):

C:\Users\AClayton\Desktop\currentmethod06_09_2013work-1.py in <module>()
    204     rel_values=relative_track(current_tracks,nb)
    205     avg_rel_track=averaged_rel_track(navg, rel_values, nb)
--> 206     avg_rel_track_nan=avg_rel_track_nan(avg_rel_track)
    207     sd_rel_track_sum=sd_of_rel_track(navg, rel_values, nb)
    208     sd_index=sd_index(sd_rel_track_sum)

TypeError: 'numpy.ndarray' object is not callable

I'm unsure what is causing this problem. It doen't seem to be reading the function as a function, but instead as the array from earlier? Thanks a lot for any advice

Answer 1

This is weird:

avg_rel_track_nan=avg_rel_track_nan(avg_rel_track)

You probably meant:

avg_rel_track_nan=averaged_rel_track_nan(avg_rel_track)

I can't tell for sure what is happening because a lot of your code depends on some other code but you are assigning the result of a (supposedly) function call to the name of this function. I'm guessing that's not what you meant to do.

Answer 2

Try:

def get_data(filename):
    '''function to read the data form the input csv file to use in the analysis'''
    reader = [] # Just in case the file open fails
    with open(filename, 'r') as f:
        reader = csv.reader(f,delimiter=',')                      
        #returns all the data from the csv file in list form
    #f.close() # May need to close the file when done
    return list(reader)  # only return the reader when you have finished.

I think part of the problem is you are returning from within the with, (and not closing the file)