Errno2 No Such File or Directory using np.save

Question

import glob
import os
import numpy as np
from numpy import genfromtxt

mycsvdir = '/home/my/file/path/CSV_Data/'

csvfiles = glob.glob(os.path.join(mycsvdir, '*.csv'))

for csvfile in csvfiles:
 numpy_data = genfromtxt(csvfile, delimiter = ",", usecols = (0,1,3,4,5,18,19,20,21,22,23,24,25,26,27,28,29,30,31,79), dtype=None)
 np.save('/home/my/file/path/numpy_array/' + csvfile, numpy_data)

I'm reading multiple CSV's files from a folder and outputting these files into a numpy array and then saving these files into a different folder.

Receiving below error message:

FileNotFoundError: [Errno 2] No such file or directory: '/home/my/file/path/numpy_array//home/my/file/path/CSV_Data/Friday-16-02-2018_TrafficForML_CICFlowMeter.csv.npy'

Can anyone explain why? I have already joined the complete file path using glob, so this is weird for me.

Answer 1

csvfiles in your case represents a list of paths to csv files in the directory '/home/my/file/path/CSV_Data/' .

Thus when you're looping through csvfiles you shouldn't repeat part of the path

for csvfile in csvfiles:
    numpy_data = genfromtxt(csvfile, delimiter = ",", usecols = (0,1,3,4,5,18,19,20,21,22,23,24,25,26,27,28,29,30,31,79), dtype=None)
    np.save(csvfile, numpy_data) # <- change here

csvfile = '/home/my/file/path/CSV_Data/Friday-16-02-2018_TrafficForML_CICFlowMeter.csv.npy'

In italics you can see part of the path that is repeated, just to clarify where you went wrong

'/home/my/file/path/numpy_array/' + csvfile = ' /home/my/file/path/numpy_array/ /home/my/file/path/CSV_Data/Friday-16-02-2018_TrafficForML_CICFlowMeter.csv.npy'