Reading multiple CSV files with headers in python with numpy?

Question

Currently I am using this code to read one complete csv file to my code:

data = np.loadtxt('csv_Complete.csv', delimiter=',', skiprows=1)

However, Now I have multiple csv files that are in this format.

Log1.csv

x1,x2,x3,x4....
1.5,3,5,7,8
2,5,1.2,5,2
1,3,3,5.5,6

log2.csv

 x1,x2,x3,x4....
 1,3.3,5,7,8
 2,5.1,1,5.5,2
 1,3,3,5,6

This is the method I am thinking of doing but it is not working. Getting a ValueError: could not convert string to float:

log1 = np.loadtxt('log1.csv', delimiter=',', skiprows=1)
log2 = np.loadtxt('log2.csv', delimiter=',', skiprows=1)
log3 = np.loadtxt('log3.csv', delimiter=',', skiprows=1)
data = np.append([log1, log2, log3])

The error I am getting is:

 File "<ipython-input-6-6155c8de61ad>", line 1, in <module>
    runfile('C:/Users/Mmyname/.spyder2-py3/setdataexp.py', wdir='C:/Users/myname/.spyder2-py3')

  File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 714, in runfile
    execfile(filename, namespace)

  File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 89, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/myname/.spyder2-py3/setdataexp.py", line 5, in <module>
    log1 = np.loadtxt('log40a.csv', delimiter=',', skiprows=1)

  File "C:\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 930, in loadtxt
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 930, in <listcomp>
    items = [conv(val) for (conv, val) in zip(converters, vals)]

  File "C:\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 659, in floatconv
    return float(x)

ValueError: could not convert string to float: 

Answer 1

It must be a missing value in file log40a.csv .

I have the same error for file like:

x1,x2,x3,x4....
1,3.3,5,7,8
2,5.1,,5.5,2
1,3,3,5,6

Base on documentation if you have missing values you should use genfromtxt function .