Python: ID error when importing csv file with pandas

Question

I am trying to Import a csv file saved in a local Folder. When I use Anaconda Python Notebook I have no Problems, while using Zeppelin I do have issues. The code I am using, that works fine in Anaconda, is:

#import csv data

frequency=pd.read_csv("C:\\Users\\L18938\\Desktop\\Vehicle_to_grid\\analysis\\Frequency_March_2018.csv", nrows=86401)

However, when running it on Zeppelin, I receive:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 646, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 389, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 730, in __init__
    self._make_engine(self.engine)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 923, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py", line 1390, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas/parser.pyx", line 373, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:4025)
  File "pandas/parser.pyx", line 667, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:8031)
IOError: File C:\Users\L18938\Desktop\Vehicle_to_grid\analysis\Frequency_March_2018.csv does not exist

Obviously, the file exists and there are no Errors in the path spelling.

I have tryied / or double \\, but nothing changes. Also

os.chdir("C:/Users/L18938/Desktop/Vehicle_to_grid/analysis")

or

os.listdir("C:/Users/L18938/Desktop/Vehicle_to_grid/analysis")

Any idea? thank you in advance

Answer 1

Your Traceback let show you that the python interpreter is running in Unix file path mode ( /usr/local/lib/python2.7/dist-packages/pandas/io/parsers.py )

When you are under Anaconda , you are in pure windows and your traceback will be something like ( C:\\ProgramData\\Anaconda3\\lib\\site-packages\\pandas\\io\\parsers.py )

Anaconda will reach file with a Windows type file-path, and Zeppelin will reach file in a UNIX type file-path.

Your issue is definitely relative to how you specify your path in Zeppelin , you can't use Windows path, but you you may try something like that:

frequency=pd.read_csv("file:///C:/Users/L18938/Desktop/Vehicle_to_grid/analysis/Frequency_March_2018.csv", nrows=86401)