cannot write file with full path in Python

Question

I am using Pandas on Mac, to read and write a CSV file, and the weird thing is when using full path, it has error and when using just a file name, it works. I post my code which works and which not works in my comments below, and also detailed error messages. Anyone have any good ideas?

sourceDf = pd.read_csv(path_to_csv)
sourceDf['nameFull'] = sourceDf['nameFirst'] + ' ' + sourceDf['nameLast']
sourceDf.to_csv('newMaster.csv') # working
sourceDf.to_csv('~/Downloads/newMaster.csv') # not working

Traceback (most recent call last):
  File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 36, in <module>
    add_full_name(path_to_csv, path_to_new_csv)
  File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 28, in add_full_name
    sourceDf.to_csv('~/Downloads/newMaster.csv')
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
    formatter.save()
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
    encoding=self.encoding)
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
    f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/newMaster.csv'

Tried to use prefix r, but not working,

    path_to_csv = r'~/Downloads/Master.csv'
    path_to_new_csv = r'~/Downloads/Master_new.csv'

  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
    formatter.save()
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
    encoding=self.encoding)
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
    f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/Master_new.csv'

thanks in advance, Lin

Answer 1

Try using os.path.join() .

import os
(...)
output_filename = 'newMaster.csv'
output_path = os.path.join('Downloads', output_filename)
(...)
sourceDf.to_csv(output_path)

Use the same methodology to point pandas.read_csv() in the right direction.

Answer 2

You didn't specify python version. On 3.4 you can use pathlib, otherwise use os.path.join() or quoting:

sourceDf.to_csv(r'~/Downloads/newMaster.csv')

Notice the r. The problem is that /n is newline, which is not allowed in a path.