saving csv files to new directory

Question

I am trying to use this code to write my edited csv files to a new directory. Does anyone know how I specify the directory?

I have tried this but it doesn't seem to be working.

dir = r'C:/Users/PycharmProjects/pythonProject1' # raw string for windows.
csv_files = [f for f in Path(dir).glob('*.csv')] # finds all csvs in your folder.

cols = ['Temperature']

for csv in csv_files: #iterate list
    df = pd.read_csv(csv) #read csv
    df[cols].to_csv('C:/Users/Desktop', csv.name, index=False)
    print(f'{csv.name} saved.')

Answer 1

I think your only problem is the way you're calling to_csv(), passing a directory and a filename. I tried that and got this error:

IsADirectoryError: [Errno 21] Is a directory: '/Users/zyoung/Desktop/processed'

because to_csv() is expecting a path to a file, not a directory path and a file name.

You need to join the output directory and CSV's file name, and pass that, like:

out_dir = PurePath(base_dir, r"processed")
# ...
#   ...
    csv_out = PurePath(out_dir, csv_in)
    df[cols].to_csv(csv_out, index=False)

I'm writing to the subdirectory processed, in my current dir ("."), and using the PurePath() function to do smart joins of the path components.

Here's the complete program I wrote for myself to test this:

import os
from pathlib import Path, PurePath

import pandas as pd

base_dir = r"."
out_dir = PurePath(base_dir, r"processed")

csv_files = [x for x in Path(base_dir).glob("*.csv")]

if not os.path.exists(out_dir):
    os.mkdir(out_dir)

cols = ["Temperature"]
for csv_in in csv_files:
    df = pd.read_csv(csv_in)
    csv_out = PurePath(out_dir, csv_in)
    df[cols].to_csv(csv_out, index=False)
    print(f"Saved {csv_out.name}")