Python. How to read multiple .csv files?

Question

I've got more than 200 files in .csv and I'd like to read and compute two of them in the same time (current and the next one). I'm trying to use glob and pandas data frames

import glob

for file in glob.glob(path+'*.csv'):
    x1 = pd.read_csv(file, delimiter=',', dtype=None, names=('x', 'y'))
    x2 = pd.read_csv(file + 1 , delimiter=',', dtype=None, names=('x', 'y'))

I've got no other ideas.

Answer 1

If you wish to work with the current and next file at every iteration, the following should do:

from glob import glob


files = glob('*.csv')

for i, file in enumerate(files[:-1]):
    x1 = pd.read_csv(file, delimiter=',', dtype=None, names=('x', 'y'))
    x2 = pd.read_csv(files[i+1] , delimiter=',', dtype=None, names=('x', 'y'))
    # Do what you want to do

This uses enumerate to keep track of the index of the current file from the files sequence. This way, you can grab the "next file" while working with the "current" file by just adding 1 the current index.

I hope this helps.

Answer 2

You can use the pairwise recipe from the itertools documentation:

from itertools import tee
def pairwise(iterable):
    "s -> (s0,s1), (s1,s2), (s2, s3), ..."
    a, b = tee(iterable)
    next(b, None)
    return zip(a, b)

From: https://docs.python.org/3/library/itertools.html#itertools-recipes

and usage:

for file1, file2 in pairwise(glob.glob(path+'*.csv')):
    ...