使用 glob 递归获取包含 CSV 的子目录和文件

Question

I am trying to concat multiple CSVs that live in subfolders of my parent directory.

/ParentDirectory
│  
│
├───SubFolder 1
│       test1.csv
│
├───SubFolder 2
│       test2.csv
│
├───SubFolder 3
│       test3.csv
│       test4.csv
│
├───SubFolder 4
│       test5.csv

When I do

import pandas as pd
import glob

files = glob.glob('/ParentDirectory/*.csv', recursive=True)
df = pd.concat([pd.read_csv(fp) for fp in files], ignore_index=True)

I get ValueError: No objects to concatenate .

But if I select a specific sub folder, it works:

files = glob.glob('/ParentDirectory/SubFolder 3/*.csv', recursive=True)

How come glob isn't able to go down a directory and get the CSVs within each folder of the parent directory?

Answer 1

Try:

files = glob.glob('/ParentDirectory/**/*.csv', recursive=True)

Answer 2

files = glob.glob('/ParentDirectory/*/*.csv')

It doesn't need to be recursive for that pattern, but does need a wildcard for the subdirectory.