Python Pandas - 遍历文件夹 of.xlsx 文件，仅使用正则表达式从 Excel 选项卡中添加数据，名称中带有 xx.xx

Question

I have dataset of nearly 30 Excel files. I've been able to loop through the folder and append them all into one dataset. My problem is some of these Excel files have more than one tab's worth of data I need. All of the tabs I need have the same pattern of dates denoted in tab name (eg, 01.21). Obviously Regex is what I need and I know the Regex pattern I need, my problem is I don't know how to use Pandas to loop through each Excel file, check the tab names with regex, and only add data from tabs that have xx.xx in the string. For example, if I opened an Excel file and there were 3 tabs: "data 01.22", "financials", and "data 03.23", I would only want it to add data from "data 01.22" and "data 03.23".

The regex pattern I need to identify the name pattern in these tabs is [0-9][0-9]+.[0-9][0-9]. I know I'm close, but I am missing something key and any help is appreciated.

import pandas as pd
import os
import re

# filenames
files = os.listdir()    
excel_names = list(filter(lambda f: f.endswith('.xlsx'), files))

# read them in
excels = [pd.ExcelFile(name, engine='openpyxl') for name in excel_names]

# turn them into dataframes
frames = [x.parse(x.sheet_names[0], header=None,index_col=None) for x in excels]

#These are the tabs 
sh = [x.sheet_names for x in excels]

# I know I need to use this regex below, but where is the question:

#sheet_match = re.findall("[0-9][0-9]+\.[0-9][0-9]", s)

# delete the first row for all frames except the first
# i.e. remove the header row -- assumes it's the first
frames[1:] = [df[1:] for df in frames[1:]]

# concatenate 
combined = pd.concat(frames)

# export 
combined.to_excel("combinedfiles.xlsx", header=False, index=False)

Answer 1

You're really close indeed, you just have to filter the sheets names with re.match . Loop through each Excel file, and for each file, open it and get the list of tab names ( excel_file.sheet_names ) use re.match with the expression you already defined to get only those tabs that match the desired pattern. Read the content of these sheets ( sheet_name=valid_sheets ) adjusting headers and index as needed for you particular case, then, add the extracted content of each excel file to a list. Concatenate the list with pd.concat and generate the new excel file.

import pandas as pd
import os
import re

# filenames
files = os.listdir()
excel_names = list(filter(lambda f: f.endswith('.xlsx'), files))

regex = r'[0-9][0-9]+\.[0-9][0-9]'

frame_list = []
# loop through each Excel file
for name in excel_names:
    # open one excel file
    excel_file = pd.ExcelFile(name, engine='openpyxl')
    # get the list of tabs that have xx.xx in the string
    valid_sheets = [tab for tab in excel_file.sheet_names if re.match(regex, tab)]
    # read the content from that tab list
    d = excel_file.parse(sheet_name=valid_sheets, header=0)
    # add the content to the frame list
    frame_list += list(d.values())

combined = pd.concat(frame_list)
combined.to_excel("combinedfiles.xlsx", header=False, index=False)