迭代和拆分excel文件名，并另存为Pandas中的数据框

Question

Say I have a folder folder1 with excel files, their filenames share same structures: city, building name and id , I want save them in dataframe and then excel file. Please note I also need to append other folders' excel filenames in result.

bj-LG center-101012.xlsx
sh-ABC tower-1010686.xlsx
bj-Jinzhou tower-101018.xlsx
gz-Zijin building-101012.xls
...

The first method I have tried:

import os
import pandas as pd
from pandas import DataFrame, ExcelWriter

path = os.getcwd()
file = [".".join(f.split(".")[:-1]) for f in os.listdir() if os.path.isfile(f)] #exclude files' extension

city = file.split('-')[0]
projectName = file.split('-')[1]
projectID = file.split('-')[2]
    #print(city)        
df = pd.DataFrame(columns = ['city', 'building name', 'id'])
df['city'] = city
df['building name'] = projectName
df['id'] = projectID    

writer = pd.ExcelWriter("C:/Users/User/Desktop/test.xlsx", engine='xlsxwriter')
df.to_excel(writer, index = False)
writer.save()

Problem:

Traceback (most recent call last):

  File "<ipython-input-203-c09878296e72>", line 9, in <module>
    city = file.split('-')[0]

AttributeError: 'list' object has no attribute 'split'

My second method:

for root, directories, files in os.walk(path):
    #print(root)
    for file in files:
        if file.endswith('.xlsx') or file.endswith('.xls'):
            #print(file)            
            city = file.split('-')[0]
            projectName = file.split('-')[1]
            projectID = file.split('-')[2]
            #print(city)        
    df = pd.DataFrame(columns = ['city', 'building name', 'id'])
    df['city'] = city
    df['building name'] = projectName
    df['id'] = projectID    

    writer = pd.ExcelWriter("C:/Users/User/Desktop/test.xlsx", engine='xlsxwriter')
    df.to_excel(writer, index = False)
    writer.save()

I got an empty test.xlsx file, how could I make it works? Thanks.

Answer 1

Method 2 is close.

You need to create the dataframe before the for loops. After your variable assignments, make a dictionary of the variables and append it to the dataframe. There is also probably a better way to find your file list using glob, but i will just work with what you have already done.

df = pd.DataFrame()
for root, directories, files in os.walk(path):

    for file in files:
        if file.endswith('.xlsx') or file.endswith('.xls'):
            #print(file)            
            city = file.split('-')[0]
            projectName = file.split('-')[1]
            projectID = file.split('-')[2]
            #append data inside inner loop
            d = {'city':city, 'building name':projectname, 'id':projectID}
            df.append(d)


writer = pd.ExcelWriter("C:/Users/User/Desktop/test.xlsx", engine='xlsxwriter')
df.to_excel(writer, index = False)
writer.save()

Answer 2

This splits off the file extension, then unpacks the split into the vairables. Creates a dictionary then appends the dictionary to the dataframe.

files = [
    "bj-LG center-101012.xlsx",
    "sh-ABC tower-1010686.xlsx",
    "bj-Jinzhou tower-101018.xlsx",
    "gz-Zijin building-101012.xls"]

df = pd.DataFrame()
for file in files:
    filename = file.split(".")[0]
    city, projectName, projectID = filename.split("-")
    d = {'city':city,'projectID':projectID,'projectName':projectName}


    df = df.append(d,ignore_index=True)

df.to_excel('summary.xlsx')

Answer 3

This should works, thanks to the hint of use glob from @Dan Wisner

import os
from glob import glob

fileNames = [os.path.splitext(val)[0] for val in glob('*.xlsx') or glob('*.xls')]

df = pd.DataFrame({'fileNames': fileNames})
df[['city', 'name', 'id']] = df['fileNames'].str.split('-', n=2, expand=True)

del df['fileNames']

writer = pd.ExcelWriter("C:/Users/User/Desktop/test1.xlsx", engine='xlsxwriter')
df.to_excel(writer, index = False)
writer.save()