Excel 到 Python 字典，用于使用 Openpyxl 进行过滤

Question

I'm trying to read an excel file to store the selected columns into a python dictionary. For further data filtering and manipulation. I consulted a lot of Stackoverflow existing questions to get the pointers. However I'm able to figure out few things. But having no prior experience with Python giving me some real challenges. Can I ask for your help please? Following is the code which I was able to make it work up to some extent.

from _collections import defaultdict
import openpyxl

SalesFunnel = defaultdict(list)

theFile = openpyxl.load_workbook('Report.xlsx')
allSheetNames = theFile.sheetnames

print("All sheet names {} " .format(theFile.sheetnames))

for sheet in allSheetNames:
    print("Current sheet name is {}" .format(sheet))
    currentSheet = theFile[sheet]


sfunnel = []


for row in range(1, currentSheet.max_row + 1):
    for column in "ADEF":  
        cell_name = "{}{}".format(column, row)
        SalesFunnel[row] = [cell_name, currentSheet[cell_name].value]
        SalesFunnel[row].append(SalesFunnel[row])
    print(SalesFunnel)

My excel dataset contains duplicate emails and the duplicate lead statuses. Each row contains a created Date. I need to find out the max and min of the date for each email within a lead status. So that I can compute the days in between for each email address. But for now I'm unable to read the data in a correct format. I also added a column for the unique index which is simply the number sequence to uniquely identify each row.

If I got something like that in a json format. It would be great.

{Index: 1, LeadStatus: Contacted, Email: joe@doe.com, CreatedDate: 7/9/2020}
{Index: 2, LeadStatus: Contacted, Email: joe@doe.com, CreatedDate: 8/10/2020}
{Index: 3, LeadStatus: Contacted, Email: joe@doe.com, CreatedDate: 9/11/2020}
{Index: 4, LeadStatus: Contacted, Email: ron@email.com, CreatedDate: 4/5/2020}
{Index: 5, LeadStatus: Contacted, Email: ron@email.com, CreatedDate: 7/6/2020}

Also adding a screenshot of my excel sheet. You can only see one email because I have thousands of records. And for each email there could be many records. And Lead Status could be something other than Contacted.

Answer 1

You can read the data using pandas . I am assuming your data starts from cell A1 and you are reading everything in the sheet.

data_df = pd.read_excel(sheet_path, sheet_name)

Now you can calculate the max and min dates for each group and find your differences

data_df['MaxDate'] = data_df.groupby(['LeadStatus','Email'])['CreatedDate'].transform('max')
data_df['MinDate'] = data_df.groupby(['LeadStatus','Email'])['CreatedDate'].transform('min')
data_df['Difference'] = pd.to_datetime(data_df['MaxDate']) - pd.to_datetime(data_df['MinDate'])

If you don't want to repeat records, use agg

agg_df = data_df.groupby(['LeadId','LeadStatus','Email']).agg(MaxDate=('CreatedDate','max'),
                     MinDate = ('CreatedDate', 'min')).reset_index()
agg_df['Difference'] = pd.to_datetime(agg_df['MaxDate']) - pd.to_datetime(agg_df['MinDate'])

You can now convert to json if you like:

data_df.to_json(orient='records')

You can also write to excel

with pd.ExcelWriter('..../new_doc.xlsx', engine='xlsxwriter') as writer:

    data_df.to_excel(writer, sheet_name='New Data', index=False)
    agg_df.to_excel(writer, sheet_name='Agg Data', index=False)