熊猫应用系列-列的顺序

Question

To aggregate and to find values per second, I am doing following in Python using pandas, however, in the output logged to a file doesn't show columns in the way they appear here. Somehow these column names are sorted and hence TotalDMLsSec shows up before UpdateTotal and UpdatesSec.

    'DeletesTotal': x['Delete'].sum(),
    'DeletesSec': x['Delete'].sum()/VSeconds,
    'SelectsTotal': x['Select'].sum(),
    'SelectsSec': x['Select'].sum()/VSeconds,
    'UpdateTotal': x['Update'].sum(),
    'UpdatesSec': x['Update'].sum()/VSeconds,
    'InsertsTotal': x['Insert'].sum(),
    'InsertsSec': x['Insert'].sum()/VSeconds,
    'TotalDMLsSec':(x['Delete'].sum()+x['Update'].sum()+x['Insert'].sum())/VSeconds
    })
)
df.to_csv(/home/summary.log,sep='\t', encoding='utf-8-sig')

Apart from above questions, have couple of other questions-

1. Despite logging as csv format, all values/columns are appearing in one column in excel, is there anyway to properly load data CSV
2. Can rows be sorted based on one column(let say InsertsSec) by default when writing to csv file?

Any help here would be really appreciated.

Answer 1

Assume that your DataFrame is something like this:

      Deletes  Selects  Updates  Inserts
Name                                    
Xxx        20       10       40       50
Yyy        12       32       24       11
Zzz        70       20       30       20

Then both total and total per sec can be computed as:

total = df.sum().rename('Total')
VSeconds = 5   # I assumed some value
tps = (total / VSeconds).rename('Total per sec')

Then you can add both above rows to the DataFrame:

df = df.append(totals).append(tps)

The downside is that all numbers are converted to float . But in Pandat there is no other way, as each column must have values of one type.

Then you can eg write it to a CSV file (with totals included).

Answer 2

This is how I eneded up doing

    df.to_excel(vExcelFile,'All')
    vSortedDF=df.sort_values(['Deletes%'],ascending=False)
    vSortedDF.loc[vSortedDF['Deletes%']> 5, ['DeletesTotal','DeletesSec','Deletes%']].to_excel(vExcelFile,'Top Delete objects')
vExcelFile.save()

For CSV, instead of using separate \\t used , and it worked just fine. df.to_csv(/home/summary.log,sep='\\t', encoding='utf-8-sig')