How to add new a new column after doing some calculation in python

Question

ID_REF      1007_s_at 1053_at 117_at 121_at 1255_g_at
GSM11111    0.08277 0.00874 0.00363 0.01877 0.00075
GSM95474    0.09503 0.00592 0.00352 0.01944 0.00055
GSM95475    0.08486 0.00678 0.00386 0.01973 0.00039
GSM95476    0.08105 0.00913 1       0.01801 0.00055
GSM95477    0.05918 0.00812 0.00428 0.01597 0.00033
GSM95478    0.07615 0.00777 0.00438 0.01799 0.00129
GSM95479    0.0976  0       0.00399 0.0216  0.00125
GSM95480    0.08499 0.00442 0.00298 0.01897 0.00015
GSM95481    0.08893 0.00734 0.00204 0.01706 0.00089
GSM99999    0.05981 0.01587 0.00365 0.01709 0.0006

This is my csv file data frame where i have certain values, i am looking to find the total count for them column-wise, with having some condition ( count all the values but ignore the 0's ) basically, what will happen is a new row will be added below GSM99999 named Final & below each column eg 1007_s_at and other columns saying 52 (where 52 is the total count of numeric values leaving the 0's in the excel file)

i want to run this operation to the whole excel file regardless of how many column and row are there. I am currently using pandas and just started learning it.

Here is the csv file image version : data csv file

Here is the output i am looking for : output

Any help will be great for me, Thanks :)

Answer 1

import pandas as pd

df = pd.read_csv("<path to file>.csv").reset_index()

# update headers
df.columns = df.iloc[0]
df = df.iloc[1:].set_index("ID_REF")

df.loc["Final"] = ((df.notnull()) & (df != 0)).sum()

Answer 2

After reading he file from excel file in the dataframe df , you need:

df = df.set_index('ID_REF')
df = df.append(pd.DataFrame(dict(((df.notnull()) & (df != 0)).sum()), index=['Final']))

Output:

          1007_s_at 1053_at 117_at  121_at  1255_g_at
GSM11111    0.08277 0.00874 0.00363 0.01877 0.00075
GSM95474    0.09503 0.00592 0.00352 0.01944 0.00055
GSM95475    0.08486 0.00678 0.00386 0.01973 0.00039
GSM95476    0.08105 0.00913 1.00000 0.01801 0.00055
GSM95477    0.05918 0.00812 0.00428 0.01597 0.00033
GSM95478    0.07615 0.00777 0.00438 0.01799 0.00129
GSM95479    0.09760 0.00000 0.00399 0.02160 0.00125
GSM95480    0.08499 0.00442 0.00298 0.01897 0.00015
GSM95481    0.08893 0.00734 0.00204 0.01706 0.00089
GSM99999    0.05981 0.01587 0.00365 0.01709 0.00060
Final      10.00000 9.00000 10.00000 10.00000 10.00000