How to convert the column to numeric in python for sorting

Question

I am new to python (learner). Please check my question and help me to resolve the issue.

I have csv file with the below content

test,cycle,date,status
func,2,09/07/17,pass
func,10,09/08/17,fail
func,3,09/08/17,pass
func,1,09/08/17,no run
func,22,09/08/17,in progress
func,11,09/08/17,on hold

when i sort 2nd column (cycle) it shows the below output

['func', '1', '09/08/17', 'no run']
['func', '10', '09/08/17', 'fail']
['func', '11', '09/08/17', 'on hold']
['func', '2', '09/07/17', 'pass']
['func', '22', '09/08/17', 'in progress']
['func', '3', '09/08/17', 'pass']

Problem I faced here is it is sorting as string, due to this it shows the output as 1, 10, 11, 2, 22, 3. but i want to get the output in sorted by numeric (int/float) so that i will get the output 1, 2, 3, 10, 11, 22.

Below is the small script i have. could you help me to modify the script to change the column it to numeric before sort?

with open ('C:\Automation\sample.csv') as csvfile:

readCSVfile = csv.reader(csvfile, delimiter=',')

for row in readCSVfile:
sort = sorted(readCSVfile, key=operator.itemgetter(1), reverse = False)
 for eachline in sort:
print eachline`

Answer 1

You could pre-process the lines as you read them in:

#!python2
import csv
import operator

with open ('sample.csv','rb') as csvfile:
    readCSVfile = csv.reader(csvfile)
    header = next(readCSVfile)
    rows = []
    for row in readCSVfile:
        test,cycle,date,status = row
        rows.append([test,int(cycle),date,status])
rows.sort(key=operator.itemgetter(1))
for row in rows:
    print row

Output:

['func', 1, '09/08/17', 'no run']
['func', 2, '09/07/17', 'pass']
['func', 3, '09/08/17', 'pass']
['func', 10, '09/08/17', 'fail']
['func', 11, '09/08/17', 'on hold']
['func', 22, '09/08/17', 'in progress']

You could also use a different sort key, leaving the column a string:

#!python2
import csv
import operator

with open ('sample.csv','rb') as csvfile:
    readCSVfile = csv.reader(csvfile)
    header = next(readCSVfile)
    rows = [row for row in readCSVfile]
rows.sort(key=lambda row: int(row[1]))
for row in rows:
    print row

Output:

['func', '1', '09/08/17', 'no run']
['func', '2', '09/07/17', 'pass']
['func', '3', '09/08/17', 'pass']
['func', '10', '09/08/17', 'fail']
['func', '11', '09/08/17', 'on hold']
['func', '22', '09/08/17', 'in progress']

Answer 2

Then you have to convert it to numeric. Python csv module do not auto recognize data types.

You can do it by something like:

numberedCSV = []
for row in readCSVfile:
    row[1] = int(row[1])
    numberedCSV.append(row)

Then perform sorting on the numberedCSV .

btw, I do not understand your intention the code you posted. Why do you need two loops?

Answer 3

Here this might be what you are looking for.

    # take second element for sort
def takeSecond(elem):
    return int(elem[1])

# random list
stuff = [['func', '1', '09/08/17', 'no run'],
 ['func', '10', '09/08/17', 'fail'],
 ['func', '11', '09/08/17', 'on hold'],
 ['func', '2', '09/07/17', 'pass'],
 ['func', '22', '09/08/17', 'in progress'],
 ['func', '3', '09/08/17', 'pass']]

# sort list with key
sortedList = sorted(stuff, key=takeSecond)

# print list
print('Sorted list:', sortedList)

cheers.

Answer 4

As other answers has stated, either you can

• using another function than operator.itemgetter to convert value to int when sorting
• or using a for loop to convert the array data before sorting.

But if you working with tabular data like this often, it's better to use pandas . You need to install it, but again: if you do this often, it's worth it.

import pandas as pd

df = pd.read_csv('sample.csv')

df['cycle'] = df['cycle'].astype(int)

print(df.sort_values(by='cycle'))

# or reverse
print(df.sort_values(by='cycle', ascending=False))