Python read csv file and add one column through function computation

Question

I have an example csv file with name 'r2.csv':

Factory | Product_Number |   Date     |   mu   |   cs   |  co 
--------------------------------------------------------------
   A    |      1         | 01APR2017  |   5.6  |  125   |  275
--------------------------------------------------------------
   A    |      1         | 02APR2017  |   4.5  |  200   |  300
--------------------------------------------------------------
   A    |      1         | 03APR2017  |   6.6  |  150   |  250
--------------------------------------------------------------
   A    |      1         | 04APR2017  |   7.5  |  175   |  325
--------------------------------------------------------------

I would like to add one more column with name 'Order_Number' . With the following function

Order_Number = np.ceil(poisson.ppf(co/(cs+co), mu))

With the following code I have:

    import numpy as np
    from scipy.stats import poisson, norm
    import csv

    # Read Data
    with open('r2.csv', 'r') as infile:
        reader = csv.DictReader(infile)
        data = {}
        for row in reader:
            for header, value in row.items():
                try:
                    data[header].append(value)
                except KeyError:
                    data[header] = [value]

    # To create a list for the following parameters 
    mu = data['mu']
    cs = data['cs']
    co = data['co']

    # Obtain Order_Number 
    Order_Number = np.ceil(poisson.ppf(co/(cs+co), mu))

Before Obtaining 'Order_Number' it works fine. And 'Order_Number' function it has the following error: TypeError: unsupported operand type(s) for /: 'list' and 'list'

How could I change my code in order to obtain the following table as output：

Factory | Product_Number |   Date    |  mu  | cs  | co  | Order_Number
----------------------------------------------------------------------
   A    |      1         | 01APR2017 |  5.6 | 125 | 275 |   ?
----------------------------------------------------------------------
   A    |      1         | 02APR2017 |  4.5 | 200 | 300 |   ?
----------------------------------------------------------------------
   A    |      1         | 03APR2017 |  6.6 | 150 | 250 |   ?
----------------------------------------------------------------------
   A    |      1         | 04APR2017 |  7.5 | 175 | 325 |   ?
----------------------------------------------------------------------

Answer 1

Looks like your content of mu , cs and co is list of strings.
First convert that to float .

mu = map(float,mu)  
cs = map(float,cs)  
co = map(float,co)

Then , since you have list of values you need to map your np.ceil(poisson.ppf(co/(cs+co), mu)) function to each value of these lists.

Order_Number =map(lambda mu_,cs_,co_:np.ceil(poisson.ppf(co_/(cs_+co_),mu_)),mu,cs,co)

Result is as follows,

>>> map(lambda mu_,cs_,co_:np.ceil(poisson.ppf(co_/(cs_+co_), mu_)),mu,cs,co)
[7.0, 5.0, 7.0, 8.0]

Hope this helps.

EDIT-1

Code to add data to an csv file. You may want to look at opening your csv to orderedDict so that you dont need to write each column headers manually. Yo can just call data.keys() .

#Covnert string element of list to float
mu = map(float,mu)  
cs = map(float,cs)  
co = map(float,co)

# Obtain Order_Number 
Order_Number =map(lambda mu_,cs_,co_:np.ceil(poisson.ppf(co_/(cs_+co_),mu_)),mu,cs,co)

#Add Oder_Number to the data dict
data['Order_Number'] = Order_Number
header = 'Factory','Product_Number','Date','mu','cs','co','Order_Number'
#Add data to csv
with open("output.csv",'wb') as resultFile:
    wr = csv.writer(resultFile,quoting=csv.QUOTE_ALL)
    wr.writerow(header)
    z = zip(data['Factory'],data['Product_Number'],data['Date'],data['mu'],data['cs'],data['co'],data['Order_Number'])
    for i in z:
        wr.writerow(i)

Result

Answer 2

As created

mu = data['mu']
cs = data['cs']
co = data['co']

are lists of strings. Look at them, or at least a subset, eg mu[:10] . You can't do array math with lists

co/(cs+co)

cs+co will concatenate the 2 lists ( + definition for lists), but / is not defined for lists.

mu = np.array(data, dtype=float)
cs = ....
co 

might do the trick, converting the lists into 1d numpy arrays.

An alternative is to use np.genfromtxt with dtype=None and names=True to load the data into a structured array. But then I'd have to explain how to access the named fields. And unfortunately adding a new field to this array (the calc results) isn't trivial. And writing a new csv from a structured array requires some extra knowledge.

Try the list to array conversion.