Python CSV copying cells from one csv file to another

Question

Edit: This has been solved. I've added the solution to the bottom of this post.

I have two files, newStock.csv and sortedBlad.csv.

newStock.csv looks like

SKU_IN,QTY_IN
3201MO-1,1
320205-1,4

sortedBlad.csv looks like

eid,sku,asd,asdf,Asdg,QTY,sdfasf,asdf,sdfsf,dfsdf,sdfs,dddd,dssss,Name,dddddpo,odommm,mdmdjkon,kkkkknn,okokokok,okokokb,jj,mb,bgtio,

1.11E+11,3201MO-1,,,,5,55,,£20.90,08-Aug-14,,New,9,FORD FOCUS MK1 98 04  OIL SUMP PAN FOR ENGINE  1.8 16V 2.0 16V  & ST RS PETROL,Oil Sumps,174039,,3,27-Jan-15 11:44:30 GMT,,,1000,FALSE,

The code i have at the moment goes through sortedBlad.csv and copies the eid, sku and name columns and inserts them into another file. It also matches up the skus from newStock and appends them to the end of each row. The issue im facing is that i cant seem to get the qty from new stock to also be appended to the end of each row. Currently the output of the code is as below.

    1.11E+11,3201MO-1,5,FORD FOCUS MK1 98 04  OIL SUMP PAN FOR ENGINE  1.8 16V 2.0 16V  & ST RS PETROL,3201MO-1,,
    1.21E+11,320205-1,6,FORD FOCUS 2004  2008 FRONT BUMPER  RADIATOR GRILLE BONNET WITH CHROME TRIM,320205-1,,

You can see that the sku is being appended to the end of the row. I would like to also append the newStock qty after the sku.

The code i have so far is as follows.

import csv

test_file = "sortedBlad.csv"
test_file2 = "sortedNewStock.csv" 

test_file2_array = []

finalFile = "final.csv"

csv_f_array = []

csv_file = csv.DictReader(open(test_file, 'r'), delimiter=',', quotechar='"')
csv_file2 = csv.DictReader(open(test_file2, 'r'), delimiter=',', quotechar='"')


with open(test_file2, 'rt') as a:
        reader = csv.reader(a, delimiter=",")
        for row in reader:
            test_file2_array.append(row[0])

#print(test_file2_array)

file = open(finalFile, 'w')

for eachline in csv_file:
    for SKU_IN in test_file2_array:
        if SKU_IN == eachline['sku']:
            print(SKU_IN+'-'+test_file2_array[1])
            # why wont you write the qty?
         file.write(eachline['eid']+','+eachline['sku']+','+eachline['QTY']+','+eachline['Name']+','+SKU_IN+','+',\n')

file.close

Thanks for the replies guys. I managed to solve the issue with some help from someone in the learnpython irc over at learnpython in reddit.

The solution was to use a dictionary to store the SKU_IN as a key with the QTY_IN as a value. Then to just call that when I was writing the file.

Answer 1

Have you thought about using a package that works natively with tabular datasets to support the operations you want - for example, pandas ?

import pandas as pd

f1=pd.read_csv("sortedBlad.csv")
f2=pd.read_csv("sortedNewStock.csv")

#Filter columns in f1
f1sub=f1[ ['eid', 'sku', 'name' ] ]

#Merge data
mergef = pd.merge(f1sub,f2,left_on='sku',right_on='SKU_IN')

The merge function will automatically handle alignment for you.

Answer 2

You are only appending the first column of sortedNewStock.csv ( SKU_IN ) to test_file2_array :

test_file2_array.append(row[0])

You also need to grab the QTY_IN column from that file. I suggest the following which adds both SKU_IN and QTY_IN :

test_file2_array.append(row[0:2])

and then changing

for SKU_IN in test_file2_array:

to

for SKU_IN, QTY_IN in test_file2_array:

So your code would look like this:

.
.
.
with open(test_file2, 'rt') as a:
    test_file2_array = [row[0:2] for row in csv.reader(a, delimiter=",")]

#print(test_file2_array)

file = open(finalFile, 'w')

for eachline in csv_file:
    for SKU_IN, QTY_IN in test_file2_array:
        if SKU_IN == eachline['sku']:
            print(SKU_IN + '-' + QTY_IN)
            file.write(','.join((eachline['eid'], eachline['sku'],
                                 eachline['QTY'], eachline['Name'],
                                 SKU_IN, QTY_IN, '\n')))

Answer 3

You aren't keeping the QTY_IN information from newStock.csv anywhere.

import csv

blad_file = "blad.csv"
newStock_file = "stock.csv" 

newStock_file_array = []

finalFile = "final.csv"

csv_blad = csv.DictReader(open(blad_file, 'r'), delimiter=',', quotechar='"')
csv_newStock = csv.DictReader(open(newStock_file, 'r'), delimiter=',', quotechar='"')

for line in csv_newStock:
    newStock_file_array.append(line)

with open(finalFile, 'w') as file_out:
    for eachline in csv_blad:
        for newStock in newStock_file_array:
            if newStock['SKU_IN'] == eachline['sku']:
                # print(newStock['SKU_IN']+'-'+newStock['QTY_IN'])
                file_out.write(eachline['eid']+','+eachline['sku']+','+eachline['QTY']+','+eachline['Name']+','+newStock['QTY_IN']+','+'\n')