如何在列中使用具有相同值的python中的CSV并创建具有唯一值的新CSV

Question

The issue that I have to face is that I have a csv file with same data on more than one column(here the unique_code), and I want to create a new csv having only one time the data on this column and the data from the other columns to be seperated by space if they are different(here the alternative_code).

Here is my csv.

Unique_code description alternative_code

33;product1;58

43;product2;95

33;product1;62

68;product3;11

43;product2;99

My desired csv result:

33;product1;58 62

43;product2;95 99

68;product3;11

Any ideas on how can I implement my new csv?

Answer 1

You can try something like:

vals = {}
names = {}
with open(input_filename,'r') as file:
    for line in file:
        l = line.replace("\n","")
        l = l.split(";")
        if(vals.has_key(l[0])):
            vals[l[0]].append(l[2])
        else:
            vals[l[0]] = [l[2]]
            names[l[0]] = l[1]

with open(output_filename,'w') as file:
    for key in vals.keys():

        res = str(key)+";"+str(names[key])+";"+str(vals[key][0])

        for i in range(0,len(vals[key])-1):
            res += " "+vals[key][i+1]
        res += '\n'

        file.write(res)

Answer 2

import csv

with open("my_file.csv", 'r') as fd:
    #import csv as list of list and remove blank line                
    data = [i for i in csv.reader(fd, delimiter=';') if i]                                       
    result = []
    for value in data:
        #check if product not in result 
        if value[1] not in [r[1] for r in result if r]:
            #add the new product to result with all values for the same product 
            result.append([value[0],
                           value[1],
                           ' '.join([line[2] for line in data if line[1] == value[1]])
                         ])
    print(result)

Answer 3

Finally I end up to this solution:

# -*- coding: utf-8 -*-
import csv

input_file_1 = "eidi.csv"
output_file = "output.csv"

parsed_dictionary={}

def concatenate_alter_codes(alter_code_list):
    result = ""
    for alter_code in alter_code_list:
        result = result + (alter_code + " ")
        print result
    return result[:-1]

#Read input csv file and create a dictionary with a list of all alter codes
with open(input_file_1,'r') as f:
    # put ; symbol as delimeter
    input_csv=csv.reader(f,delimiter=';')
    for row in input_csv:
        # if the key exists in the dictionary
    if row[0] in parsed_dictionary:
        parsed_dictionary[row[0]][0].append(row[2])
    else:
        parsed_dictionary[row[0]] = [[row[2]], row[1], row[3], row[4], row[5], row[6]]

#create new csv file with concatenated alter codes

with open(output_file,'w') as f:
    for key in parsed_dictionary:
                f.write(key + ";" + concatenate_alter_codes(parsed_dictionary[key][0]) + ";" + parsed_dictionary[key][1] + ";" + parsed_dictionary[key][2] + ";" + parsed_dictionary[key][3] + ";" + parsed_dictionary[key][4] + ";" + parsed_dictionary[key][5] + "\n")

Answer 4

littletable is a thin CSV-wrapper I wrote a number of years ago. Tables in littletable are lists of objects, with some helper methods for filtering, joining, pivoting, plus easy import/export of CSV, JSON, and fixed format data. Like pandas, it helps with the data import/export, but doesn't have all the other numeric analytical features that pandas has. It also keeps all the data in memory as a list of Python objects, so it wouldn't handle millions of rows as well as pandas would. But if your needs are modest, then it might be a shorter learning curve to work with littletable.

To load your initial raw data into a littletable Table starts with:

import littletable as lt
data = open('raw_data.csv')
tt = lt.Table().csv_import(data, fieldnames="id name altid".split(), delimiter=';')

(If there were a header row in your input file, csv_import would use that and would not require that you specify fieldnames .)

Printing out the rows looks just like iterating over a list:

for row in tt:
    print(row)

prints:

{'name': 'product1', 'altid': '58', 'id': '33'}
{'name': 'product2', 'altid': '95', 'id': '43'}
{'name': 'product1', 'altid': '62', 'id': '33'}
{'name': 'product3', 'altid': '11', 'id': '68'}
{'name': 'product2', 'altid': '99', 'id': '43'}

Because we'll be grouping and joining on the id attributes, we add an index:

tt.create_index("id")

(Unique indexes can be created also, but in this case, there are duplicate values in your raw input with the same id.)

Tables can be grouped by one or more attributes, and then each group of records can be passed to a function to give an aggregate value for that group. In your case, you want all the collected altids for each product id .

def aggregate_altids(rows):
    return ' '.join(set(row.altid for row in rows if row.altid != row.id))
grouped_altids = tt.groupby("id", altids=aggregate_altids)

for row in grouped_altids:
    print(row)

Gives:

{'altids': '62 58', 'id': '33'}
{'altids': '99 95', 'id': '43'}
{'altids': '11', 'id': '68'}

Now we'll join this table with the original tt table on id , and collapse out duplicates:

tt2 = (grouped_altids.join_on('id') + tt)().unique("id")

And print out the results:

for row in tt2:
    print("{id};{name};{alt_ids}".format_map(vars(row)))

Giving:

33;product1;58 62
43;product2;95 99
68;product3;11

The total code without the debugging looks like:

# import
import littletable as lt
with open('raw_data.csv') as data:
    tt = lt.Table().csv_import(data, fieldnames="id name altid".split(), delimiter=';')
tt.create_index("id")

# group
def aggregate_altids(rows):
    return ' '.join(set(row.altid for row in rows if row.altid != row.id))
grouped_altids = tt.groupby("id", alt_ids=aggregate_altids)

# join, dedupe, and sort
tt2 = (grouped_altids.join_on('id') + tt)().unique("id").sort("id")

# output
for row in tt2:
    print("{id};{name};{alt_ids}".format_map(vars(row)))