分钟平均CSV文件

Question

I have a big csv file with datetime and value recorded every 10 seconds. The csv file looks like this:

Datetime             Data  
2008-10-01 12:00:10, 34  
2008-10-01 12:00:20, 55  
2008-10-01 12:00:30, 46  
2008-10-01 12:00:40, 33  
2008-10-01 12:00:50, 55  
2008-10-01 12:01:00, 21  
2008-10-01 12:01:10, 2  
2008-10-01 12:01:20, 34  
2008-10-01 12:01:30, 521  
2008-10-01 12:01:40, 45  
2008-10-01 12:01:50, 32  
2008-10-01 12:02:00, 34

I want to write a script that would calculate minute average and write it in a new csv file giving the following output:

Datetime             Data  
2008-10-01 12:00:00, 40.67  
2008-10-01 12:01:00, 111.33

Any idea how this can be done and any suggestions about modules that I should look into or any examples.

Answer 1

It seems to me the easiest way is just to treat the time as a string, rather than a time, and use itertools.groupby :

from csv import reader
from itertools import groupby

lines = """Datetime             Data
2008-10-01 12:00:10, 34
2008-10-01 12:00:20, 55
2008-10-01 12:00:30, 46
2008-10-01 12:00:40, 33
2008-10-01 12:00:50, 55
2008-10-01 12:01:00, 21
2008-10-01 12:01:10, 2
2008-10-01 12:01:20, 34
2008-10-01 12:01:30, 521
2008-10-01 12:01:40, 45
2008-10-01 12:01:50, 32
2008-10-01 12:02:00, 34"""

lines = iter(lines.splitlines())

# above this is just for testing, really you'd do
# with open('filename', 'rb') as lines:
# and indent the rest

next(lines)

for minute, group in groupby(reader(lines), lambda row: row[0][:16]):
    group = list(group)
    print minute, sum(float(row[1]) for row in group) / len(group)

Answer 2

Use the csv.reader to parse the file and a dictionary to cluster the results. The str.rpartition method can split-off the seconds. Use sum and len to compute the average:

data = '''\
2008-10-01 12:00:10, 34  
2008-10-01 12:00:20, 55  
2008-10-01 12:00:30, 46  
2008-10-01 12:00:40, 33  
2008-10-01 12:00:50, 55  
2008-10-01 12:01:00, 21  
2008-10-01 12:01:10, 2  
2008-10-01 12:01:20, 34  
2008-10-01 12:01:30, 521  
2008-10-01 12:01:40, 45  
2008-10-01 12:01:50, 32  
2008-10-01 12:02:00, 34
'''.splitlines()

import csv

d = {}
for timestamp, value in csv.reader(data):
    minute, colon, second = timestamp.rpartition(':')
    if minute not in d:
        d[minute] = [float(value)]
    else:
        d[minute].append(float(value))

for minute, values in sorted(d.items()):
    avg_value = sum(values) / len(values)
    print minute + ',' + str(avg_value)