将数据库查询结果从字符串有效转换为元组

Question

I have an sqlite query which returns the following python list of tuples:

[(u'1623,0.0,1651,2.0e-06,1679,5.0e-06,1707,7.0e-06,1735,9.0e-06,1762,1.1e-05,1790,1.4e-05'),
( u'1933,458273.7,1940,460182.6,1947,464019.3,1954,465922.8,1961,467856.0,1968,479385.0')]

Each string here contains a tuple of x,y pairs ieone item is 1623,0.0 The output which I want is:

[(1623,0.0,1651,2.0e-06,1679,5.0e-06,1707,7.0e-06,1735,9.0e-06,1762,1.1e-05,1790,1.4e-05),
(1933,458273.7,1940,460182.6,1947,464019.3,1954,465922.8,1961,467856.0,1968,479385.0)]

Is there an efficient way to perform this translation?

Maybe it can be performed at the sql level but I dont think so, my query is as follows:

select group_concat( COALESCE(timestamp, "")
                     || COALESCE(",", "")
                     || COALESCE(value, "") )
              from table where type='VALUE' group by eventid;

I cant find a way to concatenate more than one column without converting it to a string

Answer 1

Don't group in SQL, group in Python and avoid having to serialize then unserialize your values:

from itertools import groupby
from operator import itemgetter


cursor.execute('''select eventid, timestamp, value from table
                  where type="VALUE" order by eventid''')
for eventid, rows in groupby(cursor, itemgetter(0)):
    print eventid
    for _, timestamp, value in rows:
        print timestamp, value

Use grouping in SQL only when you need to aggregate your data (to sum, calculate averages, count, etc.) not when you need individual rows from the groups for further processing.

If you need your output exactly as described (a tuple with timestamp, value pairs repeated) you can produce that with:

from itertools import groupby
from operator import itemgetter

cursor.execute('''select eventid, timestamp, value from table
                  where type="VALUE" order by eventid''')
result = [tuple(chain.from_iterable(r[1:] for r in rows))
          for eventid, rows in groupby(cursor, itemgetter(0))]

eg for every unique eventid produce a tuple consisting of the chained timestamp and value columns.