获取Pandas DataFrame中行之间的差异值

Question

Hi I have a result set from psycopg2 like so

(
(timestamp1, val11, val12, val13, val14),
(timestamp2, val21, val22, val23, val24),
(timestamp3, val31, val32, val33, val34),
(timestamp4, val41, val42, val43, val44),
)

I have to return the difference between the values of the row (exception for the timestamp column). Each row would subtract the previous row values. The first row would be

timestamp, 'NaN', 'NaN' ....

This has to then be returned as a generic object Ie something like an array of the following objects

Group(timestamp=timestamp, rows=[val11, val12, val13, val14]

I was going to use Pandas to do the diff. Something like below works ok on the values

df = DataFrame().from_records(data=results, columns=headers)
diffs = df.set_index('time', drop=False).diff()

But diff also performs on the timestamp column and I can't get it to ignore a column while leaving the original timestamp column in place.

Also I wasn't sure it was going to be efficient to get the data into my return format as Pandas advises against row access

What would a fast way to get the result set differences in my required output format ?

Answer 1

Why did you set drop=False ? That puts the timestamps in the index (where they will not be touched by diff ) but also leaves a copy of the timestamps as a proper column, to be process by diff .

I think this will do what you want:

diffs = df.set_index('time').diff().reset_index()

Since you mention psycopg2, take a look at the docs for pandas 0.14, released just a few days ago, which features improved SQL functionality, including new support for postgresql. You can read and write directly between the database and pandas DataFrames.