从 csv 中删除冗余时间戳

Question

I have created a CSV file from the recording of different sensors using pandas DataFrame. The CSV file basically looks like this:

I would like to get rid of the redundant timestamps and instead have all sensor entries that share a timestamp appear in the same row (for example x2 and x3 in the image). Also, the labels that share a timestamp are always identical, but would need to be reduced as well.

So far, I've come across the drop_duplicate() function which only drops entire rows.

Edit: here's a text version of the example above:

timestamp,sensor_a,sensor_b,sensor_c,label
1,x1,,,0
2,,x2,,0
2,,,x3,0
3,x4,,,1
4,,,,1
5,,x6,,1
5,,,x7,1

Answer 1

I will assume that you store the data in a text file sensors.txt , so we con consolidate the data based on timestamp according to the following code

import pandas as pd

df = pd.read_csv('sensors.txt', delimiter=',', header=0)

df2 = df.groupby('timestamp').ffill()
df2['timestamp'] = df['timestamp']
df2 = df2.groupby('timestamp').bfill()
df2['timestamp'] = df['timestamp']
df2 = df2.drop_duplicates()

df = df2[['timestamp', 'sensor_a', 'sensor_b', 'sensor_c', 'label']]
print(df)

output

   timestamp sensor_a sensor_b sensor_c  label
0          1       x1      NaN      NaN      0
1          2      NaN       x2       x3      0
3          3       x4      NaN      NaN      1
4          4      NaN      NaN      NaN      1
5          5      NaN       x6       x7      1

further editing will be based on your questions in the comments below

Good Luck