I have a dataframe like the following:
2018-10-16 15:15:00 1810.388020
2018-10-16 15:20:00 1813.299467
2018-10-16 15:25:00 1812.550495
2018-10-16 15:45:00 18140.981919
2018-10-16 15:35:00 1814.473347
2018-10-16 15:40:00 1816.349779
2018-10-16 15:45:00 740.981919
2018-10-16 15:50:00 1819.066781
2018-10-16 15:55:00 1820.333191
I need a dataframe like the following:
2018-10-16 15:15:00 1810.388020
2018-10-16 15:20:00 1813.299467
2018-10-16 15:25:00 1812.550495
2018-10-16 15:35:00 1814.473347
2018-10-16 15:40:00 1816.349779
2018-10-16 15:50:00 1819.066781
2018-10-16 15:55:00 1820.333191
That is:For every row having more value than 1.5 the previous row value or less than 0.5 the previous row value, drop it
Idea is compare previous and also next values, chaining by |
for bitwise OR
and last by bitwise AND
with filtering by boolean indexing
:
m11 = df['col'] > df['col'].shift(-1) * 0.5
m12 = df['col'] > df['col'].shift() * 0.5
m21 = df['col'] < df['col'].shift(-1) * 1.5
m22 = df['col'] < df['col'].shift() * 1.5
df = df[(m11 | m12) & (m21 | m22)]
print (df)
col
2018-10-16 15:15:00 1810.388020
2018-10-16 15:20:00 1813.299467
2018-10-16 15:25:00 1812.550495
2018-10-16 15:35:00 1814.473347
2018-10-16 15:40:00 1816.349779
2018-10-16 15:50:00 1819.066781
2018-10-16 15:55:00 1820.333191
Detail :
df['m11'] = df['col'] > df['col'].shift(-1) * 0.5
df['m12'] = df['col'] > df['col'].shift() * 0.5
df['m21'] = df['col'] < df['col'].shift(-1) * 1.5
df['m22'] = df['col'] < df['col'].shift() * 1.5
df['m1'] = (df['m11'] | df['m12'])
df['m2'] = (df['m21'] | df['m22'])
df['mask'] = df['m1'] & df['m2']
print (df)
col m11 m12 m21 m22 m1 m2 \
2018-10-16 15:15:00 1810.388020 True False True False True True
2018-10-16 15:20:00 1813.299467 True True True True True True
2018-10-16 15:25:00 1812.550495 False True True True True True
2018-10-16 15:45:00 18140.981919 True True False False True False
2018-10-16 15:35:00 1814.473347 True False True True True True
2018-10-16 15:40:00 1816.349779 True True False True True True
2018-10-16 15:45:00 740.981919 False False True True False True
2018-10-16 15:50:00 1819.066781 True True True False True True
2018-10-16 15:55:00 1820.333191 False True False True True True
mask
2018-10-16 15:15:00 True
2018-10-16 15:20:00 True
2018-10-16 15:25:00 True
2018-10-16 15:45:00 False
2018-10-16 15:35:00 True
2018-10-16 15:40:00 True
2018-10-16 15:45:00 False
2018-10-16 15:50:00 True
2018-10-16 15:55:00 True
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.