简体   繁体   中英

how to figure out trend per unique key. dataframe

I have a DataFrame with 2 cols

ColA| ColB 
D 2 
D 12 
D 15 
A 20 
A 40 
A 60 
C 60 
C 55 
C 70 
C 45 
L 45 
L 23 
L 10 
L 5

RESULT/Output would be

D UP
A UP
C FLAT
L Down
Where UP is result of adding up all the relevant Weights: each successive weight for each key, must be less than the previous weight. Example for UP you must have

Here's a simple technique, might not suit for all cases ie :

def sum_t(x):
    # Compare the value with previous value
    m = x > x.shift() 
    # If all of them are increasing then return Up
    if m.sum() == len(m)-1:
        return 'UP'
    # if all of them are decreasing then return Down
    elif m.sum() == 0:
        return 'DOWN'
    # else return flat
    else:
        return 'FLAT'

df.groupby('ColA')['ColB'].apply(sum_t)

Output:

ColA
A      UP
C    FLAT
D      UP
L    DOWN
Name: ColB, dtype: object

Using diff and crosstab

s=df.groupby('ColA').ColB.diff().dropna()#Dropna since the first value for all group is invalid 
pd.crosstab(df.ColA.loc[s.index],s>0,normalize = 'index' )[True].map({1:'Up',0:'Down'}).fillna('Flat')
Out[100]:
ColA
A      Up
C    Flat
D      Up
L    Down
Name: True, dtype: object

Variation to @Dark's idea, I would first calculate GroupBy + diff and then use unique before feeding to a custom function.

Then use logic based on min / max values.

def calc_label(x):
    if min(x) >= 0:
        return 'UP'
    elif max(x) <= 0:
        return 'DOWN'
    else:
        return 'FLAT'

res = df.assign(C=df.groupby('ColA').diff().fillna(0))\
        .groupby('ColA')['C'].unique()\
        .apply(calc_label)

print(res)

ColA
A      UP
C    FLAT
D      UP
L    DOWN
Name: C, dtype: object

Using numpy.polyfit in a custom def

This way you can tweak the gradiant you would class as 'FLAT'

def trend(x, flat=3.5):
    m = np.polyfit(np.arange(1, len(x)+1), x, 1)[0]
    if abs(m) < flat:
        return 'FLAT'
    elif m > 0:
        return 'UP'
    return 'DOWN'

df.groupby('ColA')['ColB'].apply(np.array).apply(trend)

Solution by applying linear regression on each ID associated points and specifying the trend by slope of id associated point in 2 dimensional space

import numpy as np
from sklearn import linear_model
def slope(x,min_slope,max_slope):
    reg = linear_model.LinearRegression()
    reg.fit(np.arange(len(x),x))
    slope =  reg.coef_[0][0]
    if slope < min_slope:
        return 'Down'
    if slope > max_slope:
         return 'Up'
    else 'Flat'
min_slope = -1
max_slope = 1
df['slopes'] = df.groupby('ColA').apply(lambda x: slope(x['ColB'],min_slope,max_slope))

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM