在下一行符合条件后仅保留一行

Question

I'd like to know how I can keep only the rows when a subsequent row in the group meets a certain criteria. The following data illustrates what I am trying to achieve;

Data is sorted by ID ascending and DATE in descending order.

The same ID only has one row or zero rows where Purchased = 'N' but can have zero, one, or more than one rows where Purchased = 'Y' .

I want to track the dates in which the EMPTY status changes;

ID      EMPTY   DATE
1        Y     03/01/2017
1        Y     02/01/2017
1        N     01/01/2017
2        Y     03/01/2017
3        N     03/01/2017
4        Y     03/01/2017
4        N     03/01/2017
4        Y     03/01/2017
4        Y     03/01/2017

Output:

I want to keep all the rows with EMPTY= 'N' :

ID     EMPTY   DATE
1        Y     02/01/2017
1        N     01/01/2017
2        Y     01/01/2017
3        N     03/01/2017
4        Y     03/01/2017
4        N     03/01/2017

I can use either sql or python to do this; so solutions for either or both languages are welcomed!

Answer 1

In case you are actually interested in using R:

library(dplyr)
df %>%
      mutate(lag.empty = lead(df$EMPTY,1)) %>%
      filter(lag.empty != EMPTY)  %>%
      select(-lag.empty)


#  ID EMPTY       DATE
#1  1     Y 02/01/2017
#2  1     N 01/01/2017
#3  2     Y 03/01/2017
#4  3     N 03/01/2017
#5  4     Y 03/01/2017
#6  4     N 03/01/2017

Data:

df <- structure(list(ID = c(1L, 1L, 1L, 2L, 3L, 4L, 4L, 4L, 4L), EMPTY = structure(c(2L, 
2L, 1L, 2L, 1L, 2L, 1L, 2L, 2L), .Label = c("N", "Y"), class = "factor"), 
DATE = structure(c(3L, 2L, 1L, 3L, 3L, 3L, 3L, 3L, 3L), .Label = c("01/01/2017", 
"02/01/2017", "03/01/2017"), class = "factor")), .Names = c("ID", 
"EMPTY", "DATE"), class = "data.frame", row.names = c(NA, -9L))

Answer 2

One way with dplyr in R

library(dplyr)
df1 %>% 
  group_by(ID) %>%  
  filter(n()==1 |(cumsum(cumsum(EMPTY == "N"))<2 & !duplicated(EMPTY)) )
# A tibble: 6 x 3
# Groups:   ID [4]
#     ID EMPTY       DATE
#  <int> <chr>      <chr>
#1     1     Y 03/01/2017
#2     1     N 01/01/2017
#3     2     Y 03/01/2017
#4     3     N 03/01/2017
#5     4     Y 03/01/2017
#6     4     N 03/01/2017

data

df1 <- structure(list(ID = c(1L, 1L, 1L, 2L, 3L, 4L, 4L, 4L, 4L), EMPTY = c("Y", 
 "Y", "N", "Y", "N", "Y", "N", "Y", "Y"), DATE = c("03/01/2017", 
"02/01/2017", "01/01/2017", "03/01/2017", "03/01/2017", "03/01/2017", 
"03/01/2017", "03/01/2017", "03/01/2017")), .Names = c("ID", 
 "EMPTY", "DATE"), class = "data.frame", row.names = c(NA, -9L
 ))

Answer 3

In my experience this is a much prettier task in R, but since you are looking for a python solution:

dict = {'id':id,'empty':empty,'date':date}
df1 = pd.DataFrame(dict)

After loading into a pd dataframe by method of your choice:

lag = list(df1.loc[1:,'empty'])
lag.append('NULL')                    ##to make list match frame rowcount
df1['empty_+1'] = lag
df1['check'] = df1['empty'] != df1['empty_+1']
df1.loc[(df1['check'] == True)]

Answer 4

In mysql, one approach is to

1) add automatic incremental row-id to the table

 ALTER TABLE table1 ADD row_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY;

2) left join the same table with one-row shifting

3) add selection conditions: (i) current row has 'N' empty, (ii) current row has 'Y' empty but the next row has 'N' empty

SELECT a.ID, a.Empty, a.Day 
FROM table1 a 
LEFT JOIN table1 b ON a.row_id + 1 = b.row_id
WHERE a.Empty = 'N' or (a.Empty = 'Y' and b.Empty = 'N')

RESULT

ID  Empty   Day
1   Y   02/01/2017
1   N   01/01/2017
2   Y   03/01/2017
3   N   03/01/2017
4   Y   03/01/2017
4   N   03/01/2017

DATA

CREATE TABLE table1 (ID int, EMPTY varchar(255), DAY varchar(255));
INSERT table1 VALUES (1,'Y','03/01/2017'),(1,'Y','02/01/2017'),(1,'N','01/01/2017'),(2,'Y','03/01/2017'),(3,'N','03/01/2017'),(4,'Y','03/01/2017'),(4,'N','03/01/2017'),(4,'Y','03/01/2017'),(4,'Y','03/01/2017');