Drop rows of R data.table

Question

I have a data.table:

> dat
     Mutant       F1       F2       F3       F4       F5       F6       F7
  1:  A115D  6.53193  7.19020  8.45634  8.49147  9.28304 16.83618 10.70517
  2:  A115F  0.90377  4.33477  5.71287  6.63125  5.86933  9.41705 14.59203
  3:  A115G  3.26668  4.46146  5.42433  7.80924  8.52429 10.92138 11.27432
  4:  A115H  2.91278  5.09545  6.01828  8.18154  8.11368 11.98551 11.33009
  5:  A115I  9.35627  9.29640  9.78475 10.76222 12.80510 16.13456 16.51090
 ---                                                                      
313:   Y80R -1.19326 -2.05579 -1.16474  1.74387  4.79593  5.59487 11.35956
314:   Y80S -0.77282 -1.51611 -0.07168  3.16070  3.16795  7.73116 11.60527
315:   Y80T -0.16135 -0.05859  2.02493  3.28120  6.10268 11.71562 12.45665
316:   Y80V -0.24050 -0.59869  0.36746  3.07046  3.75905  9.17579 11.83179
317:   Y80W  0.77770 -0.10166  2.27790  6.11470  6.01080  9.47050 13.95344

I would like to drop all rows where a value from any of the columns F1 - F7 is greater than 10.

Reading the vignette, I got as far as

> dat[, .SD > 10, .SDcols=2:7]

althoug I'm not sure if I'm doing a meaningfull thing here. Anyway, this gives something like:

> dat[, .SD>10, .SDcols=2:7]
        F1    F2    F3    F4    F5    F6   F7
[1,] FALSE FALSE FALSE FALSE FALSE  TRUE TRUE
[2,] FALSE FALSE FALSE FALSE FALSE FALSE TRUE
[3,] FALSE FALSE FALSE FALSE FALSE  TRUE TRUE
[4,] FALSE FALSE FALSE FALSE FALSE  TRUE TRUE
[5,] FALSE FALSE FALSE  TRUE  TRUE  TRUE TRUE
[6,] FALSE FALSE FALSE FALSE  TRUE  TRUE TRUE

So now, I would want to filter out all rows where any entry is TRUE .

Answer 1

We can try

i1 <- setDT(dat)[, Reduce(`+`,lapply(.SD, ">", 10)), .SDcols=2:7]
i1
#[1] 1 0 1 1 3 0 0 1 0 0
dat[i1==0]

NOTE: Just noticed that @nicola posted a similar approach in the comments. I saw only his original comment before the edit.

---

Or use

dat[-dat[ , .I[max(unlist(.SD))>10] ,by = 1:nrow(dat) , .SDcols= 2:7]$V1]
 #  Mutant       F1       F2       F3      F4      F5      F6       F7
 #1:  A115F  0.90377  4.33477  5.71287 6.63125 5.86933 9.41705 14.59203
 #2:   Y80R -1.19326 -2.05579 -1.16474 1.74387 4.79593 5.59487 11.35956
 #3:   Y80S -0.77282 -1.51611 -0.07168 3.16070 3.16795 7.73116 11.60527
 #4:   Y80V -0.24050 -0.59869  0.36746 3.07046 3.75905 9.17579 11.83179
 #5:   Y80W  0.77770 -0.10166  2.27790 6.11470 6.01080 9.47050 13.95344

Answer 2

You can use the command which

dat[-which(rowSums(dat[,2:8]>10)>0),]

or as TTMOTT pointed out

dat[-(rowSums(dat[,2:8]>10)>0),]

slightly faster dat[-which(max(dat[,2:8])>10),]

It basically calculates the rows for which there is a true

Hope I didn't make a mistake my toy example was:

 lines ="Mutant,F1,F2,F3,F4,F5,F6,F7
A115D,6.53193,7.19020,8.45634,8.49147,9.28304,16.83618,10.70517
A115F,0.90377,4.33477,5.71287,6.63125,5.86933,9.41705,14.59203
A115G,3.26668,4.46146,5.42433,7.80924,8.52429,10.92138,11.27432
A115H,2.91278,5.09545,6.01828,8.18154,8.11368,11.98551,11.33009
A115I,9.35627,9.29640,9.78475,10.76222,12.80510,16.13456,16.51090"

con <- textConnection(lines)
dat <- read.csv(con)
dat

dat[rowSums(dat[,2:7]>10)>0,]

Answer 3

How about simply?

dat[-which(dat$F1>10 | dat$F2>10 | dat$F3>10 | dat$F4>10 | dat$F5>10 | dat$F6>10 | dat$F7>10)]

Benchmarks for the 4 proposed solutions

dat <- data.frame("xxxxx", F1=runif(10000, 1,20),
                  F2=runif(10000, 1,20),
                  F3=runif(10000, 1,20),
                  F4=runif(10000, 1,20),
                  F5=runif(10000, 1,20),
                  F6=runif(10000, 1,20),
                  F7=runif(10000, 1,20))

benchmark(replications = 100, dat[-which(dat$F1>10 | dat$F2>10 | dat$F3>10 | 
                                      dat$F4>10 | dat$F5>10 | dat$F6> 10 | 
                                      dat$F7>10),],columns = c('elapsed'))
  elapsed
1    0.26

  benchmark(replications = 100, dat[(rowSums(dat[,2:8]>10)==0),],columns = c('elapsed'))

  elapsed
1    0.17

dat <- as.data.table(dat)

benchmark(replications = 100, dat[!(Reduce("|",dat[,lapply(.SD,function(x) x>10),.SDcols=2:8]))],
          columns = c('elapsed'))

  elapsed
1    0.32

benchmark(replications = 100, dat[-dat[ , .I[max(unlist(.SD))>10] ,by = 1:nrow(dat) , .SDcols= 2:8]$V1],
          columns = c('elapsed'))
  elapsed
1    6.05