从data.frame获取行，该行满足由R中任意一个子条件组成的条件

Question

I have a data.frame that can contains N columns (N defined at runtime), and I want to get the rows within the data frame that satisfy N-1 conditions, in other words I want to get only the rows with a specific value for the first N-1 columns.

For instance if I have a data frame with four columns (A,B,C,D) and five rows:

A B C D
1 2 3 4
9 9 9 9
1 2 9 5
4 3 2 1
1 2 3 8

I would get all the rows with A==1 & B==2 & C==3, ie:

A B C D
1 2 3 4
1 2 3 8

But as said, the data frame can be composed of any amount of rows and columns (defined at runtime), and the values of the conditions may change.

I implemented this function (simplified):

getRows<-function(dataFrame, values) {
  conditions=rep(TRUE, dim(dataFrame)[1])
  for (k in 1:length(values)) {
    conditions=conditions&(dataFrame[,k]==values[k])
  }
  return(dataFrame[conditions,])
}

Of course, this assumes the values in the values vector are sorted with respect to the columns order of the data frame, and that the length of the vector is N-1.

The function works but I've the feeling that it is not really efficient to create the vector of boolean, evaluate boolean expressions in this way and so on... especially if the data frame contains many data.

Another solution that I found is:

getRows<-function(dataFrame, values) {
  tmp=dataFrame
  for (k in 1:length(values)) {
    tmp=tmp[tmp[,k]==values[k],]
  }
  return(tmp)
}

Basically this 'reduces' the data frame by filtering out all the rows that not satisfy each condition. But I think this is even worst, because it creates a new data frame object for each condition (ok always smaller, but anyway...).

So my question is: is there a method to do that more efficiently?

Answer 1

one possibility:

# if you are only checking for equalities
f <- function(df, values){
  # values must be a list with the columns names of df as names and the conditions
  # if you 
  y <- paste(names(values), unlist(values), sep="==", collapse=" & ")
  return(df[eval(parse(text=y), envir=df),])
  }

 l <- as.vector(1:3, "list")
 names(l) <- colnames(df)[-ncol(df)]

 f(df, l)
   A B C D
 1 1 2 3 4
 5 1 2 3 8

# you can also use other conditions
f <- function(df, values){
  # values must be a list with the columns names of df as names and the conditions
  # if you 
  y <- paste(names(values), unlist(values), collapse=" & ")
  return(df[eval(parse(text=y), envir=df),])
  }

 l <- as.vector(paste0(c("==", "<=", "=="), 1:3), "list")
 names(l) <- colnames(df)[-ncol(df)]

f(df, l)
  A B C D
1 1 2 3 4
5 1 2 3 8

Answer 2

Sometimes matrices are quicker than data.frames to operate on, so something along the lines of:

mat <- t(as.matrix(df[-ncol(df)))
boolMat <- (mat==values) # if necessary use match to reorder values to match columns of df
ind <- colSums(boolMat)==nrow(boolMat)
df[ind,]

The idea being that values will get recycled along the columns of the matrix (which are the rows of the dataframe). colSums is meant to be quicker than an apply , so the final line should be somewhat optimised compared to apply(boolMat, 2, all) .

The optimal solutions will depend on the size and proportions of the data; whether the entries are all integers; and maybe what proportion of matches you get in the data. So as @droopy mentions, you'll need to benchmark. My approach involves creating a copy of the data, so if your data is already approaching memory limits, then it might struggle - but maybe then you could generate your data in matrix rather than data.frame format to save the duplication.