memory exhausted while trying to remove sd=0 column

Question

I'm trying to remove numeric columns with sd=0, and this error "Error: memory exhausted (limit reached?)" pops up, my "data" was read with fread.

data2 <- data[ - as.numeric(which(apply(data, 2, var) == 0))]
Error: memory exhausted (limit reached?)

Is there a way around this?

Answer 1

Try using the data.table syntax:

library(data.table)
idx <- data[, sapply(.SD, function(col) var(col)==0)]
data2 <- data[, -idx, with=FALSE]

Answer 2

您可以使用colVars从matrixStats一揽子方案通常快。

data2 <- data[matrixStats::colVars(as.matrix(data)) != 0, ]