如何根据R中有效列数（NA除外）选择数据框中的某些列？

Question

I'm using R, and I have a dataframe with multiple columns. I want to run a code and automatically check the number of values (valid values, not NA) in each column. Then, it should select the columns that 50% of its rows are filled by valid values, and save them in a new dataframe.

Can anybody help me doing this? Thank you very much.

Is there any way that the codes can be applied for an uncertain number of columns?

Answer 1

Using purrr package, you can write function below to check for the percentage of missing values:

pct_missing <- purrr::map_dbl(df,~mean(is.na(.x)))

After that, you can select those columns that have less than 50% missing values by their names.

selected_column <- colnames(df)[pct_missing < 0.5]

To create a new dataset, you may use:

library(dplyr)
df_new <- df %>% select(one_of(selected_column))

Answer 2

You can create a function within R base also to automatically retrieve the colums matching the critria:

Function:

ColSel <- function(df){
vals <- apply(df,2, function(fo) mean(is.na(fo))) < .5
return(df[,vals])
}

Some toy data

## example
df1 <- data.frame(
    a = c(runif(19),NA),
    b = c(rep(NA,11),runif(9)),
    d = rep(NA,20),
    e = runif(20)
    )

Test

df2 <- ColSel(df1)