R：识别一列中的非 NA 值并创建 dataframe 并选择另一列中的值

Question

I have a data frame (df) with multiple columns (45) and rows (20,000):

I want to filter each variable column by selecting only the rows with non-NA values and creating a separate data frame with the corresponding ID and Name for the rows selected. I then want to save each data frame with the corresponding variable name. For example, the output data frames would look like this and would be saved as Var1 and Var2 respectively.

Var 1 <

Var 2 <

I am currently trying to use this function on R and thinking of implementing a for loop.

df2 = lapply(df, function(x) {x[.is.na(x)]}).

This hasn't worked so well as it does not list the values from corresponding ID and Name column. This also doesn't create a dataframe.

Any suggestions will be greatly appreciated!

Answer 1

Here is how it can be done using dplyr & purrr

Note that next time instead of posting image of your data, please try create sample data in R and copy paste the dput of that sample data instead.

library(purrr)
library(dplyr)

data <- tibble(ID = c("A", "B", "C"),
  Name = c("D", "E", "F"),
  Var1 = c(1, NA, 2),
  Var2 = c(2, 2, NA),
  Var4 = c(NA, NA, 4))

columns <- names(data)[grepl("^Var", names(data))]


extract_na_item <- function(column_name, df) {
  df %>%
    filter(!is.na(!!sym(column_name))) %>%
    select(ID, Name)
}
list_var_not_na <- map(columns, extract_na_item, df = data)
names(list_var_not_na) <- columns

Here is the result

list_var_not_na
#> $Var1
#> # A tibble: 2 x 2
#>   ID    Name 
#>   <chr> <chr>
#> 1 A     D    
#> 2 C     F    
#> 
#> $Var2
#> # A tibble: 2 x 2
#>   ID    Name 
#>   <chr> <chr>
#> 1 A     D    
#> 2 B     E    
#> 
#> $Var4
#> # A tibble: 1 x 2
#>   ID    Name 
#>   <chr> <chr>
#> 1 C     F

And if you really want to have the variable assignment in global environment as you mentioned in OP you can do as below (Though I recommend just use the list to access the data instead)

list2env(list_var_not_na, envir = globalenv())

^{Created on 2021-05-03 by the reprex package (v2.0.0)}

Answer 2

You can use lapply like so:

cols <- grep('Var', names(df))
df2 <- lapply(df[cols], function(x) df[!is.na(x), -cols])
df2

#$Var1
#  ID Name
#1  A    D
#3  C    F

#$Var2
#  ID Name
#1  A    D
#2  B    E

#$Var4
#  ID Name
#3  C    F

data

df <- structure(list(ID = c("A", "B", "C"), Name = c("D", "E", "F"), 
    Var1 = c(1, NA, 2), Var2 = c(2, 2, NA), Var4 = c(NA, NA, 
    4)), class = "data.frame", row.names = c(NA, -3L))