如何按组保留值（删除NA）？

Question

My data has 3 proportion variables by geography and year. I am trying to aggregate this data by dropping the NAs and collating the values across 3 different variables by year and geography.

The example dataframe is as follows:

df <- data.frame(FIPS = c("01001", "01001", "01001","01001", "01001", "01001", "01003", "01003", "01003", "01003", "01003", "01003"),
                 Year = c(2000, 2000, 2000, 2001, 2001, 2001, 2000, 2000, 2000, 2001, 2001, 2001),
                 prop1 = c(0.7, NA, NA, 0.5, NA, NA, 0.3, NA, NA, 0.5, NA, NA),
                 prop2 = c(NA, 0.3, NA, NA, 0.5, NA, NA, 0.3, NA, NA, 0.1, NA),
                 prop3 = c(NA, NA, NA, NA, NA, NA, NA, NA, 0.4, NA, NA, 0.4))

I am guessing this can be done by aggregate or distinct command in R but not sure exactly how to proceed, as none of the way below gives me the dataframe I want.

df2 = aggregate(df,by = list(df$FIPS, df$Year), FUN = ???)

df2 <- df %>% distinct(FIPS, Year, .keep_all = TRUE)

The expected dataframe is as follows:

df2 <- data.frame(FIPS = c("01001", "01001",  "01003", "01003" ),
                  Year = c(2000,  2001,  2000,  2001),
                  prop1 = c(0.7,  0.5, 0.3, 0.5 ),
                  prop2 = c(0.3, 0.5,  0.3, 0.1),
                  prop3 = c(NA,  NA, 0.4, 0.4))

So basically, I want the code to search for the existing proportions (or NA if missing) in the 'prop' variables by Year and FIPS and create the new dataframe with unique FIPS and Year with the proportions collated. If anyone can point out the errors in what I am trying or give me another solution, it will be very much appreciated!

Answer 1

You could use dplyr for this:

library(dplyr)
df %>%
  group_by(FIPS, Year) %>%
  summarise_at(vars(prop1:prop3), mean, na.rm = T) %>%
  replace(is.na(.), NA)
# A tibble: 4 x 5
# Groups:   FIPS [?]
  FIPS   Year prop1 prop2 prop3
  <fct> <dbl> <dbl> <dbl> <dbl>
1 01001  2000   0.7   0.3  NA  
2 01001  2001   0.5   0.5  NA  
3 01003  2000   0.3   0.3   0.4
4 01003  2001   0.5   0.1   0.4

Answer 2

In base R you can try,

do.call(rbind, lapply(split(df, list(df$FIPS, df$Year)), function(i) 
                                                     sapply(i, function(j) j[!is.na(j)][1])))

#           FIPS Year prop1 prop2 prop3
#01001.2000    1 2000   0.7   0.3    NA
#01003.2000    2 2000   0.3   0.3   0.4
#01001.2001    1 2001   0.5   0.5    NA
#01003.2001    2 2001   0.5   0.1   0.4

Answer 3

You can use data.table to achieve this-

library(data.table)
setDT(df)[, lapply(.SD, function(x) x[!is.na(x)][1L]), by =.(FIPS,Year)]

Output-

FIPS Year prop1 prop2 prop3
1: 01001 2000   0.7   0.3    NA
2: 01001 2001   0.5   0.5    NA
3: 01003 2000   0.3   0.3   0.4
4: 01003 2001   0.5   0.1   0.4

Note - This will be efficient, if you have large dataset