R average values without NA

Question

My problem is explained in the attached picture (link).

I have tried following codes without result:

df[paste0("combined_", df_of_column_names)] <- lapply(df, ave, na.rm =TRUE, df[["index Z"]]) 

which does not return an average value where there are NAs present

df[paste0("combined_", df_of_column_names)] <- lapply(df, ave(FUN=function(x) mean(x, na.rm=T)), df[["index Z"]]) 

which gives the error:

Error in FUN(x) : argument "x" is missing, with no default

Can somebody help me with this? Many thanks!

Answer 1

Without a reproducible example it is hard to give a relevant answer but try out:

library(dplyr)
df2 <- df %>% # df is your data frame
        group_by(`index Z`) %>% 
        summarise_all(.funs = mean, na.rm = TRUE) 
# expected output
left_join(df1[, 1], df2, by = `index Z`)

Answer 2

Using library dplyr. Check this example:

df1 %>% group_by(index) %>%
  summarise(modreturn1 = mean(return1,na.rm = T), modreturn2 = mean(return2,na.rm = T))

It will return a table summarizing the first two variables into their means (excluding NA 's). Now, if you really want as many rows as your original dataset: First, save the above query to a variable named resumen , then:

merge(df1[,"index"],resumen,all.x = T)

You are welcome :)

Answer 3

Similar answer as ANG but using data.table

library(data.table)
df <- setDT(df)
df2 <- df[,lapply(.SD,mean), by = `index Z`]
df2[df, on = `index Z`]

Answer 4

Using base RI was able to get this to work on a simple case similar to yours.

attach(warpbreaks)
wool[5] <- NA
df <- data.frame(wool = wool, break = breaks)
df <- cbind(df, df$wool)

df
   wool breaks breaks.1
1     A     26       26
2     A     30       30
3     A     54       54
4     A     25       25
5     A     NA       NA
6     A     52       52
...

lapply(df[,-1], function(x) ave(x, df[,1], FUN = function(x) mean(x, na.rm=TRUE)))

$breaks
 [1] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
 [9] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
[17] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
[25] 29.53846 29.53846 29.53846 25.25926 25.25926 25.25926 25.25926 25.25926
[33] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926
[41] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926
[49] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926

$breaks.1
 [1] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
 [9] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
[17] 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846 29.53846
[25] 29.53846 29.53846 29.53846 25.25926 25.25926 25.25926 25.25926 25.25926
[33] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926
[41] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926
[49] 25.25926 25.25926 25.25926 25.25926 25.25926 25.25926