Assign and create values using existing factors for groups with non existing values in a R dataframe

Question

I have a huge dataset of a beetles counting experiment with the following exemplary structure:

species_name1 <- c("A", "A", "A", "A", "B") # two factors for name1
species_name2 <- c("a", "a", "b", "b", "c") # three factors for name2
date <- c("2021-06-02", "2021-08-20", "2021-06-15", "2021-08-20", "2021-08-20") # three date factors
number <- c("30", "30", "11", "15", "40") # number of encountered beetles for the "date"

df <- data.frame(species_name1, species_name2, date, number) # create dataframe

df$species_full_name <- gsub(" ", " ", paste(df$species_name1, df$species_name2)) # new column with merged data of the first two columns

df$date <- as.Date(df$date, format ="%Y-%m-%d") 
df$number  <- as.numeric(df$number)
df$species_name1 <- as.factor(df$species_name1)
df$species_name2 <- as.factor(df$species_name2)
df$species_full_name <- as.factor(df$species_full_name)

str(df)

Overall there are three date factors (2021-06-02, 2021-06-15, 2021-08-20), but not for every "species_full_name". I need to create a dataframe which includes every single of the three dates for the factors of the "species_full_name" column. For "species_full_name"-factors with not existing "date" in the originally dataframe dates R should write a '0' to the column "numbers".

I found a code which is nearly a solution for my target dataframe. The problem is that the other columns ("species_name1" and..."_name2") will disappear:

as.data.frame(xtabs(number ~ species_full_name+date, df)) # create every factor "date" for every factor "species_full_name" and give counting data in column "Freq"

I need a dataframe which is similar to this output, but with every column from the original dataframe "df". It's important to assume the values for the columns “species_name1” and “species_name2” too.

Thanks for your help!

Answer 1

You can use complete() from tidyr

complete(df, species_full_name,date) %>%
  mutate(number=if_else(is.na(number),0,number))

Output:

  species_full_name date       species_name1 species_name2 number
  <fct>             <date>     <fct>         <fct>          <dbl>
1 A a               2021-06-02 A             a                 30
2 A a               2021-06-15 NA            NA                 0
3 A a               2021-08-20 A             a                 30
4 A b               2021-06-02 NA            NA                 0
5 A b               2021-06-15 A             b                 11
6 A b               2021-08-20 A             b                 15
7 B c               2021-06-02 NA            NA                 0
8 B c               2021-06-15 NA            NA                 0
9 B c               2021-08-20 B             c                 40

However a data.table approach will be faster. You can use data.table and CJ() as follows:

# load library
library(data.table)

# set df as data.table
setDT(df)

# get unique values of species_full_name and date
species_full_name = unique(df$species_full_name)
date = unique(df$date)

# merge (and update number to 0 if NA, and the name1 and name2 columns)
merge(CJ(date,species_full_name),df,by=c('date','species_full_name'),all.x =  T) %>% 
  .[, number:=fifelse(is.na(number),0,as.double(number))] %>% 
  .[, c("species_name1","species_name2"):=tstrsplit(species_full_name, " ")] %>% 
  .[]

Output:

         date species_full_name species_name1 species_name2 number
       <Date>            <fctr>        <char>        <char>  <num>
1: 2021-06-02               A a             A             a     30
2: 2021-06-02               A b             A             b      0
3: 2021-06-02               B c             B             c      0
4: 2021-06-15               A a             A             a      0
5: 2021-06-15               A b             A             b     11
6: 2021-06-15               B c             B             c      0
7: 2021-08-20               A a             A             a     30
8: 2021-08-20               A b             A             b     15
9: 2021-08-20               B c             B             c     40