R/dplyr: Mutate based on multiple dynamic variable names

Question

I have a list of data frames each of which contains multiple variables that contain surface area values (ending in "_area"). For each surface area variable there is corresponding conversion factor (ending in “_unit”) that I want to use to calculate a third variable that contains the area in a standard unit of measurement. I want these variables to end in “_area_ha”.

Below are my sample data frames:

a <- tibble(a1_area = c(1,1,1), a2_area_unit = c(1,1,0.5), a2_area = c(1,1,1),
            a1_area_unit = c(1,0.5,0.5), abc = c(1,2,3))

b <- tibble(b1_area = c(1,1,1), b1_area_unit = c(1,1,0.5), b2_area = c(1,1,1),
            b2_area_unit = c(1,0.5,0.5), abc = c(1,2,3))

ab_list <- list(a, b)

names(ab_list) <- c("a", "b")

I know how to do to this with the help of a loop but would like to understand how this can be done in the tidyverse/dplyr logic. My loop (which gives me the desired output) looks like this:

df_names <- names(ab_list)

for (d in df_names) {

  df <- ab_list[[d]]
  var_names <- names(select(df, matches("_area$")))

    for (v in var_names) {
      int  <- df %>% select(all_of(v),)
      int2 <- df %>% select(matches(paste0(names(int), "_unit")))
      int3 <- int*int2
      names(int3) <- paste0(names(int), "_ha")
      df <- cbind(df, int3)
      rm(int, int2, int3)
      }

  ab_list[[d]] <- tibble(df)
  rm(df)
  }

> ab_list
$`a`
# A tibble: 3 x 7
  a1_area a2_area_unit a2_area a1_area_unit   abc a1_area_ha a2_area_ha
    <dbl>        <dbl>   <dbl>        <dbl> <dbl>      <dbl>      <dbl>
1       1          1         1          1       1        1          1  
2       1          1         1          0.5     2        0.5        1  
3       1          0.5       1          0.5     3        0.5        0.5

$b
# A tibble: 3 x 7
  b1_area b1_area_unit b2_area b2_area_unit   abc b1_area_ha b2_area_ha
    <dbl>        <dbl>   <dbl>        <dbl> <dbl>      <dbl>      <dbl>
1       1          1         1          1       1        1          1  
2       1          1         1          0.5     2        1          0.5
3       1          0.5       1          0.5     3        0.5        0.5

I have tried using lapply and mutate_at but my approach does not work. If I understand correctly, this is because my environment is nested and I cannot access x in the function that calculates the variable "ha".

ab_list %>% 
  lapply(function(x) mutate_at(x, vars(matches("_area$")), list(ha = ~.*x[[paste0(names(.),"_unit")]])))

Error: Column `a1_area_ha` must be length 3 (the number of rows) or one, not 0 

Is there a way to get the function within mutate_at to access a variable from the parent data frame based on the name of initial variable within the function?

I would of course be happy about any other suggestion for a tidyverse approach to calculate the "_ha" variables based on dynamic variable names.

Answer 1

Great question. Below is a base R solution. I am sure it can be adapted to a tidyverse solution (eg, with purrr::map2() ). Here I built a function that does a basic test and then used it with lapply() . Note: the answer is tailored for your example, so you'll need to adapt it if you have different column names for the value / units. Hope this helps!!

val_by_unit <- function(data) {

  df <- data[order(names(data))]

  # Selecting columns for values and units
  val <- df[endsWith(names(df), "area")]
  unit <- df[endsWith(names(df), "unit")]


  # Check names are multiplying correctly
  if(!all(names(val) == sub("_unit", "", names(unit)))) {
    stop("Not all areas have a corresponding unit")
  }

  # Multiplying corresponding columns
  output <- Map(`*`, val, unit)

  # Renaming output and adding columns   
  data[paste0(names(output), "_ha")] <- output
  data
}

Results :

lapply(ab_list, val_by_unit)

$a
# A tibble: 3 x 7
  a1_area a2_area_unit a2_area a1_area_unit   abc a1_area_ha a2_area_ha
    <dbl>        <dbl>   <dbl>        <dbl> <dbl>      <dbl>      <dbl>
1       1          1         1          1       1        1          1  
2       1          1         1          0.5     2        0.5        1  
3       1          0.5       1          0.5     3        0.5        0.5

$b
# A tibble: 3 x 7
  b1_area b1_area_unit b2_area b2_area_unit   abc b1_area_ha b2_area_ha
    <dbl>        <dbl>   <dbl>        <dbl> <dbl>      <dbl>      <dbl>
1       1          1         1          1       1        1          1  
2       1          1         1          0.5     2        1          0.5
3       1          0.5       1          0.5     3        0.5        0.5

Answer 2

The tidyverse functions work best with 'long' formatted data where each of your rows represents a unique data point. To do this, you will want to use the tidyr::pivot_longer function:

# Join dataframes
dplyr::bind_cols(a, b) %>%
# Convert to area columns to long format
tidyr::pivot_longer(
  cols = dplyr::ends_with('area'),
  names_to = 'site',
  values_to = 'area'
) %>%
# Convert unit columns to long format
tidyr::pivot_longer(
  cols = dplyr::ends_with('unit'),
  names_to = 'site2',
  values_to = 'unit'
) %>%
# Just extract first 2 characters of the site column to get unique ID
dplyr::mutate(
  site = stringr::str_sub(site, 1, 2)
) %>%
# Remove redundant columns
dplyr::select(abc, site, area, unit) %>%
# Calculate area in HA
dplyr::mutate(
  area_ha = area * unit
)

Once your data is in long format, you can just use dplyr::mutate to multiply your area column by the unit column to get an area_ha column. If you want to convert your data back to its original format, you can use tidyr::pivot_wider to convert the data back to a wide format, which would give you columns with names a1_area_ha, a2_area_ha, etc.