Compare one column in a dataframe to two columns of another dataframe
Question
I have two dataframes and I need to know if the values of the first dataframe are between two values (min and max values) in the second dataframe.
I did something similar before with two other data frames, I used a nested loop and between {dplyr} . However, the other dataset only had three variables and I could make it work with 8 if statements. This is where I get stuck, dataframe1 has 62 variables and 477 observations and dataframe2 has 124 variables and 50 observations (min values and max values). Below I have an example of the two dataframes and the result I am looking for.
So I am looking for a solution where I don't have to write around a thousand if else statements. I hope someone can help or if this is even possible.
The example of how the data looks, I can still change the dataframes, however this is the point where I am at.
Df1
id type data1 data2 data3
1 1 ab 0 0 0
2 2 cd 0 0 0
3 3 dd 0 10 5
4 4 ed 0 0 0
5 5 kd 0 0 15
6 6 xd 0 5 0
7 7 ab 0 0 0
8 8 cd 0 0 0
9 9 dd 0 10 10
10 10 ed 0 0 0
11 11 kd 0 0 12
12 12 xd 0 12 0
13 13 ab 0 0 0
14 14 cd 0 0 0
15 15 dd 0 5 15
16 16 ed 0 0 0
17 17 kd 0 0 15
18 18 xd 0 7 0
19 19 ab 0 0 0
20 20 cd 0 0 0
21 21 dd 0 18 10
22 22 ed 0 0 0
23 23 kd 0 0 5
I usually match the "type" with each other and then match if the data is between the lower and upper boundary.
Df2
type data1 data1max data2 data2max data3 data3max
1 ab NA NA NA NA NA NA
2 dd NA NA 5 20 10 100
3 xd NA NA 1 30 NA NA
4 ed NA NA NA NA NA NA
5 cd NA NA NA NA NA NA
6 kd NA NA NA NA 5 20
And resulting in a count when the observed data matches the qualifying data.
Df3
id type qualifyingfields
1 1 ab 0
2 2 cd 0
3 3 dd 1
4 4 ed 0
5 5 kd 1
6 6 xd 1
7 7 ab 0
8 8 cd 0
9 9 dd 2
10 10 ed 0
11 11 kd 1
12 12 xd 1
13 13 ab 0
14 14 cd 0
15 15 dd 2
16 16 ed 0
17 17 kd 1
18 18 xd 1
19 19 ab 0
20 20 cd 0
21 21 dd 1
22 22 ed 0
23 23 kd 1
2 answers
solution1
0 2019-11-05 23:11:30
library(dplyr)
library(tidyr)
df1 %>%
right_join(., df2, by = "type", suffix = c("val", "min")) %>%
group_by(type, id) %>%
pivot_longer(-c(id, type), names_to = "data", values_to = "value") %>%
separate(col = data, into = c("data", "var"), sep = "(?<=\\d)") %>%
pivot_wider(names_from = var, values_from = value) %>%
group_by(id, type, data) %>%
mutate(qualifyingfields = sum(between(val, min, max), na.rm = T)) %>%
group_by(id, type) %>%
summarise(qualifyingfields = sum(qualifyingfields))
#> # A tibble: 23 x 3
#> # Groups: type, id [23]
#> id type qualifyingfields
#> <int> <chr> <int>
#> 1 1 ab 0
#> 2 2 cd 0
#> 3 3 dd 1
#> 4 4 ed 0
#> 5 5 kd 1
#> 6 6 xd 1
#> 7 7 ab 0
#> 8 8 cd 0
#> 9 9 dd 2
#> 10 10 ed 0
#> # ... with 13 more rows
Data:
df1 <- read.table(text=" id type data1 data2 data3
1 1 ab 0 0 0
2 2 cd 0 0 0
3 3 dd 0 10 5
4 4 ed 0 0 0
5 5 kd 0 0 15
6 6 xd 0 5 0
7 7 ab 0 0 0
8 8 cd 0 0 0
9 9 dd 0 10 10
10 10 ed 0 0 0
11 11 kd 0 0 12
12 12 xd 0 12 0
13 13 ab 0 0 0
14 14 cd 0 0 0
15 15 dd 0 5 15
16 16 ed 0 0 0
17 17 kd 0 0 15
18 18 xd 0 7 0
19 19 ab 0 0 0
20 20 cd 0 0 0
21 21 dd 0 18 10
22 22 ed 0 0 0
23 23 kd 0 0 5",
header=T, stringsAsFactors=F)
df2 <- read.table(text=" type data1 data1max data2 data2max data3 data3max
1 ab NA NA NA NA NA NA
2 dd NA NA 5 20 10 100
3 xd NA NA 1 30 NA NA
4 ed NA NA NA NA NA NA
5 cd NA NA NA NA NA NA
6 kd NA NA NA NA 5 20",
header=T, stringsAsFactors=F, na.strings = "NA")
solution2
-1 ACCPTED 2019-11-05 23:43:28
Here is a more general solution that applies to data regardless of how many data[n] columns are
library('dplyr')
library('tidyr')
# Make dataframes tidy
Df1_tidy <- Df1 %>%
gather(key='data_name', value='value', -(id:type))
Df2_tidy <- Df2 %>%
gather(key='data_name', value='value', -type) %>%
mutate(limit=ifelse(grepl('max', data_name), 'Max', 'Min'),
data_name=gsub('max', '', data_name)) %>%
spread(limit, value)
# Count qualifying fields
Df3 <- full_join(Df1_tidy, Df2_tidy) %>%
group_by(id, type) %>%
summarise(qualifyingfields = sum(value >= Min & value <= Max, na.rm=T)) %>%
ungroup()
Df3
# # A tibble: 23 x 3
# id type qualifyingfields
# <int> <chr> <int>
# 1 1 ab 0
# 2 2 cd 0
# 3 3 dd 1
# 4 4 ed 0
# 5 5 kd 1
# 6 6 xd 1
# 7 7 ab 0
# 8 8 cd 0
# 9 9 dd 2
# 10 10 ed 0
# # ... with 13 more rows
Get data (copied from @M-- response):
df1 <- read.table(text=" id type data1 data2 data3
1 1 ab 0 0 0
2 2 cd 0 0 0
3 3 dd 0 10 5
4 4 ed 0 0 0
5 5 kd 0 0 15
6 6 xd 0 5 0
7 7 ab 0 0 0
8 8 cd 0 0 0
9 9 dd 0 10 10
10 10 ed 0 0 0
11 11 kd 0 0 12
12 12 xd 0 12 0
13 13 ab 0 0 0
14 14 cd 0 0 0
15 15 dd 0 5 15
16 16 ed 0 0 0
17 17 kd 0 0 15
18 18 xd 0 7 0
19 19 ab 0 0 0
20 20 cd 0 0 0
21 21 dd 0 18 10
22 22 ed 0 0 0
23 23 kd 0 0 5",
header=T, stringsAsFactors=F)
df2 <- read.table(text=" type data1 data1max data2 data2max data3 data3max
1 ab NA NA NA NA NA NA
2 dd NA NA 5 20 10 100
3 xd NA NA 1 30 NA NA
4 ed NA NA NA NA NA NA
5 cd NA NA NA NA NA NA
6 kd NA NA NA NA 5 20",
header=T, stringsAsFactors=F, na.strings = "NA")
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.