等效于 sparkR 中的 R 映射值 function

Question

我正在寻找与mapvalues中 R 中的映射值 function 的等效项，例如

x <- c("a", "b", "c")
mapvalues(x, c("a", "c"), c("A", "C"))

我在 Scala 中找到了与此等效的功能，例如this 。 但我在 sparkR 的文档中找不到它。

编辑：

在链接的文档中有一个名为 map_values 的map_values ：

##D # Dataframe used throughout this doc
df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))
tmp3 <- mutate(df, v3 = create_map(df$model, df$cyl))
head(select(tmp3, map_keys(tmp3$v3), map_values(tmp3$v3)))
head(select(tmp3, element_at(tmp3$v3, "Valiant")))

但它的使用方式不同， map_values和map_keys在创建它们的位置的同一列中使用：“v3”不像我可以使用变量 v3 到 map 再次值从 model 到 cyl。

Answer 1

不幸的是，Spark function map_values 与 R function mapvalues 非常不同。 它返回来自 map 的值，而不是映射值。

正如其他人提到的那样，火花中的地图值没有直接替代品

这是执行您所描述的一种方法

sparkR.session()

# Create initial dataframe
df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))

# create a data frame with one column to find and one column to replace with
to_replace <- data.frame(find    = c('Valiant',      'Datsun 710',           'Ferrari Dino'),
                         replace = c('Valiant v2.0', 'Datsun 710 (Retired)', 'Ferrari Dino (Raptor Edition)'))

# good pratice would be to match only entries that exactly match the find column, but not those that contain using regular expressions
to_replace_rexp <- data.frame(find    = c('^Valiant$',      '^Datsun 710$',           '^Ferrari Dino$'),
                              replace = c('Valiant v2.0', 'Datsun 710 (Retired)', 'Ferrari Dino (Raptor Edition)'))
# Turn it into a spark dataframe
to_replace_spark <- createDataFrame(to_replace)
head(to_replace_spark)
#     find                       replace
#      Valiant                  Valiant v2.0
#   Datsun 710          Datsun 710 (Retired)
# Ferrari Dino Ferrari Dino (Raptor Edition)

# Left join the to replace table, adding two columns with matching results
joined_data <- SparkR::join(df, to_replace_spark, df$model == to_replace_spark$find, 'left_outer')
head(joined_data)
#         model  mpg cyl disp  hp drat    wt  qsec vs am gear carb       find              replace
#     Mazda RX4 Wag 21.0   6  160 110 3.90 2.875 17.02  0  1    4    4       <NA>                 <NA>
#         Mazda RX4 21.0   6  160 110 3.90 2.620 16.46  0  1    4    4       <NA>                 <NA>
# Hornet Sportabout 18.7   8  360 175 3.15 3.440 17.02  0  0    3    2       <NA>                 <NA>
#        Datsun 710 22.8   4  108  93 3.85 2.320 18.61  1  1    4    1 Datsun 710 Datsun 710 (Retired)
#    Hornet 4 Drive 21.4   6  258 110 3.08 3.215 19.44  1  0    3    1       <NA>                 <NA>
#           Valiant 18.1   6  225 105 2.76 3.460 20.22  1  0    3    1    Valiant         Valiant v2.0

# Replace the model column with the replace column if it isn't empty
joined_data_coal <- SparkR::mutate(joined_data, model = SparkR::coalesce(joined_data$replace, joined_data$model))
head(joined_data_coal)
#                     model  mpg cyl  disp  hp drat    wt  qsec vs am gear carb         find
#                Ford Pantera L 15.8   8 351.0 264 4.22 3.170 14.50  0  1    5    4         <NA>
#                    Merc 450SL 17.3   8 275.8 180 3.07 3.730 17.60  0  0    3    3         <NA>
#                   Honda Civic 30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2         <NA>
# Ferrari Dino (Raptor Edition) 19.7   6 145.0 175 3.62 2.770 15.50  0  1    5    6 Ferrari Dino
#                 Mazda RX4 Wag 21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4         <NA>
#                     Mazda RX4 21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4         <NA>
                    replace
#                          <NA>
#                          <NA>
#                          <NA>
# Ferrari Dino (Raptor Edition)
#                          <NA>
#                          <NA>

# Drop the columns we joined
data_final <- drop(joined_data_coal, c("find", "replace"))
head(data_final)
#                     model  mpg cyl  disp  hp drat    wt  qsec vs am gear carb
#                Ford Pantera L 15.8   8 351.0 264 4.22 3.170 14.50  0  1    5    4
#                    Merc 450SL 17.3   8 275.8 180 3.07 3.730 17.60  0  0    3    3
#                   Honda Civic 30.4   4  75.7  52 4.93 1.615 18.52  1  1    4    2
# Ferrari Dino (Raptor Edition) 19.7   6 145.0 175 3.62 2.770 15.50  0  1    5    6
#                 Mazda RX4 Wag 21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4
#                     Mazda RX4 21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4

# Final results
head(filter(data_final, data_final$cyl == "6"))
                      model  mpg cyl  disp  hp drat    wt  qsec vs am gear carb
# Ferrari Dino (Raptor Edition) 19.7   6 145.0 175 3.62 2.770 15.50  0  1    5    6
#                 Mazda RX4 Wag 21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4
#                     Mazda RX4 21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4
#                     Merc 280C 17.8   6 167.6 123 3.92 3.440 18.90  1  0    4    4
#                Hornet 4 Drive 21.4   6 258.0 110 3.08 3.215 19.44  1  0    3    1
#                  Valiant v2.0 18.1   6 225.0 105 2.76 3.460 20.22  1  0    3    1