[英]Hive : Merge two maps into one column
I have a hive table as我有一个 hive 表
create table mySource(
col_1 map<string, string>,
col_2 map<string, string>
)
here is how a record might look like这是记录的样子
col_1 col_2
{"a":1, "b":"2"} {"c":3, "d":"4"}
my target table looks like this我的目标表看起来像这样
create table myTarget(
my_col map<string, string>
)
now i want to combine the two columns from mySource into a single map and feed it to my target table.现在我想将 mySource 中的两列组合成一个 map 并将其提供给我的目标表。 Basically i want to write something like基本上我想写类似的东西
insert into myTarget
select
some_method(col_1, col_2) as my_col
from mySource;
is there a built in method in hive that can do this? hive 中是否有内置方法可以做到这一点? I tried a few things with collect_set but got lots of errors我用 collect_set 尝试了一些东西,但出现了很多错误
The solution using only built-in methods.该解决方案仅使用内置方法。 Explode both maps, UNION ALL results, collect array of key:value
, concatenate array with ','
, convert string to map using str_to_map
:分解两个地图,UNION ALL 结果,收集key:value
数组,用','
连接数组,使用str_to_map
:
with mytable as (--Use your table instead of this
select
map('a','1', 'b','2') as col_1, map('c','3', 'd','4') as col_2
)
select str_to_map(concat_ws(',',collect_set(concat(key,':',val)))) as mymap
from
(
select m1.key, m1.val
from mytable
lateral view explode(col_1) m1 as key, val
union all
select m2.key, m2.val
from mytable
lateral view explode(col_2) m2 as key, val
)s
;
Result:结果:
mymap
{"a":"1","b":"2","c":"3","d":"4"}
With brickhouse library it would be much easier:使用砖房图书馆会容易得多:
ADD JAR /path/to/jar/brickhouse-0.7.1.jar;
CREATE TEMPORARY FUNCTION COMBINE AS 'brickhouse.udf.collect.CombineUDF';
select combine(col_1, col_2) as mymap from mytable;
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.