Java Spark UDF，输入参数为Map类型

Question

I have hive table structure as below -

create table sessionize_data (
ip                  string                                      
cookie                  string                                      
userid                  string                                      
dattime                 string                                      
timestamp               bigint                                      
referrer                string                                      
request                 string                                      
useragent               string                                      
absoluteips             string                                      
xff                     string                                      
calltype                string                                      
url                     string                                      
unique_visitor          string                                      
referrer_url            string                                      
referrer_dqp            map<string,string>                          
dqp                     map<string,string>                          
params                  map<string,string>                          
events_subcategory      map<string,string>                          
session_id              string                                      
source                  string                                      
dwelltime               string                                      
traffic                 string                                      
viewdate                string );

Now I want to pass params map column in spark UDF as a parameter - I did it using following code-

spark.udf().register("populateUrl",
    new UDF3 < HashMap < String, String > , String, String, HashMap < String, String >> () {
        private static final long serialVersionUID = 1 L;
        @Override
        public HashMap < String, String > call(HashMap < String, String > params, String url_type, String source)
        throws Exception {
            // some operation here
            return params;
        }
    }, DataTypes.StringType);

When I use above code it is giving type error for Map type. Can anybody suggest how to build JAVA spark udf which accepts Map type column and returns Map type column. Thanks.

Answer 1

根据_Spark SQL编程指南 ， MapType的外部类型为java.util.Map因此您应该实现：

UDF3<java.util.Map<String, String>, String, String, java.util.Map<String, String>>