Iterating through an array in pig

Question

I have records with structures as follows :

"event" : [ {"x":"1","y":"2"} , {"x":"5","y":"2"}]
"event" : [ {"random":"r", "pol" : "t", "a" : "b"} , {"x":"4","y":5"}] 
"event" : [ {"random":"f", "pol" : "w", "a" : "r"} , {"x":"12","y":5"} , {"x":"6","y":"7"}] 

The fields of interest to me are x & y. For each record I need to extract the map that has highest value of x.

IE for first event, pick {"x":"5","y":"2"} , for second {"x":"4","y":5"} and for third {"x":"12","y":5"}

I know that we can use a UDF to iterate through each map in the array and pick the one with max x value, but is there a way where i can do this without writing a UDF?

Answer 1

you can do something like this.

REGISTER elephant-bird-core-4.3.jar;
REGISTER elephant-bird-hadoop-compat-4.5.jar;
REGISTER elephant-bird-pig-4.5.jar;

DEFINE JsonLoader com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad=true');

records = LOAD '$DATA_PATH' USING JsonLoader() AS (data: map[]);
events = FOREACH records GENERATE 
                                FLATTEN(data#'event') AS event;

grouped_events = COGROUP events by event#'x', event#'y';     

result = FOREACH grouped_events GENERATE
        MAX(events.event#'x'),
        MAX(events.event#'y');

The -nestedLoad option helps load json arrays, which we can flatten to separate events as above.