使用Java在flink中进行JSON聚合

Question

I am reading JSON data from a file.

Sample Data

{"name":"user1","myparam0":false,"myparam1":"44750004-23df-4960-88be-ba0884291597","myparam2":"36A3BF29-23df-EE2A-76B9-19BC1C854BA7","myparam3":"http://www.seloger.com/","myparam4":"http://www.seloger.com/erreur-temporaire/husk-pie","ver":"4.0.0"}
{"name":"user1","myparam0":true,"myparam1":"44750004-8bff-4960-88be-ba0884291597","myparam2":"36A3BF29-88be-EE2A-76B9-19BC1C854BA7","myparam3":"","myparam4":"http://www.seloger.com/erreur-temporaire/binde","ver":"4.0.0"}

I have written a sample code to read from the file and converted data to JSON like this

DataStream<Object> input = env.readTextFile("file:///home/ravisankar/workspace/temporary/input.file")
                .map((line) -> {
                    return JSON.parseFull(line);
                });

Now I need to calculate how many myparam3 are empty in 15 seconds based on the name. and group by myparam4

Ex: {
  "user1": {
    "myparams3": 1,
    "myparam4": {
      "http://www.seloger.com/erreur-temporaire/binde": 1,
      "http://www.seloger.com/erreur-temporaire/husk-pie": 1
    }
  }
}

Is it possible to extract such data like this from Flink?? I don't see any examples working on JSON using Java. Thanks for your time

Answer 1

您可以将json字符串解析为对象，即通过jackson库，并照常操作Java对象流

Answer 2

you can use jackson to parse the json to an object then make a loop to count your elements

private ObjectMapper objectMapper = new ObjectMapper() ;
...
Object element = objectMapper.readValue( jsonString , Object.class );

or you can use a regex that matches "myparam3":"" and calculate tha matches

public static void main( String[] args ) throws IOException
{
    String str = "{\"name\":\"user1\",\"myparam0\":false,\"myparam1\":\"44750004-23df-4960-88be-ba0884291597\",\"myparam2\":\"36A3BF29-23df-EE2A-76B9-19BC1C854BA7\",\"myparam3\":\"http://www.seloger.com/\",\"myparam4\":\"http://www.seloger.com/erreur-temporaire/husk-pie\",\"ver\":\"4.0.0\"}\r\n" + 
            "{\"name\":\"user1\",\"myparam0\":true,\"myparam1\":\"44750004-8bff-4960-88be-ba0884291597\",\"myparam2\":\"36A3BF29-88be-EE2A-76B9-19BC1C854BA7\",\"myparam3\":\"\",\"myparam3\":\"\",\"myparam3\":\"\"\"myparam4\":\"http://www.seloger.com/erreur-temporaire/binde\",\"ver\":\"4.0.0\"}";

    Pattern pattern = Pattern.compile("\"myparam3\":\"\"");


    Matcher matcher = pattern.matcher(str);

    int count = 0;
    while (matcher.find()) {
        count++;
    }
    System.out.println("Matches found : " + count );
}