简体   繁体   English

Java - 逐行从S3对象读取(JSON)数据

[英]Java - Read (JSON) data from S3 object line by line

I have an S3 object that contains JsonL lines. 我有一个包含JsonL行的S3对象。 I want to read this object line by line in Java so that I can recursively keep parsing each line (ie, each json) into a POJO and eventually convert the object contents into a list of POJO. 我想在Java中逐行读取这个对象,这样我就可以递归地将每一行(即每个json)解析为POJO,并最终将对象内容转换为POJO列表。

As I understand, AmazonS3.getObject().getObjectContent() returns a stream to the object's content. 据我了解, AmazonS3.getObject().getObjectContent()返回对象内容的流。 My next step is to read a line and convert to string which I can then parse into a POJO. 我的下一步是读取一行并转换为字符串,然后我可以将其解析为POJO。 I am not sure how to operate on this InputStream to keep obtaining the next line in String format. 我不知道如何操作此InputStream以继续以String格式获取下一行。

Sample data : 样本数据 :

{"key1":"val", "key2":"val1", "key3":"val2"}
{"key1":"val3", "key2":"val4", "key3":"val5"}
{"key1":"val6", "key2":"val7", "key3":"val8"}

You have a lot of options for converting the InputStream to String. 您有很多选项可以将InputStream转换为String。 Take a look here . 看看这里

 String json = "{\"key1\":\"val\", \"key2\":\"val1\", \"key3\":\"val2\"}\n" +
        "{\"key1\":\"val3\", \"key2\":\"val4\", \"key3\":\"val5\"}\n" +
        "{\"key1\":\"val6\", \"key2\":\"val7\", \"key3\":\"val8\"}";

InputStream in = new ByteArrayInputStream(json.getBytes());

try (BufferedReader buffer = new BufferedReader(new InputStreamReader(in))) {
    buffer.lines().forEach(System.out::println);
}

If you can afford it you can use the jackson libraries and simplify your implementation: 如果你负担得起,你可以使用jackson库并简化你的实现:

Maven dependency Maven依赖

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.9.8</version>
</dependency>

Code

ObjectMapper mapper = new ObjectMapper();
MyObject obj = mapper.readValue(inputStream, MyObject.class);

For applying this logic to each line then loop over each line as explained here and apply the same readValue method which is overloaded for String as well. 为了将这个逻辑应用于每一行,然后按照此处的说明遍历每一行,并应用相同的readValue方法,该方法也为String重载。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM