[英]Spring Batch Json custom ItemWriter
I want to write json format files from a database. 我想从数据库中写入json格式的文件。 I have this prototype of
ItemWriter
implementation, very simple. 我有这个
ItemWriter
实现的原型,非常简单。
import java.util.ArrayList;
import java.util.List;
import org.springframework.batch.core.ExitStatus;
import org.springframework.batch.core.StepExecution;
import org.springframework.batch.core.StepExecutionListener;
import org.springframework.batch.item.ItemWriter;
import org.springframework.core.io.Resource;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
public class CustomItemWriter<T> implements ItemWriter<T>, StepExecutionListener {
private Gson gson;
private Resource resource;
private boolean shouldDeleteIfExists = true;
private List<T> allItems = new ArrayList<T>();
@Override
public void write(List<? extends T> items) throws Exception {
System.out.println("this is the begin " + items.size());
allItems.addAll(items);
}
public Resource getResource() {
return resource;
}
public void setResource(Resource resource) {
this.resource = resource;
}
public boolean isShouldDeleteIfExists() {
return shouldDeleteIfExists;
}
public void setShouldDeleteIfExists(boolean shouldDeleteIfExists) {
this.shouldDeleteIfExists = shouldDeleteIfExists;
}
@Override
public ExitStatus afterStep(StepExecution arg0) {
//write ALL to the output file
System.out.println(gson.toJson(allItems));
return null;
}
@Override
public void beforeStep(StepExecution arg0) {
gson = new GsonBuilder().setDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'").disableHtmlEscaping().create();
}
}
The problem solved here, is that the write
method sends to the output file each commitInterval
a JSON array, I just wanted a unique JSON array in my file. 此处解决的问题是
write
方法将每个commitInterval
发送到输出文件一个JSON数组,我只想在文件中使用一个唯一的JSON数组。
Implementing StepExecutionListener
after the step runs; 步骤运行后实现
StepExecutionListener
; I can send to the output file the entire array and transform it to JSON, and write it to the output file (fine!). 我可以将整个数组发送到输出文件,并将其转换为JSON,然后将其写入输出文件(很好!)。
My question is, Is this the correct way to do that? 我的问题是, 这是正确的方法吗? I think in the benefit of write to the file each
commitInterval
, but I'm not sure with my solution. 我认为对每个
commitInterval
写入文件都commitInterval
,但是我不确定我的解决方案。 It works, but I don't want to solve this problem and to provake another. 它有效,但是我不想解决这个问题并提出另一个问题。
You want to actually flush the write to your file with each chunk otherwise you'll lose restartability. 您实际上希望将每个块刷新到文件的写入中,否则将失去可重新启动性。
Assuming you're okay with one JSON object per line, I would probably just use a FlatFileItemWriter
in combination with a custom LineAggregator
that converts each object to a JSON string. 假设您可以每行使用一个JSON对象,那么我可能只将
FlatFileItemWriter
与自定义LineAggregator
结合使用,该方法将每个对象转换为JSON字符串。 Something along these lines: 遵循以下原则:
public class JsonLineAggregator<T> implements LineAggregator<T>, StepExecutionListener {
private Gson gson = new Gson();
private boolean isFirstObject = true;
@Override
public String aggregate(final T item) {
if (isFirstObject) {
isFirstObject = false;
return "[" + gson.toJson(item);
}
return "," + gson.toJson(item);
}
public void setGson(final Gson gson) {
this.gson = gson;
}
@Override
public void beforeStep(final StepExecution stepExecution) {
if (stepExecution.getExecutionContext().containsKey("isFirstObject")) {
isFirstObject = Boolean.parseBoolean(stepExecution.getExecutionContext().getString("isFirstObject"));
}
}
@Override
public ExitStatus afterStep(final StepExecution stepExecution) {
stepExecution.getExecutionContext().putString("isFirstObject", Boolean.toString(isFirstObject));
return null;
}
}
EDIT: Updated the LineAggregator
implementation above to demonstrate how you'd make it output something that looked like an JSON list. 编辑:更新了上面的
LineAggregator
实现,以演示您如何使其输出类似于JSON列表的内容。
Note that you'd also want to register a FlatFileFooterCallback
to your FlatFileItemWriter
that added the final "]". 请注意,您还想将
FlatFileFooterCallback
注册到添加了最后的“]”的FlatFileItemWriter
。
public class JsonFlatFileFooterCallback implements FlatFileFooterCallback {
@Override
public void writeFooter(final Writer writer) throws IOException {
writer.write("]");
}
}
Thanks for a Solution . 感谢您的解决方案。
In Xml you can add this like that . 在Xml中,您可以像这样添加。
<property name="resource" value="file:opt/output.json" /> <!-- #{jobParameters['input.file.name']} -->
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean
class="com.package.JsonLineAggregator">
</bean>
</property>
</bean>
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.