[英]Spring batch FlatFileItemReader token error
I have two csv files in my filesystem which I need to process with spring batch based on some request type.我的文件系统中有两个 csv 文件,我需要根据某些请求类型使用 spring 批处理进行处理。
File for request type: EOD (without headers)请求类型的文件:EOD(无标题)
12345,BBG
23232,BBG
Another file for request type: ANCIENT (without headers)请求类型的另一个文件:ANCIENT(无标题)
12345,BBG,20201115
23232,BBG,20201115
Both file have mandatory first two fields, id
and source
.这两个文件都有强制性的前两个字段, id
和source
。 ANCIENT file can optionally have 3rd and 4th field startDate
and endDate
ANCIENT 文件可以选择具有第 3 和第 4 字段startDate
和endDate
How can I create an FlatFileItemReader
that will work for both files?如何创建适用于这两个文件的FlatFileItemReader
? Currently I have something like:目前我有类似的东西:
My ItemReader implementation to Read the csv file:我的 ItemReader 实现读取 csv 文件:
@Bean
@Scope(value = "step", proxyMode = ScopedProxyMode.INTERFACES)
public FlatFileItemReader<FixingDTO> fileReader(@Value("#{jobExecutionContext['type']}") String type) {
SomeType type = forName(type);
return new FlatFileItemReaderBuilder<MyDTO>()
.name("fileReader")
.resource(new ClassPathResource(MAP.get(type)))
.delimited()
.delimiter(",")
.names("id", "source", "startDate", "endDate")
.fieldSetMapper(myDTOMapper())
.build();
}
Mapper For the Fields:字段映射器:
public class MyDTOFieldMapper implements FieldSetMapper<MyDTO> {
@Override
public MyDTO mapFieldSet(FieldSet fieldSet) throws BindException {
MyDTO dto = new MyDTO();
dto.setId(fieldSet.readString("id"));
dto.setSource(fieldSet.readString("source"));
dto.setStartDate(formatDate(fieldSet.readString("startDate")));
dto.setEndDate(formatDate(fieldSet.readString("endDate")));
return dto;
}
private LocalDate formatDate(String dateString) {
return LocalDate.parse(dateString, DateTimeFormatter.ofPattern("yyyyMMdd"));
}
}
I get a Token Exception when running the job:运行作业时出现令牌异常:
Caused by: org.springframework.batch.item.file.transform.IncorrectTokenCountException: Incorrect number of tokens found in record: expected 4 actual 2
I want to simply process whatever is in the line and assign to the object variables.我想简单地处理行中的任何内容并分配给 object 变量。 StartDate and endDate can be null. StartDate 和 endDate 可以是 null。
There is field strict
(default value: true) inside of DelimitedLineTokenizer
, that property exactly what you are looking for. DelimitedLineTokenizer
内部有字段strict
(默认值:true),该属性正是您要查找的属性。 You should set that field in false.您应该将该字段设置为 false。
Description of behavior:行为描述:
/**
* Public setter for the strict flag. If true (the default) then number of
* tokens in line must match the number of tokens defined
* (by {@link Range}, columns, etc.) in {@link LineTokenizer}.
* If false then lines with less tokens will be tolerated and padded with
* empty columns, and lines with more tokens will
* simply be truncated.
*
* @param strict the strict flag to set
*/
public void setStrict(boolean strict) {
this.strict = strict;
}
Unfortunately there is not nice way to set strict field in builder, otherwise we can set in following way:不幸的是,在builder中设置严格字段不是很好的方法,否则我们可以通过以下方式设置:
@Bean
@Scope(value = "step", proxyMode = ScopedProxyMode.INTERFACES)
public FlatFileItemReader<FixingDTO> fileReader(@Value("#{jobExecutionContext['type']}") String type) {
SomeType type = forName(type);
return new FlatFileItemReaderBuilder<MyDTO>()
.name("fileReader")
.fieldSetMapper(myDTOMapper())
.lineTokenizer(new DelimitedLineTokenizer() {
{
setDelimiter(",");
setNames("id", "source", "startDate", "endDate");
setStrict(false);
}
})
.build();
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.