简体   繁体   中英

Does Spring Batch release the heap memory after processing each batch?

I have a Spring batch job which contains only one step that reads a CSV file(containing approximately 2000 rows) using a FlatFileItemReader and writes the objects to the database. I have my own custom BeanWrapperFieldSetMapper which maps the rows to objects. The chunk size is set to 50, so I am expecting after writing the objects from each batch (50 objects), the heap memory to be released of those objects.

Since I'm leveraging the batch processing, I'm expecting at each given time to only have 50 CreditCardDebt objects. But instead, while processing the last batch I find the heap memory to contain 2000 CreditCardDebt objects.

What am I missing?

My BeanWrapperFieldSetMapper implementation:

@Component("CREDIT_CARD_DEBT_FIELD_SET_MAPPER_TEST")
public class TestDebtFieldSetMapper extends BeanWrapperFieldSetMapper<CreditCardDebt> {

    public TestDebtFieldSetMapper() {
        super.setPrototypeBeanName("CREDIT_CARD_DEBT_FIELD_SET_MAPPER_TEST");
    }

    @NonNull
    @Override
    public CreditCardDebt mapFieldSet(FieldSet fieldSet) {
        CreditCardDebt creditCardDebt = new CreditCardDebt();
        creditCardDebt.setAccount(fieldSet.readString(0));
        creditCardDebt.setCardholderId(fieldSet.readString(1));
        creditCardDebt.setDueDate(convertToLocalDateViaInstant(fieldSet.readString(2)));
        creditCardDebt.setDaysPastDue(fieldSet.readInt(3));
        creditCardDebt.setOverdueAmount(fieldSet.readDouble(4));
        creditCardDebt.setDirectDebitMinimumPayment(fieldSet.readDouble(5));
        creditCardDebt.setDirectDebitBalance(fieldSet.readDouble(6));
        creditCardDebt.setDirectDebitStatus(fieldSet.readChar(7));
        creditCardDebt.setDirectDebitType(DirectDebitType.valueOf(fieldSet.readString(8)));
        creditCardDebt.setCreatedDate(LocalDateTime.now());
        creditCardDebt.setFileName("BAL");
        return creditCardDebt;
    }

    private LocalDate convertToLocalDateViaInstant(String dateToConvert) {
        DateTimeFormatter formatters = DateTimeFormatter.ofPattern("yyyyMMdd");
        return LocalDate.parse(dateToConvert, formatters);
    }

This is left to the garbage collector. The relevant code section related to this question is in the ChunkOrientedTasklet . In the most basic form of ChunkOrientedTasklet , there are two calls:

Chunk<I> inputs = chunkProvider.provide(contribution);
chunkProcessor.process(contribution, inputs);

The ChunkProvider uses the ItemReader to read commit-interval items (or less if the item reader returns null ). And the ChunkProcessor uses ItemProcessor and ItemWriter to process and write items:

Chunk<O> outputs = transform(contribution, inputs);
write(contribution, outputs); // details of adjustments of output omitted here

This process is run repeatedly until the datasource is exhausted. So items of processed chunks should be garbage collected when the GC kicks in (since variables inputs / outputs are re-used) unless something is holding them in memory during the whole job execution.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM