cassandra 数据库中的 spring 批处理元数据

Question

The question is quite simple: can I create metadata schema for Spring Batch in Cassandra database? How can I do this if yes? I've read that Spring Batch requires RDBMS database for that and No-SQL databases are not supported. Is that still the limitation in Spring Batch and how can I override that issue eventually?

Answer 1

由于卡桑德拉不支持键的简单序列，Spring Batch 不支持将其用于作业存储库。

Answer 2

It is possible to extend Spring Batch to support Cassandra by customising ItemReader and ItemWriter.

Reader:

@Override
public Company read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
    final List<Company> companies = cassandraOperations.selectAll(aClass);

    log.debug("Read operations is performing, the object size is  {}", companies.size());

    if (index < companies.size()) {
        final Company company = companies.get(index);
        index++;
        return company;
    }

    return null;
}

Writer:

@Override
public void write(final List<? extends Company> items) throws Exception {
    logger.debug("Write operations is performing, the size is {}" + items.size());
    if (!items.isEmpty()) {
        logger.info("Deleting in a batch performing...");
        cassandraTemplate.deleteAll(aClass);
        logger.info("Inserting in a batch performing...");
        cassandraTemplate.insert(items);
    }

    logger.debug("Items is null...");
}

@Beans:

@Bean
public ItemReader<Company> reader(final DataSource dataSource) {
    final CassandraBatchItemReader<Company> reader = new CassandraBatchItemReader<Company>(Company.class);
    return reader;
}

@Bean
public ItemWriter<Company> writer(final DataSource dataSource) {
    final CassandraBatchItemWriter<Company> writer = new CassandraBatchItemWriter<Company>(Company.class);
    return writer;
}

Full source code can be found in GithubSpring-Batch-with-Cassandra