Spring Boot Kafka EmbeddedKafka 事务

Question

Hi I want test kafka consumer method with kafka-test. I have this method:

@KafkaListener(topics = TOPIC_OWNER)
@Transactional("chainedKafkaTransactionManager")
public void consumeFromOwnerTopic(ConsumerRecord<String, AccessAvro> accessAvro) {
    final UUID userId = textMapper.toUUID(accessAvro.value().getUserId());
    User user = getUserFromDatabase(userId);
    userRepository.save(user);
    }
}

Test

@SpringBootTest
@ActiveProfiles("test")
@TestPropertySource(locations = "classpath:bootstrap-test.yml")
@EmbeddedKafka(partitions = 1, topics = "access-owner", bootstrapServersProperty =         
"spring.kafka.bootstrap-servers")
class AccessConsumerTest extends Specification {

@Autowired
private KafkaTemplate<String, AccessAvro> template

AccessAvro prepareAccessAvro(){
    AccessAvro accessAvro = new AccessAvro();
    accessAvro.restaurantId = UUID.randomUUID().toString()
    accessAvro.userId = UUID.randomUUID().toString()
    return accessAvro
}

def "should add role from consumed topic"(){
    given:
    sendKafkaAcces()
    expect:
    1==1
}


private ListenableFuture<SendResult<String, AccessAvro>> sendKafkaAcces() {
    template.send("access-owner", prepareAccessAvro())
}

}

Here I have error with transaction when:

• I add annotation @Transaction("chained...") to the test method, I have error about replicas
• Try to delete "transaction-id-prefix" form properties, I have error about ChainedKafkaTransaction.class bc it's try to find it in properties
• When I try to not add @Transaction on test method, I have error: run in transaction How I can test it ?

Answer 1

1. "When I try to not add @Transaction on test method, I have error: run in transaction"

No transaction is in process; possible solutions: run the template operation within the scope of a template.executeInTransaction() operation, start a transaction with @Transactional before invoking the template method, run in a transaction started by a listener container when consuming a record

2. "I add annotation @Transaction("chained...") to the test method, I have error about replicas"

948 ERROR 5020 --- [quest-handler-7] kafka.server.KafkaApis                   : 
[KafkaApi-0] Number of alive brokers '1' does not meet the required replication factor 
'3' for the transactions state topic (configured via 
'transaction.state.log.replication.factor'). This error can be ignored if the cluster 
is starting up and not all brokers are up yet.
2020-11-10 22:18:53.023  WARN 5020 --- [           main] 
o.s.test.context.TestContextManager      : Caught exception while invoking 
'beforeTestMethod' callback on TestExecutionListener 

[org.springframework.test.context.transaction.TransactionalTestExecutionListener@5b619d14] for test method [public void com.przemarcz.auth.service.AccessConsumerTest.$spock_feature_0_0()] and test instance [com.przemarcz.auth.service.AccessConsumerTest@63c31664]

org.springframework.transaction.CannotCreateTransactionException: Could not create 
Kafka transaction; nested exception is 
org.apache.kafka.common.errors.TimeoutException: Timeout expired after 
60000milliseconds while awaiting InitProducerId; nested exception is 
org.springframework.transaction.CannotCreateTransactionException: Could not create 
Kafka transaction; nested exception is 
org.apache.kafka.common.errors.TimeoutException: Timeout expired after 
60000milliseconds while awaiting InitProducerId

3. "Try to delete "transaction-id-prefix" form properties, I have error about ChainedKafkaTransaction.class bc it's try to find it in properties"

Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.kafka.transaction.KafkaTransactionManager<java.lang.Object, java.lang.Object>' available: expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}