How to configure Spring Boot App to connect to Axon Server and register event handler

Question

First, I have a spring boot application using axon 4.5.5 libraries connecting to server 4.2.4. Is this supported?

This spring boot app is suppose to listen to several events coming from the (main app emitted to the) axon server and here is my sprint boot axon client configuration in application.yml below.

Second, this application connects to the axon server but fails to handle any events causing all the said events to be blacklisted. I have trace it to the event handler registration is probably causing the issue. We are using the EventStore as the StreamableMessageSource when calling registerTrackingEventProcessor().

Do you guys have any ideas why the registered event handlers are not firing? I can see the spring boot app is connected to the axon server on the dashboard as well as the main app that fires the events. I can also see the fired events when searching it in the dashboard and their blacklisting in the axon server log. So my guess it is the configuration causing issues.

Here is my library versions (from pom.xml):

Version 4.5.5
axon-test
axon-spring-boot-autoconfigure
axon-spring-boot-starter
axon-modelling
axon-metrics
axon-messaging
axon-eventsourcing

Version 2.6.0
spring-boot-starter-web

Here is my application.yml axon fragment:

axon:
  axonserver:
    client-id: reporting-etl-client
    component-name: reporting-etl
    query-threads: ${AXON_QUERY_THREADS_MAX:50}
    servers: ${AXON_AXONSERVER_SERVERS:axonserver}
  serializer:
    events: jackson

and AxonConfig.java:

package com.fedeee.reporting.config;

import com.fedeee.reporting.axon.events.EventHandlerProjector;
import com.thoughtworks.xstream.XStream;
import org.axonframework.axonserver.connector.query.AxonServerQueryBus;
import org.axonframework.config.EventProcessingConfigurer;
import org.axonframework.config.ProcessingGroup;
import org.axonframework.eventhandling.EventBus;
import org.axonframework.eventhandling.TrackingEventProcessorConfiguration;
import org.axonframework.eventhandling.gateway.DefaultEventGateway;
import org.axonframework.eventhandling.gateway.EventGateway;
import org.axonframework.queryhandling.DefaultQueryGateway;
import org.axonframework.queryhandling.QueryBus;
import org.axonframework.queryhandling.QueryGateway;
import org.axonframework.serialization.AbstractXStreamSerializer;
import org.axonframework.serialization.Serializer;
import org.axonframework.serialization.xml.XStreamSerializer;
import org.reflections.Reflections;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.Assert;

import java.util.Set;

@Configuration
public class AxonConfig {

  private static final Logger LOG = LoggerFactory.getLogger(AxonConfig.class);

  /**
   * Correctly configuring the XStream serializer to avoid security warnings.
   */
  @Autowired
  public void configureXStream(Serializer serializer) {
    if (serializer instanceof AbstractXStreamSerializer) {
      XStream xStream = ((XStreamSerializer) serializer).getXStream();
      XStream.setupDefaultSecurity(xStream);
      xStream.allowTypesByWildcard(new String[] {"com.fedeee.pkg.api.events.**", "org.axonframework.**"});
    }
  }

  /**
   *
   * @param configurer
   * @param context
   */
  @Autowired
  public void configure(EventProcessingConfigurer configurer, ApplicationContext context) {
    LOG.info("Setting up TrackingEventProcessors for threads, batch size and other configurations..."
        + " annotated with @ProcessingGroup...");
    // find classes in the com.fedeee.* package that has methods annotated with @ProcessingGroup to configure
    Reflections reflections = new Reflections("com.fedeee.reporting.axon.events");
    Set<Class<?>> annotatedClasses = reflections.getTypesAnnotatedWith(ProcessingGroup.class);

    // Configure each identified class
    annotatedClasses.stream().forEach(annotatedClass -> {
      // Locate the appropriate spring bean to get appropriate values from each one.
      String beanName = annotatedClass.getName().substring(annotatedClass.getName().lastIndexOf(".") + 1);
      beanName = beanName.substring(0,1).toLowerCase() + beanName.substring(1);
      Object projObj = context.getBean(beanName);
      if (projObj instanceof EventHandlerProjector) {
        EventHandlerProjector projector = (EventHandlerProjector) projObj;
            LOG.info("Configuring EventHandlerProjector Bean '{}' with maxThreads: {} and batchSize: {}.",
            beanName, projector.getMaxThreads(), projector.getBatchSize());
        ProcessingGroup pgAnnotation = annotatedClass.getAnnotation(ProcessingGroup.class);
        String processingGroup = pgAnnotation.value();

        configurer.registerTrackingEventProcessor(
                processingGroup,
                org.axonframework.config.Configuration::eventStore,
                conf -> TrackingEventProcessorConfiguration.forParallelProcessing(projector.getMaxThreads())
                        .andBatchSize(projector.getBatchSize())
        ).registerHandlerInterceptor(processingGroup, configuration -> new EventHandlerLoggingInterceptor());
        // Enable logging for EventHandlers

        LOG.info(".. '{}' successfully configured with processing group '{}'.", beanName, processingGroup);
      } else {
        LOG.info(".. '{}' failed to configure with any processing group.", beanName);
      }
    });

    // TODO: handle tracking event processor initialization. See the axon mailing list thread:
    // *****************************************************************************************************************
    // https://groups.google.com/forum/#!topic/axonframework/eyw0rRiSzUw
    // In that thread there is a discussion about properly initializing the token store to avoid recreating query models.
    // I still need to understand more about this...
    // *****************************************************************************************************************
  }

//  @Autowired
//  public void configureErrorHandling(
//      EventProcessingConfigurer configurer, ErrorHandler errorHandler
//  ) {
//    configurer.registerDefaultListenerInvocationErrorHandler(c -> errorHandler);
//  }

  @Autowired
  public void registerInterceptors(QueryBus queryBus) {
    Assert.notNull(queryBus, "Invalid configuration, queryBus is null!");
    if (AxonServerQueryBus.class.isAssignableFrom(queryBus.getClass())) {
      queryBus.registerHandlerInterceptor(InterceptorSupport.authorizationHandlerInterceptor());
    }

  }

  @Bean
  public QueryGateway queryGateway(QueryBus queryBus) {
    return DefaultQueryGateway.builder().queryBus(queryBus).build();
  }

  @Bean
  public EventGateway eventGateway(EventBus eventBus) {
    return DefaultEventGateway.builder().eventBus(eventBus).build();
  }

}

Here is the EventHandlerProjector.java:

package com.fedeee.reporting.axon.events;

/**
 * Defines a contract to ensure we specify the number of threads and batch size to be allowed by the event
 */
public interface EventHandlerProjector {

  /**
   * Specifies the number of records per batch to be handled by the tracking processor.
   * @return
   */
  public default Integer getBatchSize() {
    return 1;
  }

  /**
   * Specifies the maximumn number of threads the tracking processor can be specified to use.
   * @return
   */
  public default Integer getMaxThreads() {
    return 1;
  }
}

And finally the event handler class:

package com.fedeee.reporting.axon.events.pkg;

import com.fedeee.api.UserInfo;
import com.fedeee.pkg.api.events.*;
import com.fedeee.reporting._shared.utils.MdcAutoClosable;
import com.fedeee.reporting.axon.events.EventHandlerProjector;
import com.fedeee.reporting.recover.packageevents.PackageCreatedDto;
import com.fedeee.reporting.recover.packageevents.RecoverPackageCreatedRequest;
import com.fedeee.reporting.recover.packageevents.RecoverPackageEditedDto;
import com.fedeee.reporting.recover.packageevents.RecoverPackageEventsService;
import com.fedeee.reporting.recover.translators.PackageEventTranslator;
import com.fedeee.reporting.recover.translators.UserInfoTranslator;
import com.fedeee.reporting.recover.user.RecoverUserDto;
import org.axonframework.config.ProcessingGroup;
import org.axonframework.eventhandling.EventHandler;
import org.axonframework.eventhandling.Timestamp;
import org.axonframework.messaging.annotation.MetaDataValue;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import java.time.Instant;

import static com.fedeee.api.AxonMessageMetadataKeys.USER_INFO;

/**
 * Event TrackingProcessor for PackageRecord-based Events handled by Axon.
 *
 * Configurations can be provided with the given batchSize and maxThreads options via .env or docker-compose.
 *
 * IMPORTANT! <code>AxonConfig</code> looks for the <em>@ProcessingGroup</em> annotation to set everything up properly.
 */
@ProcessingGroup(value = "Package-Record")
@Component
public class PackageRecordProjector implements EventHandlerProjector {

  @Value("${reporting-etl.tp-batch-size.package-record:1}")
  private Integer batchSize;

  @Value("${reporting-etl.tp-max-threads.package-record:5}")
  private Integer maxThreads;

  private RecoverPackageEventsService recoverPackageEventsService;

  @Autowired
  public PackageRecordProjector(RecoverPackageEventsService recoverPackageEventsService) {
    super();
    this.recoverPackageEventsService = recoverPackageEventsService;
  }

  private final Logger LOG = LoggerFactory.getLogger(this.getClass());

  /**
   * Event handler to handle packages created in Recover.
   *
   * This replaces the REST endpoint exposed and used by Recover in RecoverPackageEventsController.created().
   *
   * @param event
   * @param occurrenceInstant
   * @param userInfo
   */
  @EventHandler
  void on(PackageCreated event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
      mdcInit(event, userInfo, mdc);

      LOG.info("Handling PackageCreated event...");

      PackageCreatedDto createdDto = PackageEventTranslator.from(event, occurrenceInstant, userInfo);
      RecoverUserDto recoverUserDto = UserInfoTranslator.from(userInfo);
      RecoverPackageCreatedRequest request = new RecoverPackageCreatedRequest(createdDto, recoverUserDto);

      /* Once we are ready, comment this in and make appropriate changes to RecoverPackageEventsController to
        disallow duplication via the REST endpoint. (There are comments in there already.  :)  )
       */
      recoverPackageEventsService.save(request);

      LOG.info("Finished handling PackageCreated event.");
    } catch (Exception e) {
        LOG.info("An Exception has been thrown : ", e);
        throw e;
    }
  }

  @EventHandler
  void on(PackageTypeCorrected event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
      mdcInit(event, userInfo, mdc);

      // TODO: not implemented (Recover and Reporting-ETL)
      LOG.info("Finished handling PackageTypeCorrected event.");
    } catch (Exception e) {
      LOG.info("An Exception has been thrown : ", e);
      throw e;
    }
  }

  @EventHandler
  void on(PackageDeleted event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
      mdcInit(event, userInfo, mdc);

      // TODO: not implemented (Reporting-ETL)
      LOG.info("Finished handling PackageDeleted event.");
    } catch (Exception e) {
      LOG.info("An Exception has been thrown : ", e);
      throw e;
    }
  }

// TODO: not implemented (Recover and Reporting-ETL)
//  @EventHandler
//  void on(PackageIntegrated event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
//    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
//      mdcInit(event, userInfo, mdc);
//    } catch (Exception e) {
//      LOG.info("An Exception has been thrown : ", e);
//      throw e;
//    }
//  }

  @EventHandler
  void on(DataCaptureStarted event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
      mdcInit(event, userInfo, mdc);

      RecoverPackageEditedDto editedDto = PackageEventTranslator.from(event, occurrenceInstant, userInfo);

      /* Once we are ready, comment this in and make appropriate changes to RecoverPackageEventsController to
        disallow duplication via the REST endpoint. (There are comments in there already.  :)  )
       */
      recoverPackageEventsService.save(editedDto);

      LOG.info("Finished handling DataCaptureStarted event.");
     } catch (Exception e) {
      LOG.info("An Exception has been thrown : ", e);
      throw e;
    }
  }

  @EventHandler
  void on(DataCaptureEnded event, @Timestamp Instant occurrenceInstant, @MetaDataValue(USER_INFO) UserInfo userInfo) {
    try (MdcAutoClosable mdc = new MdcAutoClosable()) {
      mdcInit(event, userInfo, mdc);

      RecoverPackageEditedDto editedDto = PackageEventTranslator.from(event, occurrenceInstant, userInfo);

      /* Once we are ready, comment this in and make appropriate changes to RecoverPackageEventsController to
        disallow duplication via the REST endpoint. (There are comments in there already.  :)  )
       */
      recoverPackageEventsService.update(event.getPackageId(), editedDto);

      LOG.info("Finished handling DataCaptureEnded event.");
    } catch (Exception e) {
      LOG.info("An Exception has been thrown : ", e);
      throw e;
    }
  }

  @Override public Integer getBatchSize() {
    return batchSize;
  }

  @Override public Integer getMaxThreads() {
    return maxThreads;
  }

  private void mdcInit(PackageEvent event, UserInfo userInfo, MdcAutoClosable mdc) {
    mdc.put("PackageId", event.getPackageId());
    mdc.put("UserId", userInfo.getUserId());
    LOG.info("Handling package record event: {}", event);
  }
}

Here is the logs for today 2023/01/27...

.
.
.
2023-01-27 17:19:32.924 DEBUG 8 --- [MessageBroker-4] i.a.a.message.command.CommandCache       : Checking timed out commands
2023-01-27 17:19:37.924 DEBUG 8 --- [MessageBroker-4] i.a.axonserver.message.query.QueryCache  : Checking timed out queries
2023-01-27 17:19:37.924 DEBUG 8 --- [MessageBroker-3] i.a.a.message.command.CommandCache       : Checking timed out commands
2023-01-27 17:19:40.299 DEBUG 8 --- [ool-5-thread-16] i.a.axonserver.grpc.PlatformService      : Registered client : ClientComponent{client='reporting-etl-client', component='reporting-etl', context='default'}
2023-01-27 17:19:40.299  INFO 8 --- [ool-5-thread-16] i.a.a.logging.TopologyEventsLogger       : Application connected: reporting-etl, clientId = reporting-etl-client, context = default
2023-01-27 17:19:40.299 DEBUG 8 --- [ool-5-thread-16] i.a.a.c.version.ClientVersionsCache      : Version update received from client reporting-etl-client.default to version 4.4.
2023-01-27 17:19:40.332  INFO 8 --- [ool-5-thread-15] i.a.a.message.event.EventDispatcher      : Starting tracking event processor for : - 209301
2023-01-27 17:19:42.925 DEBUG 8 --- [MessageBroker-7] i.a.axonserver.message.query.QueryCache  : Checking timed out queries
2023-01-27 17:19:42.925 DEBUG 8 --- [MessageBroker-3] i.a.a.message.command.CommandCache       : Checking timed out commands
.
.
.
2023-01-27 18:56:08.163 DEBUG 8 --- [MessageBroker-1] i.a.a.message.command.CommandCache       : Checking timed out commands
2023-01-27 18:56:08.163 DEBUG 8 --- [MessageBroker-7] i.a.axonserver.message.query.QueryCache  : Checking timed out queries
2023-01-27 18:56:09.242 DEBUG 8 --- [pool-5-thread-9] i.a.a.message.command.CommandDispatcher  : Dispatch com.fedeee.pkg.api.commands.StartDataCapture to: recover-api-client.default
2023-01-27 18:56:09.257 DEBUG 8 --- [ool-5-thread-10] i.a.a.message.command.CommandDispatcher  : Sending response to: io.axoniq.axonserver.message.command.CommandInformation@17b317ff
2023-01-27 18:56:09.294 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Send request io.axoniq.axonserver.grpc.SerializedQuery@158b1b9a, with priority: 0
2023-01-27 18:56:09.294 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Remaining time for message: 72152808-ca89-4565-82dd-2675e52686e2 - 3600000ms
2023-01-27 18:56:09.300 DEBUG 8 --- [pool-5-thread-5] i.a.axonserver.message.query.QueryCache  : Remove messageId 72152808-ca89-4565-82dd-2675e52686e2
2023-01-27 18:56:09.300 DEBUG 8 --- [pool-5-thread-5] i.a.a.message.query.QueryDispatcher      : No (more) information for 72152808-ca89-4565-82dd-2675e52686e2 on completed
2023-01-27 18:56:09.300 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Send request io.axoniq.axonserver.grpc.SerializedQuery@6e93a34b, with priority: 0
2023-01-27 18:56:09.301 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Remaining time for message: 53aa1974-4012-452a-b451-2957003e4b9f - 3599999ms
2023-01-27 18:56:09.306 DEBUG 8 --- [pool-5-thread-4] i.a.axonserver.message.query.QueryCache  : Remove messageId 53aa1974-4012-452a-b451-2957003e4b9f
2023-01-27 18:56:09.306 DEBUG 8 --- [pool-5-thread-4] i.a.a.message.query.QueryDispatcher      : No (more) information for 53aa1974-4012-452a-b451-2957003e4b9f on completed
2023-01-27 18:56:09.319 DEBUG 8 --- [ool-5-thread-13] i.a.a.m.q.s.handler.DirectUpdateHandler  : SubscriptionQueryResponse for subscription Id 55748e24-e26f-4863-a5d0-a4eeff43da69 send to client.
2023-01-27 18:56:10.509 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Send request io.axoniq.axonserver.grpc.SerializedQuery@fd8d154, with priority: 0
2023-01-27 18:56:10.510 DEBUG 8 --- [st-dispatcher-2] i.a.a.grpc.GrpcQueryDispatcherListener   : Remaining time for message: d2d91224-557f-4735-933b-e4195e7e42f9 - 3599999ms
2023-01-27 18:56:10.514 DEBUG 8 --- [ool-5-thread-15] i.a.axonserver.message.query.QueryCache  : Remove messageId d2d91224-557f-4735-933b-e4195e7e42f9
2023-01-27 18:56:10.514 DEBUG 8 --- [ool-5-thread-15] i.a.a.message.query.QueryDispatcher      : No (more) information for d2d91224-557f-4735-933b-e4195e7e42f9 on completed
2023-01-27 18:56:13.163 DEBUG 8 --- [MessageBroker-2] i.a.a.message.command.CommandCache       : Checking timed out commands
2023-01-27 18:56:13.163 DEBUG 8 --- [MessageBroker-5] i.a.axonserver.message.query.QueryCache  : Checking timed out queries
.
.
.

Answer 1

As you might be running into known bugs, it's better to use 4.6 of server and framework, that said, it should likely work since the server API is pretty stable.

Could you also share the com.fedeee.reporting.axon.events.EventHandlerProjector class? Has that class @EventHandler annotated methods? Are there more classes with @EventHandler annotated methods?

I assume you are using Spring Boot and the Axon Framework starter, is did indeed the case?