Is there a better way to implement multitenant using kafka?

Question

I'm trying to implement a multi tenant micro service using Spring Boot. I already implemented the web layer and the persistence layer. On web layer, I've implement a filter which sets the tenant id in a prototype bean (using ThreadLocalTargetSource), on persistence layer I've used Hibernate multi tenancy configuration (schema per tenant), they work fine, data is persisted in the appropriate schema. Currently I am implementing the same behaviour on messaging layer, using spring-kaka library, so far ir works the way I expected, but I'd like to know if there is a better way to do it.

Here is my code:

This si the class that manage a KafkaMessageListenerContainer:

@Component
public class MessagingListenerContainer {

    private final MessagingProperties messagingProperties;

    private KafkaMessageListenerContainer<String, String> container;

    @PostConstruct
    public void init() {
        ContainerProperties containerProps = new ContainerProperties(
                messagingProperties.getConsumer().getTopicsAsList());

        containerProps.setMessageListener(buildCustomMessageListener());

        container = createContainer(containerProps);
        container.start();
    }

    @Bean
    public MessageListener<String, String> buildCustomMessageListener() {
        return new CustomMessageListener();
    }

    private KafkaMessageListenerContainer<String, String> createContainer(
            ContainerProperties containerProps) {
        Map<String, Object> props = consumerProps();
        …
        return container;
    }

    private Map<String, Object> consumerProps() {
        Map<String, Object> props = new HashMap<>();
        …
        return props;
    }

    @PreDestroy
    public void finish() {
        container.stop();
    }

}

This is the CustomMessageListener:

@Slf4j
public class CustomMessageListener implements MessageListener<String, String> {

    @Autowired
    private TenantStore tenantStore; // Prototype Bean

    @Autowired
    private List<ServiceListener> services;

    @Override
    public void onMessage(ConsumerRecord<String, String> record) {
        log.info(“Tenant {} | Payload: {} | Record: {}", record.key(),
                record.value(), record.toString());

        tenantStore.setTenantId(record.key()); // Currently tenant is been setting as key

        services.stream().forEach(sl -> sl.onMessage(record.value()));

    }

}

This is a test service which would use the message data and tenant:

@Slf4j
@Service
public class ConsumerService implements ServiceListener {

    private final MessagesRepository messages;
    private final TenantStore tenantStore;

    @Override
    public void onMessage(String message) {
        log.info("ConsumerService {}, tenant {}", message, tenantStore.getTenantId());
        messages.save(new Message(message));
    }

}

Thanks for your time!

Answer 1

Just to be clear ( correct me if I'm wrong ): you are using the same topic(s) for all your tenants. The way that you distinguish the message according to each tenant is by using the message key which in your case is the tenant id.

A slight improvement can be done by using message headers to store the tenant id instead of the key. By doing this then you will not be limited to partitioning messages based on tenants.

Although the model described by you works it has a major security issue. If someone gets access to your topic then you will be leaking data of all your tenants.

A more secure approach is using topic naming conventions and ACL's ( access control lists ). You can find a short explanation here . In a nutshell, you can include the name of your tenant in the topic's name by either using a suffix or a prefix. eg: orders_tenantA, orders_tenantB or tenantA_orders, tenantB_orders

Then, using ACL's you can restrict which applications can connect to those specific topics. This scenario is also helpful if one of your tenants need to connect one of their applications directly to your Kafka cluster.