Apache Camel Route Template / multiple Routes / aggregate definition

Question

I tried to instantiate multiple routes from the same camel route template which resulted in a misbehaviour in the evaluation of several simple expressions.

I am using camel 3.20.1 in a spring boot application and i am having problems creating routes from the following route template. I am using constants for the specific template parameter keys - these are also used in several expressions (simple expression etc). At one point in the route template / instantiation of a route based on the following route template, especially at the aggregate / completionSize definition, an expression (templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE)) evaluates to a value specified for a second route which is using this template.

Route template:

@Override
public void configure() throws Exception {
    routeTemplate("generic-data-file-based-template")
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX)
            .from(templateParameterString(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE))
            .filter(FILENAME_FILTER_PREDICATE)
            .aggregate(templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX), new GroupedMessageAggregationStrategy())
            .completionSize(templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE))
            .log("correlation completed by ${header." + Exchange.AGGREGATED_COMPLETED_BY + "} with ${header." + Exchange.AGGREGATED_SIZE + "} files")
            .setHeader(INTERNAL_HEADER_REFERENCE_DATE, headerSubstring2(header(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX), Exchange.FILE_NAME))
            .to(templateParameterString(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI));
}

private Expression templateParameterExpression(String value) {
    return simple("{{"+value+"}}");
}

Route I based on this template:

public void configure() throws Exception {
    templatedRoute("generic-data-file-based-template")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI, "sftp:localhost:22/test/application/cashflows?username=tester&password=password")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI, "mock:cashflow-watch-mock")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE, "CASHFLOW")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX, "[0-9]{8}(Flow_tot|Head_tot|IntPeriod_tot){1}.csv")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX, "[0-9]{8}")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX, "[0-9]{8}")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE, "3")
            .routeId("file-watch-1");
}

Route II based on this template:

public void configure() throws Exception {
    templatedRoute("generic-data-file-based-template")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI, "sftp:localhost:22/test/application/bookvalues?username=tester&password=password")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI, "mock:bookvalue-watch-mock")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE, "BOOKVALUE")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX, "BW_BVA_[0-9]{8}.csv")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX, "BW_BVA_[0-9]{8}.csv")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX, "[0-9]{8}")
            .parameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE, "1")
            .routeId("file-watch-2");
}

It depends on the order in which these two routes are added to the camel context, to which value the expression templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE) evaluates for both routes. For example, if file-watch-1 was added first, then the expression in file-watch-2 will evaluate to the value 3 instead of 1.

I debugged my code and saw that camel uses an expressions cache which returns the upper value for the second route - but only for the expression used in the completionSize definition. Other expressions have the right value.

I already took a look on the official documentation which isn't exactly stating that this is common behaviour for the way i specified the templateParameter.

So, am i doing something wrong? Is this an error in the framework? Common behaviour? Should i use templateBeans instead?

Thanks in advance!

Answer 1

I investigated my problem further and found out that the two routes i am instantiating are using the same instance of AggregationDefinition which evaluates the simple expression in completionSize always to the same value once set (expressions cache?.).

I fixed this behaviour using another signature of the method completionSize in the template itself (completionSize(String completionSize)) and used the property method to specify the size:

    @Override
public void configure() throws Exception {
    routeTemplate("generic-data-file-based-template")
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX)
            .templateParameter(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX)
            .from(property(RouteTemplateConstants.TEMPLATE_PARAMETER_FROM_URI))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILENAME_FILTER_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_GENERIC_DATA_TYPE))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_REFERENCE_DATE_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX))
            .setHeader(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE, templateParameterExpression(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE))
            .filter(FILENAME_FILTER_PREDICATE)
            .aggregate(simple(property(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX)), new GroupedMessageAggregationStrategy())
            .completionSize(property(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_COMPLETION_SIZE))
            .log("correlation completed by ${header." + Exchange.AGGREGATED_COMPLETED_BY + "} with ${header." + Exchange.AGGREGATED_SIZE + "} files")
            .setHeader(INTERNAL_HEADER_REFERENCE_DATE, headerSubstring2(header(RouteTemplateConstants.TEMPLATE_PARAMETER_FILE_CORRELATION_REGEX), Exchange.FILE_NAME))
            .to(property(RouteTemplateConstants.TEMPLATE_PARAMETER_TO_URI));
}

Now my code works as expected.