Camel Unmarshal CSV 测试结果不同

Question

I'm unmarshalling a csv file in my route, but the actual application and the tests are giving me different results. This is the code:

 from("direct:start")
    .routeId("report")
    .pollEnrich("file:///Users/xxx/csv")
    .log(LoggingLevel.INFO, " File detected: ${header.CamelAwsS3Key}")
    .unmarshal()
    .csv()
    .convertBodyTo(String.class)
    .process(new Processor() {
      @Override
      public void process(Exchange exchange) throws Exception {

        List<String> ids = exchange.getIn().getBody(List.class);

In the application execution the expression exchange.getIn().getBody() returns b3e486c6-635b-4383-955c-06934ae6e5e4,00f8757a-baac-409d-bc6f-13e90aa75361 which is the expected result that I can convert to a list.

But in the test the same expression returns [[b3e486c6-635b-4383-955c-06934ae6e5e4], [00f8757a-baac-409d-bc6f-13e90aa75361]] , a list of lists. Then the conversion List<String> ids = exchange.getIn().getBody(List.class); returns null .

The unit test code is a simple use case:

    @ExtendWith(MockitoExtension.class)
class RouteBuilderTest extends CamelTestSupport {
 @Test
  void sendReport() throws Exception {
    
    context.start();
    template.sendBody("direct:report", null);
  }

}

What could be causing this behaviour?

Answer 1

I'm unmarshalling a csv file in my route, but the actual application and the tests are giving me different results.

It's much more likely that you're simply reading different file. The.log(LoggingLevel.INFO, " File detected: ${header.CamelAwsS3Key}") part makes me question the if you're even using the same RouteBuilder in your actual application.

Try checking out the body before marshalling it to see if its really the Camel CSV Unmarshal giving you the trouble or if the file contents are different or if it's the consumer endpoint that is acting differently.

rom("direct:start")
    .routeId("report")
    .pollEnrich("file:///Users/xxx/csv")
    // To allow us to read body multiple times
    .convertBodyTo(String.class)
    // Display contents of the file 
    .log("file contents: \n ${body}")
    .unmarshal()
    .csv()
    // ...

Unit testing, folder and files

When it comes to unit testing applications that read files from folders you should avoid using hard coded paths. Contents of these paths can change at any moment which will invalidate the test.

There's also the fact that camels file component will by default move processed files to hidden .camel folder so the file wont be consumed again. Even with noop option set to true camel will keep track of which files have been already consumed using in-memory registry.

Instead you should create temporary directory and copy files from test resources to said directory for testing. Both Junit4 and Junit5 come with Temporary folder feature and you can use Apache Commons IO to quickly copy test resources to said folder.

Example: Apache Camel 3.18.1

package com.example;

import java.io.File;
import java.util.Properties;

import org.apache.camel.RoutesBuilder;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.mock.MockEndpoint;
import org.apache.camel.test.junit5.CamelTestSupport;
import org.apache.commons.io.FileUtils;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Example extends CamelTestSupport {
    
    static final Logger LOGGER = LoggerFactory.getLogger(InputOutputFileRoutesTests.class);

    @TempDir
    File inputFolder;

    @Test
    public void readFileExampleTest() throws Exception {
        
        copyResourceFileToInputFolder("HelloWorld.txt");
        MockEndpoint resultMockEndpoint = getMockEndpoint("mock:result");
        
        resultMockEndpoint.expectedMessageCount(1);
        resultMockEndpoint.message(0).equals("Hello World!");

        template.sendBody("direct:readFileExample", null);

        resultMockEndpoint.assertIsSatisfied();
    }

    @Override
    protected RoutesBuilder createRouteBuilder() throws Exception {
        return new RouteBuilder() {

            @Override
            public void configure() throws Exception {

                from("direct:readFileExample")
                    .routeId("readFileExample")
                    .pollEnrich("file:{{input.path}}")
                    .convertBodyTo(String.class)
                    .log("Body is: ${body}")
                    .to("mock:result");
            }
        };
    }

    private void copyResourceFileToInputFolder(String resourceName){

        ClassLoader classLoader = InputOutputFileRoutesTests.class.getClassLoader();
        File resourceFile = new File(classLoader.getResource(resourceName).getFile());
        File testFile = new File(inputFolder, resourceName);

        try {
            FileUtils.copyFile(resourceFile, testFile);
        } catch (Exception e) {
            LOGGER.error(e.getMessage(), e);
        }
    }
    
    @Override
    protected Properties useOverridePropertiesWithPropertiesComponent() {

        Properties properties = new Properties();
        properties.put("input.path", inputFolder.getPath());
        return properties;
    }
}

Apache Commons IO dependency.

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.11.0</version>
    <scope>test</scope>
</dependency>

As side note pollEnrich with adviceWith is pretty nice for testing routes that use polling consumer endpoint like camel-file or camel-ftp.

@Test
public void readFileExampleTest() throws Exception {
    
    AdviceWith.adviceWith(context(), "readFileExample", a -> {

        a.replaceFromWith("direct:readFileExample");
        a.weaveAddFirst()
            .pollEnrich("file:{{input.path}}");

        a.weaveAddLast()
            .to("mock:result");
    });

    copyResourceFileToInputFolder("HelloWorld.txt");

    MockEndpoint resultMockEndpoint = getMockEndpoint("mock:result");        
    resultMockEndpoint.expectedMessageCount(1);
    resultMockEndpoint.message(0).equals("Hello World!");

    startCamelContext();
    template.sendBody("direct:readFileExample", null);

    resultMockEndpoint.assertIsSatisfied();
}

@Override
public boolean isUseAdviceWith() {
    return true;
}