Test a Camel sFTP endpoint

Question

I've got the following route:

public void configure() throws Exception {
    from(ftpEndpoint)
        .routeId("import-lib-files")
        .log(INFO, "Processing file: '${headers.CamelFileName}' from Libri-FTP")
        .choice()
        .when(method(isFilenameAlreadyImported))
            .log(DEBUG, "'${headers.CamelFileName}' is already imported.")
        .endChoice()
        .otherwise()
        .bean(method(unzipLibFile))
        .bean(method(persistFilename))
        .log(DEBUG, "Import file '${headers.CamelFileName}'.")
        .endChoice()
    .end()
    .end();
}

inside the unzipLibFile processor bean the file from the ftp gets uncompressed and is written to the HD.

I want to test (integration test) this route, like:

• 1. Copy file to ftp
• 2. Start the route
• 3. evaluate the 'outcome'

I like:

  @Before
  public void setUp() throws Exception {
    // delete test-file from sftp
    final String uploaded = ftpPath + "/" + destination + "/libri-testfile.zip";
    final File uploadedFile = new File(uploaded);
    uploadedFile.delete();

    // delete unzipped test-file
    final String unzippedFile = unzipped + "/libri-testfile.xml";
    final File expectedFile = new File(unzippedFile);
    expectedFile.delete();

    // delete entries from db
    importedLibFilenameRepository.deleteAll();

    // copy file to ftp
    final File source =
    new ClassPathResource("vendors/references/lib.zip/libri-testfile.zip").getFile();
    final String target = ftpPath + "/" + destination + "/libri-testfile.zip";
    FileUtils.copyFile(new File(source.getAbsolutePath()), new File(target));
  }


  @Test
  @Ignore
  public void testStuff() throws Exception {
    // Well here is a problem, I can't fix at the moment
    // the Camel-Context within the SpringContext get started when the tests starts
    // during this process the Camel-Routes are executed and because i copied the file to
    // the ftp all is fine... but I don't want to have a sleep in a test, I want to start the
    // route (like commented code beneath the sleep)
    Thread.sleep(2000);

//    final Map<String, Object> headers = Maps.newHashMap();
//    headers.put("CamelFileName", "libri-testfile.zip");
//
//    final File file =
//        new ClassPathResource("vendors/references/lib.zip/libri-testfile.zip").getFile();
//    final GenericFile<File> genericFile =
//        FileConsumer.asGenericFile(file.getParent(), file, StandardCharsets.UTF_8.name(), false);
//
//    final String uri = libFtpConfiguration.getFtpEndpoint();
//    producer.sendBodyAndHeaders(uri, InOut, genericFile, headers);

    // test if entry was made in the database
    final List<ImportedLibFilename> filenames = importedLibFilenameRepository.findAll();
    assertThat(filenames).usingElementComparatorIgnoringFields("id", "timestamp")
    .containsExactly(expectedFilename("libri-testfile.zip"));

    // test if content of unzipped file is valid
    final String expected = unzipped + "/libri-testfile.xml";
    final Path targetFile = Paths.get(expected);
    final byte[] encoded = Files.readAllBytes(targetFile);
    final String actualFileContent = new String(encoded, Charset.defaultCharset());

    final String expectedFileContent = "This is my little test file for Libri import";
    assertThat(actualFileContent).isEqualTo(expectedFileContent);
  }

  private ImportedLibFilename expectedFilename(final String filename) {
    final ImportedLibFilename entity = new ImportedLibFilename();
    entity.setFilename(filename);
    return entity;
  }

The problem is:
All camel route are started automaticly and because I copied the file to the FTP the test is green. But I've a #sleep inside my test, which I don't want. I want no camel route starting and start only the route I need.

My questions are:

• 1. How can I prevent the Camel-Routes from starting automaticly
• 2. Is the commented code (in the test method) the right way to start a route manually?
• 3. What are best practices to test a camel route with a ftp

Answer 1

1. Use .autoStartup(yourVariable) in your routes to make their startup configurable. Set the variable to true in normal environments and to false in your test cases.
2. I don't see code to start a route???
3. Well, take a step back. Think about splitting your FTP route. For testing and more reasons:

For example split the route into an FTP and a processing route. The first does only the FTP transfer and then sends the received messages to the processing route (for example a direct: route).

Benefits:

• SRP : Both routes do just one thing and you can concentrate on it.
• Testability : You can test the processing route easily by sending messages to the direct: endpoint of the processing route. The tests can focus on one thing too.
• Extensibility : Imagine there is a new input channel (JMS, HTTP, whatever). Then you just add another input route that also sends to your processing route. Done.

When you really want to test the whole process from FTP file drop until the end, think about using the Citrus test framework or similar tooling. Camel route tests are (in my opinion) a kind of "Unit tests for Camel routes", not full integration tests.

Answer 2

Thx to @burki...

His advise to split the routes (Single Responsibility) helped me to solve my problem:

Here is the route:

The "Main-Route" consuming from the sFTP:

  @Override
  public void configure() throws Exception {
    // @formatter:off
    from(endpoint)
      .setHeader("Address", constant(address))
      .log(INFO, "Import Libri changeset: Consuming from '${headers.Address}' the file '${headers.CamelFileName}'.")
      .to("direct:import-new-file");
    // @formatter:on
  }

The first sub-route:

  @Override
  public void configure() throws Exception {
    // @formatter:off
    from("direct:import-new-file")
        .choice()
          .when(method(isFilenameAlreadyImported))
          .log(TRACE, "'${headers.CamelFileName}' is already imported.")
        .endChoice()
        .otherwise()
          .log(TRACE, "Import file '${headers.CamelFileName}'.")
          .multicast()
          .to("direct:persist-filename", "direct:unzip-file")
        .endChoice()
      .end()
    .end();
    // @formatter:on
  }

The two multicasts:

  @Override
  public void configure() throws Exception {
    // @formatter:off
      from("direct:persist-filename")
        .log(TRACE, "Try to write filename '${headers.CamelFileName}' to database.")
        .bean(method(persistFilename))
      .end();
    // @formatter:on
  }

and

  @Override
  public void configure() throws Exception {
    // @formatter:off
      from("direct:unzip-file")
        .log(TRACE, "Try to unzip file '${headers.CamelFileName}'.")
        .bean(method(unzipFile))
      .end();
    // @formatter:on
  }

And with this setup I can write my tests like:

  @Test
  public void testRoute_validExtractedFile() throws Exception {
    final File source = ZIP_FILE_RESOURCE.getFile();
    producer.sendBodyAndHeaders(URI, InOut, source, headers());

    final String actual = getFileContent(unzippedPath, FILENAME);
    final String expected = "This is my little test file for Libri import";
    assertThat(actual).isEqualTo(expected);
  }

  @Test
  public void testRoute_databaseEntryExists() throws Exception {
    final File source = ZIP_FILE_RESOURCE.getFile();
    producer.sendBodyAndHeaders(URI, InOut, source, headers());

    final List<ImportedFilename> actual = importedFilenameRepository.findAll();
    final ImportedFilename expected = importedFilename(ZIPPED_FILENAME);
    assertThat(actual).usingElementComparatorIgnoringFields("id", "timestamp")
    .containsExactly(expected);
  }

  private String getFileContent(final String path, final String filename) throws IOException {
    final String targetFile = path + "/" + filename;
    final byte[] encodedFileContent = Files.readAllBytes(Paths.get(targetFile));
    return new String(encodedFileContent, Charset.defaultCharset());
  }

  private Map<String, Object> headers() {
    final Map<String, Object> headers = Maps.newHashMap();
    headers.put("CamelFileName", ZIPPED_FILENAME);
    return headers;
  }

I can start the camel route with the ProducerTemplate (producer) and send a message to a direct endpoint (instead the ftp endpoint).