Apache骆驼-SQL到CSV

Question

I have tried to convert sql result to csv file this is the route (bluprint.xml):

<route id="extractfromafbl">
  <from uri="timer:foo?period=10000"/>
  <to uri="sql:select * from performances?dataSource=afbl_source"/>
  <to uri="bean:ngtrend.afbl2afc.transformer?method=tocsv(Exchange)"/>
  <log message="${body}"/>
</route>

This is transformer class to create csv format:

    public class transformer {
            public void tocsv(Exchange exchange)
            {
                StringBuilder csv = new StringBuilder();
                List received = exchange.getIn().getBody(List.class);
                for (int i = 0; i < received.size(); i++) 
                 {
                   Map<String,Object> row = (Map<String,Object>) received.get(i);
                   csv.append(row.get("id"));
                   csv.append(",").append(row.get("lastname"));
                   csv.append(",").append(row.get("firstname"));
                   csv.append("\r");
                 }
            }
        }

in this case I should know the name of fields (like id, lastname ...) and I want to use an index instead because the table or fields can change.

How can I use index instead of name fields ?

Answer 1

I know you can make the code above work with some tweaks, but I thought I would make a suggestion that could make this whole problem way easier for you. The Camel SQL component lets you automatically map your database response back to an Object. Syntax is like this:

from("sql:select * from performances?dataSource=afbl_source?outputType=SelectList&outputClass=com.myapp.MyDatabaseModel")

Once you have this setup you will get an object in your camel body of a

List<MyDatabaseModel>

At this point you can leverage a camel component called Bindy. It will let you have an Object represent a CSV file. The way you would bind this is with a few annotations on your database model so it can also represent your csv output.

@CsvRecord(isOrdered = true)
public Class MyDatabaseModel{

   @DataField(pos = 1, position = 11)
   private int orderNr;

   @DataField(pos = 2, position = 10)
   private String clientNr;

   ...
}

The call to Bindy would be setup something like this:

// Camel 2.15 or older (configure by package name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat("com.myapp");
// Camel 2.16 onwards (configure by class name)
BindyCsvDataFormat bindy = new BindyCsvDataFormat(com.myapp.MyDatabaseModel.class);

.marshal(bindy)

Bringing it all together you can break your project down into 2 lines of camel and 1 java object for this task. "Full camel below"

from("sql:select * from performances?dataSource=afbl_source?outputType=SelectList&outputClass=com.myapp.MyDatabaseModel")
    .marshal(bindy);

I think I should also mention if you don't want to worry too much about configuration of your csv order you can actually skip the whole annotation process and simply do:

marshal().csv();

Answer 2

Find a small snippet to show that it will not work as you whish.

List<Map<String, Object>> received = new ArrayList<>();
Map<String, Object> rowOne = new HashMap<>();
rowOne.put("id", 42);
rowOne.put("lastname", "Doe");
rowOne.put("firstname", "John");
Map<String, Object> rowTwo = new HashMap<>();
rowTwo.put("id", 23);
rowTwo.put("lastname", "Doe");
rowTwo.put("firstname", "Jane");
received.add(rowOne);
received.add(rowTwo);
StringBuilder csv = new StringBuilder();
for (Map<String, Object> i : received) {
    StringJoiner line = new StringJoiner(",");
    for (Map.Entry<String, Object> j : i.entrySet()) {
        line.add(j.getValue().toString());
    }
    csv.append(line.toString()).append("\n");
}
System.out.println(csv);

output

John,42,Doe
Jane,23,Doe

The order of the values depends on the name of the column and the type of the Map .

If you use a TreeMap you might force the names of the columns returned by the SQL statement to be in the order you want to output them (I would not suggest to do so)

See this small snippet to show the idea

List<Map<String, Object>> received = new ArrayList<>();
Map<String, Object> rowOne = new TreeMap<>();
rowOne.put("1id", 42);
rowOne.put("2lastname", "Doe");
rowOne.put("3firstname", "John");
Map<String, Object> rowTwo = new TreeMap<>();
rowTwo.put("id", 23);
rowTwo.put("lastname", "Doe");
rowTwo.put("firstname", "Jane");
received.add(rowOne);
received.add(rowTwo);
StringBuilder csv = new StringBuilder();
for (Map<String, Object> i : received) {
    StringJoiner line = new StringJoiner(",");
    for (Map.Entry<String, Object> j : i.entrySet()) {
        line.add(j.getValue().toString());
    }
    csv.append(line.toString()).append("\n");
}
System.out.println(csv);

output

42,Doe,John
Jane,23,Doe

Answer 3

I'm just getting started with Camel so perhaps there's another way but I'm able to dump from a table to a CSV file without any knowledge of the field names with:

CsvDataFormat csv = new CsvDataFormat();
csv.setQuoteMode("ALL");

from("sql:select * from MY_TEST_TABLE?dataSource=oracleDataSource&useIterator=false").marshal(csv).to("file:///tmp?fileName=MY_TEST_FILE.csv");

and add the dependency

    <dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-csv</artifactId>
<version>version</version>