如何从下一个Transformer模块中的Spring云数据流聚合器应用程序输出中提取Collection有效负载记录

Question

I am using SCDF [ Spring Cloud Data Flow ] and in my stream I am aggregating payloads and in next transformer module I would like to get size of the received payload [ Which is a collection object output by aggregator application as per aggregation feature ]

Example Stream :

stream create file_agr_trc_log  --definition "file --directory=/Users/keerthikanth/Documents/GroovySampleCodes/test  --filename-pattern='UNITINFO_DWH.txt' --mode=lines --outputType=plain/text   | aggregator  --release='size() == 1000'  --aggregation=\"#this.![new String(payload)]\" --group-timeout=4 --message-store-type=simple  --correlation='T(Thread).currentThread().id'   | log " --deploy

File /Users/keerthikanth/Documents/GroovySampleCodes/test/UNITINFO_DWH.txt contains

A

B

C

D

E

F

G

H

I

After execution of stream , Aggregator app which used aggregation [ --aggregation=\\"#this.![new String(payload)]\\" ] produced output a collection like below in log application

[ A, B,C,D,E,F,G,H,I ]

Actual Problem :

When I applied new custom transformer application after aggregator app inorder to find size and first record of collection , I am getting wrong records when i tried to fetch first record of collection i,e Getting [ as first element "," as second element "A" as third element .

So indirectly when collection payload output from aggregator module , it is getting converted to string.

I tried to apply spring.cloud.stream.bindings.output.content-type='application/x-java-object;type=java.util.List' in aggregator and spring.cloud.stream.bindings.input.content-type='application/x-java-object;type=java.util.List' in custom transformer but still facing same issue like it is considering all the elements as part of collection including [ , ] charectors

Another Stream with content-type

stream create file_agr_trc_log  --definition "file --directory=/Users/keerthikanth/Documents/GroovySampleCodes/test  --filename-pattern='UNITINFO_DWH.txt' --mode=lines --outputType=plain/text   | aggregator --aggregation=\"#this.![new String(payload)]\"  --release='size() == 1000' --group-timeout=4 --message-store-type=simple  --correlation='T(Thread).currentThread().id' --spring.cloud.stream.bindings.output.content-type='application/x-java-object;type=java.util.List'  |  transCount --spring.cloud.stream.bindings.input.content-type='application/x-java-object;type=java.util.List'   | log " --deploy

Customer Transformer :

@EnableBinding(Processor.class)
public class processorApp {
  @ServiceActivator(inputChannel = Processor.INPUT, outputChannel = Processor.OUTPUT)
  public String transformer(Collection payloadList)  {
     System.out.println("Size of Payload" + payloadList.size());
    System.out.println("Payload "+ payloadList.iterator().next().toString() );

     return payload;
}

Can anyone suggest me on how to fetch size of the received collection payload from aggregator app in custom transformer. even after applying content-type it is not considering. Even with default aggregation also same situation as providing list output but receiving transformer considering every character as a element of the list

My output should be like First element - total size of elements in that collection

A - 8 Size

Answer 1

Why would you expect it to be a collection if your aggregation app actually produces the String based on instruction provided by you: --aggregation=\\"#this.![new String(payload)]\\" .

Basically you don't need to provide any aggregation option for your case since it will default to java.util.List . Please see the aggregator's README for more details