最新文件和文件动态命名

Question

I'm working on a talend job that makes an HTTP GET request to get multiple PDF documents. The request returns a JSON file consisting of: documentDate and documentLink. I want to get the latest documentLink(s) from the documentDate and upload the document(s) with a FileFetch component with the filename being "Document_1" if only one document exist of the latest date. If 2 documents have the latest date then the FileFetch component should upload 2 documents with one filename being "Document_1" and the other "Document_2" I'm unsure on how to loop over the JSON file to get the latest date and naming the document(s) correctly.

What I have done so far:

 tHTTPRequest_1 --> tExtractJSONFields_1 --> tXMLMap_1 --> tFileFetch_1

This works uploading one file, but there is no check made for the latest documentDate or naming of the filenames in the tFileFetch_1 component.

The returned JSON looks like this:

{
"documents": [
    {
        "documentDate ": 200119,
        "documentLink": "someLink1",
    },
    {
        "documentDate ": 200119,
        "documentLink": "someLink2",
    },
    {
        "documentDate ": 150119,
        "documentLink": "someLink3",
    }
   ]
}

Do you guys have any idea on how to solve this problem?

Answer 1

I believe you are looking for something similar to the following:

The first part of the Job consists of :

tFileInputJson (or in your case the tHttpRequest1) -> tSetGlobalVar -> tExtractJsonFields ->tJavaRow

tHttpRequest1 will grab the JSON response (can be sorted or not)

tsetGlobalVar will save the JSON in a global variable to be used in the second part.

tExtractJson will extract the DocumentDate from every JSON array entry.

tJavaRow will contain the Java Logic of comparing the dates from different documents and setting the maximum one in a global variable:

String maxDate = (String) globalMap.get("MaxDate");

if (maxDate != null && !maxDate.trim().isEmpty() )
    //Some Logic here to take the max i.e. Convert it to proper date format and compare them.
    globalMap.put("MaxDate", *comparedMaxDate*);
else
    globalMap.put("MaxDate", row4.DOCDATE);

Once this is completed, in our Global Variable we will have the original JSON response and maximum/recent Date.

The second Part which will run once the first part(SubJob) is completed will consist of:

tJava -> tExtractJsonFields -> tMap -> tFileFetch

tJava will simply grab the JSON message and MaxDate from the global variables set in part one.

tExtractJsonFields will extract the documentLink and documentDate for each array entry and pass them along the max date to the tMap component.

tMap component will simply perform a comparison between the documentDate and max date and check if equal to pass the documentLink to tFileFetch, else it will ignore it.

This allows you to only send the documentLinks that have the latest date.

Hope this helps and it is clear enough.