Azure DataFactory - Filter activity condition expression

Question

I am looking at a folder and filtering files based on condition to pass onto next activity. There are many files in the folder and have a filter activity: Files I need to filter are in below format: Extract_2020Mar20_ENV1_200.csv Extract_2020Mar20_ENV2_200.csv

Condition I need to filter above files are - Files starting with name 'Extract' AND contain 'ENV' How do I reflect that in my condition part of my filter activity: I can do one condition but not two … @and(equals(item().type,'File'),startswith(item().name,'Extract'))

NB - I am only picking files from the folder. Pls assist...

Answer 1

You can use a wildcard characters (? for just one, and * for many) in your dataset (no need for expressions in the pipeline! helps to have a cleaner config) to reflect this, so in place of the file name you can type:

Extract_*_ENV?_*.csv

And that dataset will only pick those files that match the condition in that folder.

This is just an example to get you the idea, if the number after ENV can get to 2 digits:

Extract_*_ENV*_*.csv

Doc here: https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-store#copy-activity-properties

Hope this helped!