Azure 数据湖 - 条件

Question

I would like to use if-else statement to decide at what location I have to export data. My case is:

1. I extract several files from the azure blob storage ( it's possible that there are no files!! ).
2. I calculate count of records in file set.
3. If count of records is > 20 then I export files into specific report location
4. If in file set are no records, I have to output dummy empty file into different location, because I don't want replace existing report by empty report.

The solution may be IF..ELSE confition. The problem is that if I calculate count of records I got rowset variable and I cannot compare it with scalar variable.

@RECORDS =
SELECT COUNT(id) AS IdsCount
FROM @final; 


IF @RECORDS <= 20 THEN //generate dummy empty file 
    OUTPUT @final_result
    TO @EMPTY_OUTPUT_FILE
    USING Outputters.Text(delimiter : '\t', quoting : true, encoding : Encoding.UTF8, outputHeader : true, dateTimeFormat : "s", nullEscape : "NULL");
ELSE 
    OUTPUT @final_result
    TO @OUTPUT_FILE
    USING Outputters.Text(delimiter : '\t', quoting : true, encoding : Encoding.UTF8, outputHeader : true, dateTimeFormat : "s", nullEscape : "NULL");
END;

Answer 1

U-SQL's IF statement is currently only used during compilation time. So you can do something like

IF FILE.EXIST() THEN

But if you want to output different files depending on the number of records you would have to write it at the SDK/CLI level:

The first job writes a temp file output (and maybe a status file that contains number of rows). Then you check (for example in Powershell) whether the file is empty (or whatever criteria you want to use) and if not, copy the result over otherwise create the empty output file.