Extract fields from object present in multiple nested levels using jq

Question

I have a text file having data records as below:

{"input":{"payload":{"id":"rec1","var2":"imp_val1","var3":"","var4":"000000"},"recordName":"typeABC"}}
{"input":{"recordName":"typeBCD","payload":{"var5":"val_var66","recordType":"typeA","id":"rec2","var2":"imp_val2","var3":"","var4":"000000"}}}
{"recordName":"typeEFG","payload":{"var5":"val_var55","recordType":"typeA","id":"rec3","var2":"imp_val3","var3":"","var4":""}}
{"payload":{"id":"rec4","var2":"imp_val4","var3":"","var4":"000000"},"recordName":"typeABC"}

There is a recordName and payload key which has my values of interest. Some records are wrapped inside another key input. What i want to extract is id and var2 from all these records into a new csv file.

I figured if the data format were uniform, i could do:

cat file | jq -r "[.payload.id, .payload.var2] | @csv" > newFile

OR

cat file | jq -r "[.input.payload.id, .input.payload.var2] | @csv" > newFile

Any pointers?

Answer 1

Since the payload object is nested in multiple levels in each of the object, you can apply Recursive Descent .. and ignore if the key is not present

jq -r '..| .payload? // empty | [ .id, .var2 ] | @csv' file

^{jqplay - Online demo}

Answer 2

BTW i figured following works too:

jq -r 'if .input then .input.payload else .payload end | [.id, .var2] | @csv' file > newFile