消耗来自kafka主题的json值并将其写入并使用JQ将其格式化为csv文件

Question

I am trying to write into a csv file keys and values that are in a kafka topic. I have been able to select the keys and values that I want, but I am not able to get them separated by rows (three values per row values in rows separated by commas).

This is an example of two json records that I consumed from my kafka topic without doing any filtering. The command that I used is:

./kafka-run-class.sh kafka.tools.ConsoleConsumer --bootstrap-server kafka1.example.net:9092 --topic prod.example.v1 --max-messages 2 | jq -r '. '

{ "count": "0", "source": 3, "lastModified": "2018-03-09T21:03:54.039Z", "isBusiness": false, "countryCode": " MX", "phone": "52/4446789864" } { "count": "0", "source": 3, "lastModified": "2018-03-09T21:03:54.039Z", "isBusiness": false, "countryCode": " GB", "phone": "44/0187567846" }

I tried using this command, but each value is being put into its own row:

./kafka-run-class.sh kafka.tools.ConsoleConsumer --bootstrap-server kafka1.example.net:9092 --topic prod.example.v1 --max-messages 3 | jq -r ' .isBusiness, .countryCode, .phone ' > file.csv

Ideal output would be:

false, MX, 52/4446789864
false, GB, 44/0187567846
true, BE, 32/8745687645

Answer 1

jq -r '[.isBusiness, .countryCode, .phone] | @csv'

produces CSV:

false," MX","52/4446789864"
false," GB","44/0187567846"

The filter:

"\(.isBusiness), \(.countryCode), \(.phone)"

produces

false,  MX, 52/4446789864
false,  GB, 44/0187567846

You might want to "trim" the string values, eg using:

def trim: sub("^ +";"") | sub(" +$";"");