How to produce Kafka message with kcat?

Question

I'm trying to produce Kafka message with kcat tool. I have a message that follows topic schema:

{
  "meta": {
    "correlationId": "244c7ed3-0472-496e-bed9-7071f8ddb921",
    "payload": {
      "id": "3ae843e9-0c96-4ee8-b1f0-fbdfef30e9dd",
      "timestamp": 1658846522
    }
  }
}

store it to the file k-msg-test.json and run kcat :

kcat \
  -b localhost:9092 \
  -r localhost:8081 \
  -t test-topic \
  -T \
  -P \
  /tmp/k-msg-test.json

and I get an error:

% Failed to open  : No such file or directory
/tmp/k-msg-test.json: line 2: meta:: command not found
/tmp/k-msg-test.json: line 3: correlationId:: command not found
/tmp/k-msg-test.json: line 4: payload:: command not found
/tmp/k-msg-test.json: line 5: id:: command not found
/tmp/k-msg-test.json: line 6: timestamp:: command not found
/tmp/k-msg-test.json: line 8: syntax error near unexpected token `}'
/tmp/k-msg-test.json: line 8: `  }'

So the question is – how can I produce a message?

Answer 1

Your file is apparently being interpreted as literal text to your shell rather than read by kcat.

You need to use shell redirection as kcat accepts stdin, rather than file arguments.

You'll also need to remove the new lines and produce as a single line. You can do that with jq

kcat \
  -b localhost:9092 \
  -r localhost:8081 \
  -t test-topic \
  -T \
  -P < $(jq -rc '.' /tmp/k-msg-test.json)