如何在Elasticsearch中索引Twitter数据？

Question

I have fetched some data from twitter using streaming api. The file has json lines corresponding to the data. I would like to index this data in elasticsearch so that it could be searched easily.

I used following command to for index

cat file.json | jq -c '.[] | {"index": {"_index": "bookmarks", "_type": "bookmark", "_id": .id}}, .' | curl -XPOST localhost:9200/_bulk --data-binary @-

but it results in following error.

jq: error (at <stdin>:127687): Cannot index string with string "id"

Answer 1

Instead of doing all that manually, you can simply use the Logstash Twitter Input.

I wrote a blog post about it.