将 JSON 数组插入 Elasticsearch 使用 Python 批量 API

Question

I have a JSON array of 100 records, I want to insert it into elasticsearch,I tried it using the below code but it gives me JSON Error exception. I am not sure where I am doing it wrong. The code which I am using to insert record is

from esService.esClient import ESCient
from elasticsearch.helpers import bulk

#data is the json array object which has around 100 records in it .

#I want to insert it into elasticsearch in such a way 

#that each record is a entry into ES (I can remove the ID if required  ,I don't need the ID ) 

    def insert_bulk_record(self,index,data,id):
        docs = []
        doc = {
            "_index": index,
            "_id": id,
            "_source": data
        }
        docs.append(doc)

        bulk(self.esconnect, docs)

When I do bulk insert using the above code I get the below exception,

elasticsearch.helpers.errors.BulkIndexError: ('1 document(s) failed to index.', [{'index': {'_index': 'data_record', '_type': '_doc', '_id': '7908745568_0.csv', 'status': 400, 'error': {'type': 'mapper_parsing_exception', 'reason': 'failed to parse', 'caused_by': {'type': 'not_x_content_exception', 'reason': 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes'}}

Answer 1

This error means that you are not passing the correct format to Elastic for indexing the data. Bulk helpers accept single JSON objects and not JSON arrays.

For example, you might try something like this:

for d in data:
    docs = []
    doc = {
        "_index": index,
        "_source": d
    }
    docs.append(doc)

    bulk(self.esconnect, docs)

Have a look at this blog as well.