Getting `TypeError: unhashable type: 'dict'` while doing bulk upload in elasticsearch

Question

I'm indexing data in elasticsearch using the bulk method to minimize the time of indexing data in elasticsearch. the problem is after using the bulk method my old queries failed(means returning 0 hits) even simple query match query returns zero match

elasticsearch version 6.3, language-python, library- Python Elasticsearch Client

Initially, I have indexed data in Elasticsearch using this code.

temp_entities_list = []
for each_row in master_entities:
    entity_data = {}
    entity_data['entity_id'] = each_row.id
    entity_data['createdat'] = each_row.createdat
    entity_data['updatedat'] = each_row.updatedat
    entity_data['individual_business_tag']=each_row.individual_business_tag
    temp_entities_list.append(entity_data)

def indexing(entity_list):
    for entity in entity_list:
        index_name = "demo"
        yield{
            "_index":index_name,
            "_type":"businesses",
            "_source" :{
                "body":entity
            }
        }
try:
    helpers.bulk(es,testing(temp_entities_list))
except Exception as exe:
    indexing_logger.exception("Error:"+str(exe))

This is my old query which works fine when I index a single object at a time.

{
    "query": {
        "match" : {
            "entity_name" : {
                "query" : "Premium Market",
                "operator" : "and"
            }
        }
    }
}

As per the documentation https://elasticsearch-py.readthedocs.io/en/master/helpers.html#example , I tried this code

def indexing(entity_list):
    for entity in entity_list:
        index_name = "demo"
        yield{
            "_index":index_name,
            "_type":"businesses",
            "doc" :{entity
            }
        }

Getting this error:

Traceback (most recent call last):
  File "sql-to-elasticsearch.py", line 90, in <module>
    helpers.bulk(es,indexing(temp_entities_list),chunk_size=500,)
  File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elasticsearch\helpers\__init__.py", line 257, in bulk
    for ok, item in streaming_bulk(client, actions, *args, **kwargs):
  File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elasticsearch\helpers\__init__.py", line 180, in streaming_bulk
    client.transport.serializer):
  File "C:\Users\AppData\Local\Programs\Python\Python37-32\lib\site-packages\elasticsearch\helpers\__init__.py", line 58, in _chunk_actions
    for action, data in actions:
  File "sql-to-elasticsearch.py", line 81, in indexing
    index_name = "demo"
TypeError: unhashable type: 'dict'

Answer 1

I believe this causes error:

"doc" :{entity}

As your entity seems to be a dictionary and you are trying to put it in a set, and in Python only immutable objects can be stored inside set (strings, integers, floats, tuples...) as they are hashable.

Please take note that this notation is used for sets {} .

If you wanted to put it into container I suggest using a list:

"doc" : [entity]

Or if you are just pointing to entity with doc use:

 "doc" : entity

Hope this helps.