elasticsearch批量方法使用字母数字ID失败

Question

I can import data from pandas dataframe to elasticsearch using the following code. I simply need to add an id column with auto-generated serial number. But can I use messageid column as id?

# message id looks like nucb-9a7ff0885b95efae
df["id"] = [x for x in range(len(df["messageid"])) ]

# the above statement works but the following does not
#df["id"] = df["messageid"]

tmp = df.to_json(orient = "records")
df_json= json.loads(tmp)
import elasticsearch
es = elasticsearch.Elasticsearch('https://some_site.com')

for id in df_json:
    es.index(index='fromdf', doc_type='mydf', body=id)

id in elasticsearch need not be numeric. But while using python, I get an error

RequestError: TransportError(400, u'MapperParsingException[failed to parse [id]]; nested: NumberFormatException[For input string: "nucb-a006fd8dd60ac7a6"]; ')

How do I make sure that I can use bulk method with non-numeric ids?

In other words, the code should work with

df["id"] = df["messageid"]

Answer 1

index method signature:

def index(self, index, doc_type, body, id=None, params=None):
...
    :arg index: The name of the index
    :arg doc_type: The type of the document
    :arg body: The document
    :arg id: Document ID
...

so your data should go to body and identifier that identifies your data should go to id. If you want to store messages that are identified by mesageid you could do so like:

for row_dict in df_json:
    es.index(index='fromdf', doc_type='mydf', body=row_dict, id=row_dict['messageid'])

You could also greatly simplify your code by using already defined functions like pandas.DataFrame.to_dict so that you don't have to convert to json and load json just to get dictionary.