使用Spacy NLP使用多行命名实体识别

Question

In Spacy NLP, I am not able to get exact output for named entity. My string value is on multiple lines. Please check below code:

from spacy import displacy
from collections import Counter
import en_core_web_sm
nlp = en_core_web_sm.load()
m = (u"""Release the container 6th August

USG11223
USG12224
USG21113""")

doc = nlp(m)
print([(X.text, X.label_) for X in doc.ents])

OUTPUT: [('6th August', 'DATE')]

But i want output like

['USG11223', 'USG12224', 'USG21113',6th August]

Answer 1

One thing that most people do not realize about Named Entity Recognition in libraries like Spacy, AllenNLP, etc, is that it is usually a Machine Learning model trained on a general corpus for general entities.

Your data is from a specific context , where strings like "USG11223" have some special meaning. However, in general context, your string is no more than a random combination of letters and numbers and might even be discarded by the model preprocessing.

If you want the NER to recognize your tags as entities, you can train your own model to be able to recognize these tokens as entities, but you would have to provide several examples. You can learn more about how to do it here: https://spacy.io/usage/training/