使用Python在NLP中命名实体识别

Question

I have lots of CVs text documents. In that, there is different formats of dates are available eg Birthdate - 12-12-1995, Experience-year - 2000 PRESENT or 1995-2005 or 5 years of experience or 1995/2005, Date-of-Joining - 5th March, 2015 etc. From these data I want to extract only years of experience. How can I do this in Python using NLP? Please answer.

I have tried with following :

#This gives me all the dates from documents
import datefinder
data = open("/home/system/Desktop/samplecv/5c22fcad79fcc1.33753024.txt")
str1 = ''.join(str(e) for e in data)
matches = datefinder.find_dates(str1)
for match in matches:
    print(match)

Answer 1

If you already have extracted the dates then it seems like what you're missing is the "type of date" each is. If datefinder isn't able to keep track of the positional structure of the dates within the corpus then date extraction using it won't be too useful.

However, this isn't just a entity recognition problem. You'll have to pair a NER with a POS tagger (and maybe even a Syntatic Dependency Parser) Spacy is a good one.

You should first run a POS tagger on your corpus and see whether it picks up phrases like "Experience" or "Work History". If not, you should add your own labels to it so that it will specifically tag those words as you desire.

Then you can run a NER to pick up Dates. Keep in mind that the NER at best will tag all your dates as DATE entities and will not be able to find the distinction between what type of dates these are.

You'll have to link the respective date to a preceding or following Part of Speech using some language grammar or a regular expression.

For instance you can associate all Dates that follow the word Experience to the Experience POS Tag.

Alternatively you can try NLTK (which is an alternative to Spacy but you'll need to run the same pipeline with it too). Read here for more.