斯坦福NER CRF分类器的训练数据格式应该是什么？

Question

I am trying to train my own Address classifier model using Stanford CRF-NER but the performance is very low. I am confused about the format of the training data I have trained with. The training data is typically the list of districts, cities, provinces and their respective labels. But the model is not tagging the respective address tags to its tokens.

The format of the training data is as below:

• BARAT PROVINCE
• MALUKU PROVINCE
• MALUKU PROVINCE
• KABUPATEN REGENCY
• SIMEULUE REGENCY
• KABUPATEN REGENCY
• ACEH REGENCY

This is the just a sample of training data in csv format, There are 3 labels PROVINCE, REGENCY and DISTRICT

Here is the output of tagged tokens:

You can all tokens has been tagged as DISTRICT though I have REGENCY, DISTRICT AND PROVINCE as labelled data.

I wanted to know if my format of training data is correct is only works on contextual data at sentence level Since I saw Stanford NER working well on sentence level.

Answer 1

Since you're trying to make an address classifier, I would suggest you train your model with actual (tagged) addresses and not a dictionary comprising a list of Regency, District, Province. CRF will then be able to take contextual information into account when trying to tag it depending on the features you've configured.

You use CoNLL style data to train a CRF.

-DOCSTART-    O 

5461    O
North   O
Ave     O
Miami   DISTRICT
Florida PROVINCE
88754   O

8888    O
South   O
Drive   O
Miami   DISTRICT
Florida PROVINCE
99965   O

More proper use of the list of Districts, Provinces will be as a Gazette.