I'm currently working on a NER model. I have a bunch of data stored in CoNLL format that need to be converted to Spacy format. In CoNLL, each word of a sentence has a tag next to it. In Spacy, the tag is only shown to the words that have an actually tag. How can I convert from this format below (CoNLL)
From O
2001 B-DateTime
to I-DateTime
2004 I-DateTime
, O
I O
was O
a O
stagehand O
for O
Hartford B-Company
Stage I-Company
Company O
. O
to this format below (Spacy)
TRAIN_DATA = [('what is the price of polo?', {'entities': [(21, 25, 'PrdName')]}),
('what is the price of ball?', {'entities': [(21, 25, 'PrdName')]}),
('what is the price of jegging?', {'entities': [(21, 28, 'PrdName')]}),
('what is the price of t-shirt?', {'entities': [(21, 28, 'PrdName')]}),
('what is the price of jeans?', {'entities': [(21, 26, 'PrdName')]}),
('what is the price of bat?', {'entities': [(21, 24, 'PrdName')]}),
('what is the price of shirt?', {'entities': [(21, 26, 'PrdName')]}),
('what is the price of bag?', {'entities': [(21, 24, 'PrdName')]}),
('what is the price of cup?', {'entities': [(21, 24, 'PrdName')]}),
('what is the price of jug?', {'entities': [(21, 24, 'PrdName')]}),
('what is the price of plate?', {'entities': [(21, 26, 'PrdName')]}),
('what is the price of glass?', {'entities': [(21, 26, 'PrdName')]}),
('what is the price of watch?', {'entities': [(21, 26, 'PrdName')]})]
Just use spacy convert .
spacy convert input.conll -c conll -o ./output/
Note that by default this produces a binary .spacy
file. The JSON format is deprecated in v3 and isn't really helpful for much.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.