spacy和displacy输出不同

Question

My sentence is: She had another chemotherapy protocol history with 5-FU alone before this protocol without any significant side effects.

When I put this in displacy ( https://demos.explosion.ai/displacy/ ), the output contains a reference to 5-FU as a noun phrase.

However, when I annotate the text and search for noun chunks, I am not shown 5-FU as a noun chunk.

nlp = spacy.load('en') ax = nlp(mySentence) for w in ax.noun_chunks: print(w)

edit Additionally, when I search for the tags with the below code I am shown 5-FU as a NN. If Spacy's annotation understands this singleton word as a noun surrounded by prepositions, why shouldn't the word be picked up as a noun phrase? end-edit

My spacy version:

What am I doing wrong? Is there a version difference between displaCy and the version I am using? Is there a spaCy help team to address this issue?

Thanks much!

Answer 1

Displacy does some pre-processing while showing the parse tree. Here is a link to the parsing service ( built on spacy ) used by displacy : https://github.com/explosion/spacy-services/blob/master/displacy/displacy_service/parse.py#L25

if collapse_phrases:
    for np in list(self.doc.noun_chunks):
        np.merge(np.root.tag_, np.root.lemma_, np.root.ent_type_)

Spacy merges the noun chunks in the sentence instead of treating them as seperate tokens, this is why your output is different.

The other difference would be the models that you use. You might be using the smallest en_core_web_sm whereas Spacy might be using the bigger en_core_web_md ( though it is not mentioned officially anywhere )

Answer 2

I am trying to solve the same problem. DisplayCy and SpaCy outputs are different (both POS tags and relationships between words).

It doesn't look like the pre-processing merging is to blame as you can disable that in DisplayCy - Settings > Collapse Phrases - for me the output still doesn't match.

It's possible that you need to use en_core_web_md model (not en_core_web_sm):

python -m spacy download en_core_web_md

However I haven't tested that yet.

Answer 3

Because they upgrades to V2.0 I faced the similar problem.Then i moved to V2.0 To install a model, you'll have to download it with its full name, using the --direct flag:

python -m spacy download en_core_web_sm-2.0.0-alpha --direct   # English
python -m spacy download xx_ent_wiki_sm-2.0.0-alpha --direct   # Multi-language NER

You can load a model by calling spaCy's loader. eg nlp = spacy.load('en_core_web_sm') , or import it as a module ( import en_core_web_sm ) and call its load() method, .eg nlp = en_core_web_sm.load() .

Follow the Documentation at https://github.com/explosion/spaCy/releases/tag/v2.0.0-alpha