无效数组枚举上标量变量的索引

Question

I'm trying to run the following sentence compression locally: https://github.com/zhaohengyang/Generate-Parallel-Data-for-Sentence-Compression

So I copied the files and installed all dependencies with conda. I have made minor modifications such as reading the data from url instead of local disk and placed his parallel_data_gen.py bundled in my single py file.

However when I run it I get:

Spacy library couldn't parse sentence into a tree. Please ignore this sentence pair

----------------59-------------------
reducing sentence: This year the Venezuelan government plans to continue its pace of land expropriations in order to move towards what it terms ``agrarian socialism''.
reducing headline: Venezuelan government to continue pace of land expropriations for ``agrarian socialism''
Traceback (most recent call last):
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 701, in <module>
    reduce_sentence(sample)
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 641, in reduce_sentence
    sentence_info = parse_info(sentence)
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 616, in parse_info
    heads = [index + item[0] for index, item in enumerate(doc.to_array([HEAD]))]
IndexError: invalid index to scalar variable.

I'm not sure how to fix that as i'm a rather newbie python user.

Here is the full code i'm running to reproduce the problem: https://gist.github.com/avidanyum/3edfbc96ea22807445ab5307830d41db

The internal snippet that fails:

def parse_info(sentence):
    doc = nlp(sentence)

    heads = [index + item[0] for index, item in enumerate(doc.to_array([HEAD]))]

and here is now I loaded the nlp :

import spacy
print('if you didnt run: python -m spacy download en')
import spacy.lang.en
nlp = spacy.load('en')

More info about my env:

/home/user/home/user/dev/anaconda3/envs/pymachine/bin/python --version
Python 2.7.15 :: Anaconda, Inc.

Answer 1

Quick note that I am running spaCy 2.0 on python 3.6, but just running a quick test on a sample sentence:

nlp = spacy.load('en_core_web_lg')

doc = nlp("Here is a test sentence for me to use.")

I get a few errors running your code, both of which are in the line you specify:

heads = [(index, item) for index, item in enumerate(doc.to_array([HEAD]))]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'HEAD' is not defined

This is because the to_array call takes a list of string objects. Fixing it to this:

# Note that HEAD is now a string, rather than a variable
heads = [(index, item) for index, item in enumerate(doc.to_array(['HEAD']))]
heads
[(0, 3), (1, 1), (2, 1), (3, 0), (4, 18446744073709551615), (5, 1), (6, 18446744073709551614), (7, 18446744073709551612)]

solves the problem. You'll also notice that the item returned by enumerate is an int or scalar type, therefore it has no index attribute. Get rid of your index[0] and that should fix your problem.

Your method with no errors:

def parse_info(sentence):
    doc = nlp(sentence)

    heads = [index + item for index, item in enumerate(doc.to_array(['HEAD']))]