Python Scrapy分段错误

Question

I have a django project where I use nltk. I get segmentation error while trying to

import nltk

I found the exact line where it happens and it's collocations.py:38. I also found out that I get this error only in django unit-tests and django manage.py shell(actually shell_plus, but I don't thing it changes something), but not in custom management commands (which maybe means, that I won't get this error in production)

I use django django 1.4.5, nltk 2.0.4, python 2.7.3.

Thanks!

UPDATE: Update of python to 2.7.4 hasn't effected

UPDATE: Update of numpy to 1.7.1 and scipy to 0.12.0 hasn't effected

UPDATE: I found the statement, which causes the segfault (I suppose, as much as I moved it from module scope to function scope, which caused segfault not to happen)

from scrapy.crawler import CrawlerProcess

It's the class from crawling scrapy framework, which I used in custom module for executing scrapy spider as a python script. It seems like segfault is not NLTK fault, but scrapy. Probably somehow this class rewrote some data of nltk, that's why it's segfaulted.

Answer 1

I had the same issue with CrawlerProcess. Scrapy had failed to install libxml2. Try entering this into the command prompt:

easy_install lxml
pip install scrapy --upgrade

My program executed correctly after this.