Scrapy installation error pip and easy_install

Question

I am trying to install Scrapy on Windows and have followed steps in the Scrapy installation guide to install all the dependencies. However, I got the following error message when I tried to use easy_install :

Download error on https://pypi.python.org/simple/Scrapy/: **[Errno 10061] 
No connection could be made because the target machine actively refused 
it -- Some packages may not be found!**
Couldn't find index page for 'Scrapy' (maybe misspelled?)

Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
Download error on https://pypi.python.org/simple/: **[Errno 10061] 
No connection could be made because the target machine actively refused 
it -- Some packages may not be found!**
No local packages or download links found for Scrapy
error: Could not find suitable distribution for 
Requirement.parse('Scrapy')

I also tried to use pip but it doesn't work either:

Downloading/unpacking Scrapy
 **Cannot fetch index base URL https://pypi.python.org/simple/
  Could not find any downloads that satisfy the requirement Scrapy**
Cleaning up...
No distributions at all found for Scrapy

I went to pip.log and it says

Could not fetch URL https://pypi.python.org/simple/Scrapy/: **connection
error: HTTPSConnectionPool(host='pypi.python.org', port=443): Max 
retries exceeded with url: /simple/Scrapy/ (Caused by 
<class 'socket.error'>: [Errno 10061] No connection could be made 
because the target machine actively refused it)**
  Will skip URL https://pypi.python.org/simple/Scrapy/ when looking 
for download links for Scrapy

I can access https://pypi.python.org/simple/Scrapy/ directly from internet explorer but not sure why pip or easy_install can't access the link.

I'm using Anaconda python so easy_install and pip are already part of the package. I also had the following error at the last stage of installing pywin32 :

close failed in file object destructor:

sys.excepthook is missing

lost sys.stderr

Is this the reason easy_install and pip failed? Could anyone help with these please?

Answer 1

Step By Step way to install scrapy on Windows 7

1. Install Python 2.7 from Python Download link (Be sure to install Python 2.7 only because currently scrapy is not available for Python3 in Windows)
2. During pyhton install there is checkbox available to add python path to system variable click that option. Otherwise you can add path variable manually. You need to adjust PATH environment variable to include paths to the Python executable and additional scripts. The following paths need to be added to PATH C:\\Python27\\;C:\\Python27\\Scripts\\;

If you have any other problem in adding path variable please refer to this link
3. To update the PATH open a Command prompt in administration mode and run: :\\python27\\python.exe c:\\python27\\tools\\scripts\\win_add2path.py .Close the command prompt window and reopen it so changes take effect, run the following command, to check ever thing added to path variable.
python -–version which will give output as Python 2.7.12 (your version might be different than mine)
pip --version which will give output as pip 9.0.1 (your version might be different than mine)
4. You need to install visual basic C++ Python complier. You can download that from Download link
5. Then you install to install libxml which python library used by scrapy. You download it by writing a command pip install libxml into command prompt. but if you face some problem in pip installation you can download it from http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml download libxml package according to your system architecture . Open command prompt into that download directory and pip install NAME_OF_PACKAGE.whl
6. Install pywin32 from Download link . Be sure you download the architecture (win32 or amd64) that matches your system
7. Then open command prompt and run this command pip install scrapy
I hope this will help in successful installing scrapy 8. For Reference use can your these links Scrapy official Page and Blog on how to install scrapy on windows

Answer 2

How to install Scrapy 1.4 on Python 3.6 on Windows 8.1 Pro x64

pip install virtualenv
pip install virtualenvwrapper
pip install virtualenvwrapper-win
mkvirtualenv my_scrapy_project

I advice to use virtualenv. In my example I am using name my_scrapy_project for my virtual environment. If you want to go out of virtualenv, simply type deactivate , if you want to go back into, simply type workon my_scrapy_project .

• Go to: http://landinghub.visualstudio.com/visual-cpp-build-tools
• Click on button: Download Visual C++ Build Tools 2015

• Install these tools.

• Go to: https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

• Find and download: lxml-4.1.1-cp36-cp36m-win32.whl
• Move this file to your active directory in commandline and install it:

pip install lxml-4.1.1-cp36-cp36m-win32.whl

pip install scrapy

And that is all, it should work.