Pip install results in this error " cl.exe' failed with exit code 2 "

Question

I've read all of the other questions on this error and frustratingly enough, none give a solution that works.

If I run pip install sentencepiece in the cmd line, it gives me the following output.

 src/sentencepiece/sentencepiece_wrap.cxx(2809): fatal error C1083: Cannot open include file: 'sentencepiece_processor.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
      [end of output]

I'm running python 3.10.1 and pip 22.0.3.

*I have the following Microsoft Visual C++ programs on my windows machine,which I've just done a fresh install of as it was complaining of not having a particular C++ program. MS VC++

I've even added the.exe file to my PATH variables but still I get the same error.

Am I missing a particular Microsoft program on my pc?

Answer 1

I haven't seen this problem in Windows, but for Linux, I would normally reinstall Python after installing the dependencies (such as the MSVC thing). In that case this is especially helpful because I'm often rebuilding (compiling and other related steps) Python/Pip.

Could also just be an error specific to the module and Python version combo you're trying.

---

From a discussion in the comments:

I have the pyenv-win version manager, so I was able to create venvs and test this for you. With Python 3.10.2, it fails; with Python 3.8.10, it's successful. So, yes, reinstalling does seem to be worthy of your time.

Answer 2

With python3.10

On Windows: First, install vcpkg and install sentencepiece:x64-windows-static

copy the header and lib files from vcpkg/installed/x64-windows-static/include and lib to

C:/Program Files/python310/build/root/include and lib

This should work as the setup.py install expects the library in a..\build\root\lib directory.

As the C:\Program Files\python310\lib is in the /LIBPATH, the resulting..\build\root\lib will point to the sentencepiece.lib

If still encounting errors, then set the INCLUDE and LIB environment variables so that the cl.exe, which is called from pip install sentencetransformers finds them.

Answer 3

People having windows+python 3.10 env, here are exact steps to install it via vcpkg.

Other instructions are covered in zweistein's answer.

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install sentencepiece:x64-windows-static