sp_execute_external_script 找不到 setuptools 安装的模块

Question

I am actively developing a Python module that I would like to deploy in SQL Server 2017 installed locally, so I deploy the module in c:\\Program Files\\Microsoft SQL Server\\<Instance Name>\\PYTHON_SERVICES\\Lib\\site-packages using setuptools like so:

"c:\Program Files\Microsoft SQL Server\<Instance_Name>\PYTHON_SERVICES\python" setup.py develop

This produces an .egg-info directory in my project root, and a .egg-link file in the site-packages directory mentioned above. The .egg-link file correctly points to the .egg-info directory in my project root, so it appears setuptools is working correctly.

Here's my setup.py for reference:

from setuptools import setup

setup(
    setup_requires=['pbr'],
    pbr=True,
)

And here's the corresponding setup.cfg file:

[metadata]
name = <module_name>
description = <Module Description>
description-file = README.md
description-content-type = text/markdown

[files]
package_root = py/src

Since I am just trying to make the plumbing work, I have a single python script called uploader.py in <project_root>/py/src :

#uploader.py
class Upload:
    pass

With this deployment in place, I am hoping to simply import the module I just published through .egg-link into a sp_execute_external_script call like so:

execute sp_execute_external_script @language= N'Python', @script= N'from <module_name>.uploader import Upload';

However, executing this stored procedure from SSMS produces the following error message:

Msg 39004, Level 16, State 20, Line 10
A 'Python' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 2, Line 10
An external script error occurred: 

Error in execution.  Check the output for more information.
Traceback (most recent call last):
  File "<string>", line 5, in <module>
  File "C:\SQL-MSSQLSERVER-ExtensibilityData-PY\MSSQLSERVER01\C08BB9A7-66B5-4B5E-AAFC-B0248EE64199\sqlindb.py", line 27, in transform
    from <module_name>.uploader import Upload
ImportError: No module named '<module_name>'

SqlSatelliteCall error: Error in execution.  Check the output for more information.
STDOUT message(s) from external script: 
SqlSatelliteCall function failed. Please see the console output for more information.
Traceback (most recent call last):
  File "C:\Program Files\Microsoft SQL Server\<Instance_Name>\PYTHON_SERVICES\lib\site-packages\revoscalepy\computecontext\RxInSqlServer.py", line 587, in rx_sql_satellite_call
    rx_native_call("SqlSatelliteCall", params)
  File "C:\Program Files\Microsoft SQL Server\<Instance_Name>\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 358, in rx_native_call
    ret = px_call(functionname, params)
RuntimeError: revoscalepy function failed.

I have obviously redacted module_name and Instance_Name from the error message.

I tried using install command instead of develop just to make sure the .egg-link file is not a problem. install installs the .egg-info file in site-packages but I get the same error.

I also tried removing pbr from the mix, but got the same error.

Lastly, I tried adding my <project_root> to sys.path as suggested by How can I use an external python module with SQL 2017 sp_execute_external_script? , but that didn't help either.

So at this point, I don't have a clue what I might be doing wrong.

The python version is 3.5.2 and I don't think an __init__.py is needed in the project for it to qualify as a module. Inserting a blank __init__.py in py/src doesn't help either.

My pip version is 19.3.1 and setuptools version is 44.0.0 and pbr version is 5.4.4 and I have confirmed all modules are installed in the site-packages directory mentioned above.

Answer 1

Based on my extensive experimentation, it appears that sp_execute_external_script doesn't follow symlinks (ie through the .egg-link file). Therefore, development mode installations will not work, whether you use setuptools , pip , pbr or anything else.

I even tried symlinking <package_name> folder as an OS symlink. Since I am on Windows, I used mklink /D command on Command Prompt to symlink /py/src/<package_name> inside site-packages . While the command goes through correctly, and I can see the symlinked folder in File Explorer, sp_execute_external_script fails to detect the package. Which tells me that there is probably something in sp_execute_external_script code that avoids traversing symbolic links.

I wonder if there is a way to make it traverse symbolic links.

The only workable solution is to develop a package's code under its own directory, so, in my case /py/src/<package_name> . Then, before running exec sp_execute_external_script @language=N'python', @script=N'...' copy the <package_name> folder to the site-packages directory.

This is, sort of, equivalent to setup.py install , but bypasses the creation of intermediate files and directories. So I am going to stick with this simple--though odious--approach.

I am hoping somebody more knowledgeable would offer a better way to solve this problem.