Unable to use pydobc to connect to SQL Server using pyodbc in Python 3.6 in AWS Lambda

Question

I have been struggling, and am unable to figure out how to connect to SQL Server using python 3.6 via pydobc and AWS Lambda.

I followed the instructions provided by AWS to create an Amazon Linux AMI on which I was able to install the Microsoft ODBC drivers (v13 and v17), upgrade the unixODBC to a supported version, and get my python code to connect to the AWS RDS SQL Server instance.

However, I have not been able to figure out how to package those changes successfully to deploy this code to AWS Lambda and have it work.

I get one of two errors, depending on how I try to reference the ODBC driver. Using this syntax

cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=servername.account.region.rds.amazonaws.com,port;DATABASE=database;UID=user;PWD=password'

I get the error:

"errorMessage": "('01000', \"[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)\")",

I have tried using other driver aliases (ODBC Driver 17 for SQL Server, ODBC Driver 13 for SQL Server), with the same results.

Using the syntax:

    cnxn = pyodbc.connect('DRIVER=lib/libmsodbcsql-13.so;SERVER=servername.account.region.rds.amazonaws.com,port;DATABASE=database;UID=user;PWD=password')

I get the error:

'IM004', "[IM004] [unixODBC][Driver Manager]Driver's SQLAllocHandle on SQL_HANDLE_HENV failed (0) (SQLDriverConnect)")

I have the following code in my .ZIP deployment file:

1. simple_db.py - My code to create the connection
2. pyodbc.so - Lambda version of pyodbc from https://github.com/Miserlou/lambda-packages/tree/master/lambda_packages/pyodbc )
3. odbcinst.ini - Attempt to use a Linux-version of odbcinst to list the driver(s).
4. lib/libmsodbcsql-13.so - Copied from the Amazon Linux install
5. libodbc.so.2 - Copied from Amazon Linux install as well, attempt to deploy unixODBC version.

I've toyed around with directories, and adding more libodbc*.* files from the /usr/lib64 directory, but nothing has worked so far. As well as bringing over the entire msodbcsql directory (with /etc, /include, /lib64, and /share).

Any help would be greatly appreciated!

Answer 1

Based in this answer: AWS Lambda function to connect to SQL Server with Python

You will need to compile unixODBC, take its shared libraries. Then install the MS driver, takes its shared libs and gather together

# Start a container that mimic the lambda environment
docker run -it --rm --entrypoint bash  -e ODBCINI=/var/task -e ODBCSYSINI=/var/task -v "$PWD":/var/task  lambci/lambda:build-python2.7

# Then, download ODBC source code, compile and take the output
curl ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.5.tar.gz -O
tar xvzf unixODBC-2.3.5.tar.gz
cd unixODBC-2.3.5
./configure  --sysconfdir=/var/task  --disable-gui --disable-drivers --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE --prefix=/home
make install
cd ..
mv /home/* .
mv unixODBC-2.3.5 unixODBC-2.3.5.tar.gz /tmp/

# Install MSsql odbc driver
curl https://packages.microsoft.com/config/rhel/6/prod.repo > /etc/yum.repos.d/mssql-release.repo
ACCEPT_EULA=Y yum -y install msodbcsql
export CFLAGS="-I/var/task/include"
export LDFLAGS="-L/var/task/lib"

# Then you can install pyodbc (or pip install -t . -r requirements.txt)
pip install pyodbc -t .

cp -r /opt/microsoft/msodbcsql .

cat <<EOF > odbcinst.ini
[ODBC Driver 13 for SQL Server]
Description=Microsoft ODBC Driver 13 for SQL Server
Driver=/var/task/msodbcsql/lib64/libmsodbcsql-13.1.so.9.2
UsageCount=1
EOF

cat <<EOF > odbc.ini
[ODBC Driver 13 for SQL Server]
Driver      = ODBC Driver 13 for SQL Server
Description = My ODBC Driver 13 for SQL Server
Trace       = No
EOF

# Test if it works
python -c "import pyodbc; print(pyodbc.drivers());"
python -c 'import pyodbc;conn = pyodbc.connect("DRIVER={ODBC Driver 13 for SQL Server}; SERVER=YOUr_SERVER:ADD;PORT=1443;DATABASE=TestDB;UID=SA;PWD=<YourStrong!Passw0rd>");crsr = conn.cursor();rows = crsr.execute("select @@VERSION").fetchall();print(rows);crsr.close();conn.close()'