通过python访问钻取时出现ODBC错误

Question

I get the below error while I try to use drill through python:

Traceback (most recent call last): File "/home/ubuntu/ChurnPrediction/OBDCtest.py", line 5, in <module> conn = pyodbc.connect("DSN=drill64", autocommit=True) pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/mapr/drillodbc/lib/64/libmaprdrillodbc64.so' : file not found (0) (SQLDriverConnect)") [Finished in 0.3s with exit code 1]

Below is the sample code:

import pyodbc from pandas import * conn = pyodbc.connect("DSN=drill64", autocommit=True) print conn

Below is additional information on running iodbctest:

root@ubuntu-Vostro-1450:/home/ubuntu# iodbctest iODBC Demonstration program This program shows an interactive SQL processor Driver Manager: 03.52.0709.0909

Enter ODBC connect string (? shows list): ?

DSN | Driver

Sample MapR Drill DSN 32 | MapR Drill ODBC Driver 32-bit
Sample MapR Drill DSN 64 | MapR Drill ODBC Driver 64-bit

Enter ODBC connect string (? shows list): Sample MapR Drill DSN 64 1: SQLDriverConnect = [iODBC][Driver Manager]libdrillClient.so: cannot open shared object file: No such file or directory (0) SQLSTATE=00000 2: SQLDriverConnect = [iODBC][Driver Manager]Specified driver could not be loaded (0) SQLSTATE=IM003

Answer 1

I think you probably need to set your LD_LIBRARY_PATH, like this:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mapr/drillodbc/lib/64

Then you should not see any "not found" errors when running this command:

ldd /opt/mapr/drillodbc/lib/64/libmaprdrillodbc64.so

Here are all the steps I used to install Drill on Ubuntu:

First download and install the latest MapR ODBC rpm, like this:

wget http://package.mapr.com/tools/MapR-ODBC/MapR_Drill/MapRDrill_odbc_v1.3.0.1009/maprdrill-1.3.0.1009-1.x86_64.rpm
rpm2cpio maprdrill-1.3.0.1009-1.x86_64.rpm | cpio -idmv
sudo mv opt/mapr/drill /opt/mapr/drillodbc
cd /opt/mapr/drillodbc/Setup
cp mapr.drillodbc.ini ~/.mapr.drillodbc.ini
cp odbc.ini ~/.odbc.ini
cp odbcinst.ini ~/.odbcinst.ini
# Edit the properties in those ini files according to your needs
# Put the following exports also in ~/.bashrc
export ODBCINI=~/.odbc.ini
export MAPRDRILLINI=~/.mapr.drillodbc.ini
export LD_LIBRARY_PATH=/usr/local/lib:/opt/mapr/drillodbc/lib/64:/usr/lib64

Then update those .ini files, accordingly. Here is how I setup my ini files to connect to Drill:

odbc.ini:

[ODBC]
Trace=yes
Tracefile=/tmp/trace.txt

[ODBC Data Sources]
MapR Drill 64-bit=MapR Drill ODBC Driver 64-bit

[drill64]
# This key is not necessary and is only to give a description of the data source.
Description=MapR Drill ODBC Driver (64-bit) DSN

# Driver: The location where the ODBC driver is installed to.
Driver=/opt/mapr/drillodbc/lib/64/libdrillodbc_sb64.so

# The DriverUnicodeEncoding setting is only used for SimbaDM
# When set to 1, SimbaDM runs in UTF-16 mode.
# When set to 2, SimbaDM runs in UTF-8 mode.
#DriverUnicodeEncoding=2

# Values for ConnectionType, AdvancedProperties, Catalog, Schema should be set here.
# If ConnectionType is Direct, include Host and Port. If ConnectionType is ZooKeeper, include ZKQuorum and ZKClusterID
# They can also be specified on the connection string.
# AuthenticationType: No authentication; Basic Authentication
ConnectionType=Direct
HOST=nodea
PORT=31010
ZKQuorum=[Zookeeper Quorum]
ZKClusterID=[Cluster ID]
AuthenticationType=No Authentication
UID=[USERNAME]
PWD=[PASSWORD]
DelegationUID=
AdvancedProperties=CastAnyToVarchar=true;HandshakeTimeout=5;QueryTimeout=180;TimestampTZDisplayTimezone=utc;ExcludedSchemas=sys,INFORMATION_SCHEMA;NumberOfPrefetchBuffers=5;
Catalog=DRILL
Schema=

odbcinst.ini:

[ODBC Drivers]

MapR Drill ODBC Driver 32-bit=Installed
MapR Drill ODBC Driver 64-bit=Installed

[MapR Drill ODBC Driver 32-bit]
Description=MapR Drill ODBC Driver(32-bit)
Driver=/opt/mapr/drillodbc/lib/32/libdrillodbc_sb32.so

[MapR Drill ODBC Driver 64-bit]
Description=MapR Drill ODBC Driver(64-bit)
Driver=/opt/mapr/drillodbc/lib/64/libdrillodbc_sb64.so

mapr.drillodbc.ini

[Driver]

DisableAsync=0
DriverManagerEncoding=UTF-16
ErrorMessagesPath=/opt/mapr/drillodbc/ErrorMessages
LogLevel=0
LogPath=[LogPath]
SwapFilePath=/tmp
ODBCInstLib=/usr/lib/x86_64-linux-gnu/libodbcinst.so.1.0.0

Finally, if you've installed and configured the ODBC driver correctly, then the command python -c 'import pyodbc; print(pyodbc.dataSources()); print(pyodbc.connect("DSN=drill64", autocommit=True))' python -c 'import pyodbc; print(pyodbc.dataSources()); print(pyodbc.connect("DSN=drill64", autocommit=True))' python -c 'import pyodbc; print(pyodbc.dataSources()); print(pyodbc.connect("DSN=drill64", autocommit=True))' should output information like this:

{'ODBC': '', 'drill64': '/opt/mapr/drillodbc/lib/64/libdrillodbc_sb64.so'} <pyodbc.Connection object at 0x7f5ec4a20200>

For more information about setting up Apache Drill, check out this blog post: http://www.bigendiandata.com/2017-05-01-Apache_Drill/ .