如何通过Spark即服务笔记本连接到在BigInsights on Cloud上运行的远程配置单元？

Question

I would like to connect to a hive service running on BigInsights from a spark notebook using jdbc. The jdbc url format is:

jdbc:hive2://${env.hostname}:10000/default;ssl=true;sslTrustStore=./truststore.jks;trustStorePassword=mypassword;

As you can see from the url, this connection requires a truststore. How should I make the truststore available to spark as a service?

Update 1:

• The certificate is not issued by a well known CA.
• Tenants have no access to the JRE/JDK on the service.

Update 2:

I can add the certificate and truststore using the following:

with open('certificate', 'w') as f:
    f.write('''
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
''')

!keytool -import -trustcacerts -alias biginsights -file certificate -keystore truststore.jks -storepass mypassword -noprompt

The final part of the question now is how to add a jar to python notebook on bluemix spark?

Answer 1

Here is one of the may be odd way around(Not tried):-

In Notebook, If you are already not in python shell, then switch to python shell and then if you have your truststore available to be downloaded from URL , you can download it this way and run cell:- !wget

if biginsights server lets you use SSH to access the keystore, use !scp to download the truststore.

Once downloaded, i would suggest try to use !pwd which will give you path /gpfs/fs01/user/s027-20bcfe6e4297e8-2c631c8ff999/notebook/notebooks

If you do !ls, you can see your downloaded trustore file. See if you can give the FULL Absolute path to trustore in JDBC URL.

Thanks, Charles.

Answer 2

First find out if the SSL certificate is issued by a "well-known" CA authority. By "well-known", I meant those CA authorities whose signing certificates are already included in the jdk truststore. If yes, find out from the jdk documentation the location and name of the truststore and provided that in the sslTrustStore parameter. If no, then has to download the cert, add to a truststore and upload the truststore to a location accessible by spark as a service.