How to install gdal on databricks cluster?

Question

I am trying to install the package GDAL on an Azure Databricks cluster. In no way I can get it to work.

Approaches that I've tried but didn't work:

1. Via the library tab of the corresponding cluster --> Install New --> PyPi ( under Library Source ) --> Entered gdal under Package

2. Tried all approaches mentioned on https://forums.databricks.com/questions/13738/gdal-installation.html . None of them worked.

Details:

1. Runtime: 6.1 (includes Apache Spark 2.4.4, Scala 2.11) (When using runtime 3.5 I got GDAL to work, however an update to a higher runtime was necessary for other reasons.)

2. We're using python 3.7.

Answer 1

I have already replied similar type of question. Please check the below link would help you to install the required library:

How can I download GeoMesa on Azure Databricks?

For your convenience I am pasting the Answer again... just you need to choose your required library from the search area.

You can install GDAL Library directly into your Databricks cluster.

1) Select the Libraries option then a new window will open.

2) Select the maven option and click on 'search packages' option

3) Search the required library and select the library/jar version and choose the 'select' option. Thats it.

After the installation of the library/jar, restart your cluster. Now import the required classes in your Databricks notebook. I hope it helps. Happy Coding..

Answer 2

Finally we got it working by using an ML runtime in combination with the answer given in forums.databricks.com/answers/21118/view.html. Apparently the ML-runtimes contain conda, which is needed for the answer given in the previous link.