Can't get Spark NLP working on Databricks

Question

I've done the following:

import pyspark
from pyspark.sql import SparkSession

from pyspark import SparkContext, SparkConf, SQLContext

spark = SparkSession \
    .builder \
    .appName('Amazon ETL') \
    .config('spark.jars.packages', 'com.johnsnowlabs.nlp:spark-nlp_2.11:2.4.5') \
    .getOrCreate()

Plus...

import sparknlp
from sparknlp.annotator import *

and I get this error message: Error Message

Any help would be much appreciated!

Thanks, Kelvin

Answer 1

I solved the problem by preinstalling the JAVA dependencies by install the sparknlp lib using maven.

In your Cluster-Libraries section, choose install new , then choose maven , and type in com.johnsnowlabs.nlp:spark-nlp_2.11:2.5.0