Apache Spark Maven POM错误

Question

I am trying out Apache Spark and have a little problem that I am stuck with. I have this JavaSparkHiveExample.java example from the Apache Spark Github together with a pom.xml that I have created as a Maven project in IntelliJ IDEA.

I am able to run other Spark examples (using another simpler POM) but this one is giving me the following errors:

• Scala is installed in Idea

I am new with Maven and would therefore appreciate some things I could try in order to solve this problem.

Answer 1

Issue is with the value of $project.version. It is referring to your project' version (2.3.0-snapshot). There isn't any maven dependency with this version in Maven central repository, hence you are facing this issue. Instead of using the project version, you can add a new property like this and refer it for all dependency version

<properties>
 <spark.version>1.6.2</spark.version>
</properties>

and then use it in the dependency

<dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-core_${scala.binary.version}</artifactId>
      <version>${spark.version}</version>
    </dependency>

Make sure the version you are using is available in maven repository https://mvnrepository.com/