是否可以在scala中具有编译器特定的代码段

Question

I have a situation where, I need to certain functionality that is available in Spark library version 1.1.0, But I have two different platforms I need to run this application one. One uses Spark 1.1.0 and the other uses Spark 0.9.1. The functionality available in Spark 1.1.0 is not available in Spark 0.9.1.

That said, is it possible to have some compiler flags in the scala code, so that when compiling with Spark 1.1.0 certain code gets compiled and when compiling using the Spark 0.9.1. library another piece of code gets compiled?

like so :

#ifSpark1.1.0
val docIdtoSeq: RDD[(String, Long)] = listOfDocIds.zipWithIndex()
#endifSpark1.1.0

#ifSpark0.9.1
    val docIdtoSeq: RDD[(String, Long)] = listOfDocIds.mapPartitionsWithIndex{case(partId,it) => it.zipWithIndex.map{case(el,ind) => (el,ind+partId*constantLong)}}
#endifSpark0.9.1

Many thanks

Answer 1

There are several options.

1. Since the two Spark versions are obviously not binary compatible, you would anyway need to provide two artifacts of your projects. Create a simple common API layer and then add two thin sub-projects in a multi-project sbt build that provide that layer for either Spark version.
2. Use sbt-buildinfo to generate compile-time symbols for your Spark version, then use a macro method that pastes the two different types of method invocations above, depending on the Spark version.
3. Use runtime reflection