在Windows 8 x64 PC上为本地模式编译Apache Spark 1.1.0？ 查看特定错误

Question

I've explicitly installed Java 8 SDK, Scala-IDE (modified eclipse), the normal Eclipse IDE, and sbt though not in that order. I don't really remember the order in which I installed those. I was sort of confused how to set up my environment variables when I have both the vanilla Eclipse IDE and Scala IDE's installed as opposed to just the command line tools, as these installations seem to organized the language environment differently that the command line installations do? So I wasn't what to point the variables to?

My understanding is that the CLASS_PATH environment variable is for user-defined classes, of which for my current project I have none. Beyond this. I have the following environment variable setup. I am on windows 8 x64 .

SBT_HOME = C:\\Program Files (x86)\\sbt

JRE_HOME = C:\\Program Files\\Java\\jre1.8.0_20

JAVA_HOME = C:\\Program Files\\Java\\jdk1.8.0_20

Path = %JRE_HOME%\\bin;%JAVA_HOME%\\bin;C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Program Files (x86)\\scala\\bin;C:\\Program Files (x86)\\sbt\\bin;...(only relevant paths shown)

Upon downloading and extracting Apache Spark 1.1.0 and uncompressing it and entering into the uncompressed top level folder ~/(apache spark folder)/ , and attempting to run sbt/sbt assembly of which I receive the following error using the cygwin command prompt.

[error] (streaming-flume-sink/compile:compile) java.io.IOException: Cannot run p
rogram "javac": CreateProcess error=2, The system cannot find the file specified

[error] (core/compile:compile) java.io.IOException: Cannot run program "javac":
CreateProcess error=2, The system cannot find the file specified
[error] Total time: 551 s, completed Oct 28, 2014 5:24:00 PM

So the error appeared to be that I can't run javac in cygwin. Well, I can't. So I tried the windows prompt as well, where my command javac actually works (since the enviro variables are setup.) And I get the following error when again attempting

`sbt/sbt assembly`. 
C:\Program Files (x86)\spark-1.1.0>sbt\sbt assembly
'sbt\sbt' is not recognized as an internal or external command,
operable program or batch file.
>sbt/sbt assembly
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; sup
port was removed in 8.0
[info] Loading project definition from C:\Program Files (x86)\spark-1.1.0\project\projec
t
[info] Loading project definition from C:\Users\jim\.sbt\0.13\staging\ec3aa8f391
11944cc5f2\sbt-pom-reader\project
[warn] Multiple resolvers having different access mechanism configured with same
 name 'sbt-plugin-releases'. To avoid conflict, Remove duplicate project resolve
rs (`resolvers`) or rename publishing resolver (`publishTo`).
[info] Loading project definition from C:\Program Files (x86)\spark-1.1.0\project
[error] Expected symbol
[error] Not a valid command: /
[error] /sbt
[error]  ^

but all kinds of variants of the same command don't work either, switching out slash types( sbt\\sbt assembly ).

I've also got spark in a folder that has no spaces: C:\\Program Files (x86)\\spark .

What in tarnation do I need to do to assemble/compile Spark?

Thanks a million.

Answer 1

Ran into the same problem in Windows8 and following steps fixed the issue

• Create a folder which doesn't have space and copy over the content from Program Files/Java
• Update the PATH and JAVA_HOME to the new path
• Launch Cygwin and Try running javac command to make sure the environment variables are set properly
• Though not related to this issue, Scala Installed directory should not have any space. I see that you have installed in Program Files