[英]Found multiple defaults.yaml resources
when i tried to submit the topology i found this 当我试图提交拓扑时,我发现了这一点
Exception in thread "main" java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar.
at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:115)
at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:135)
at backtype.storm.utils.Utils.readStormConfig(Utils.java:155)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:61)
at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:40)
at trident.myproject.main(myproject.java:288)
But this error appeared after updated in pom.xml by 但是在pom.xml中更新后出现此错误
<scope>compile</scope> instead of <scope>provided</scope>
that because i was an error 因为我是一个错误
An exception occured while executing the Java class. storm/trident/state/StateFactory
here pom file 这里是pom文件
<plugins>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
<archive>
<manifest>
<mainClass>trident.myproject</mainClass>
<!-- <mainClass>crawler.Crawler</mainClass> -->
</manifest>
</archive>
</configuration>
part 2 of pom file pom文件的第2部分
<executions>
<execution>
<id>make-assembly</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
part 3 of pom file pom文件的第3部分
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
</plugins>
There is a fundamental difference in running a topology in LocalCluster
or remotely via StormSubmitter
(which is the default setting in the project). 在
LocalCluster
中运行拓扑或通过StormSubmitter
(这是项目中的默认设置)远程运行拓扑存在根本区别。
The scope of storm-core
is set to <scope>provided</scope>
be the default because those class files are available in the cluster anyway. storm-core
的范围设置为<scope>provided</scope>
作为默认值,因为无论如何这些类文件在集群中都可用。 provided
tells maven, that those classes must not be included in the jar
file that is assembled, thus reducing the size of the jar. provided
告诉maven,这些类不能包含在组装的jar
文件中,从而减小了jar的大小。 Furthermore, this avoids conflicts if files are provided multiple times -- that is what happens with default.yaml
if you change the scope to compile
. 此外,如果多次提供文件,这可以避免冲突 - 如果您将范围更改为
compile
,则会发生default.yaml
。 For those case, all files from storm-core
are packaged into you jar
and submitted to the cluster. 对于这些情况,来自
storm-core
所有文件都打包到jar
并提交给集群。 Storm finds the file defaults.yaml
"locally" (ie, locally on the worker machine in the cluster) and in your jar
. Storm发现文件
defaults.yaml
“本地”(即,在集群中的工作机器上本地)和jar
。 Thus, Storm does not know which one to use and raises an error. 因此,Storm不知道使用哪一个并引发错误。
However, provided
excludes those class files if you run locally, too. 然而,
provided
不包括那些类文件,如果你在本地运行了。 Of course, locally those files are not available automatically but must be included in CLASSPATH when starting up the local JVM. 当然,本地这些文件不能自动使用,但在启动本地JVM时必须包含在CLASSPATH中 。 As
provided
excludes the files from storm-core
you get the ClassNotFound
exception. provided
从storm-core
排除文件,您将获得ClassNotFound
异常。
As an alternative to change the scope each time you want to submit to a different environment, you can set the scope to compile
and include your topology Main/Bolt/Spout classes explicitly in your maven-jar-plugin
settings. 作为每次要提交到不同环境时更改范围的替代方法,您可以将范围设置为
compile
并在maven-jar-plugin
设置中显式包含拓扑Main / Bolt / Spout类。 This explicit inclusion automatically excludes all other files from the jar, ie, all files from storm-core
. 这个显式包含自动从jar中排除所有其他文件,即来自
storm-core
所有文件。
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>2.6</version>
<executions>
<execution>
<id>MyTopology</id>
<phase>package</phase>
<goals>
<goal>jar</goal>
</goals>
<configuration>
<includes>
<include>my/topology/package/**/*.class</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.