堆栈内存溢出
  繁体   English   中英

将 jar 文件加载到 Apache Flink 仪表板时出现错误

[英]I am getting an error while loading a jar file to Apache Flink dashboard

 原文  2021-05-18 15:04:48  7   2    java/ apache-flink

问题描述

我是 Java 和 Apache Flink 的新手。 我使用 Eclipse 和 Java 创建了（Apache Maven 项目）。 这是一个从 Apache Kafka 主题中读取数据的 Flink 程序。

我在用：

  • Eclipse
  • Flink（版本：1.7.2）
  • Java（版本：1.8）

这是我的程序：


import java.util.Properties;

import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer;

public class FlinkApplication {
    
    public static void main(String[] args) throws Exception {
        
        
    String bootstrapServers = "XX.XX.XXX.XXX:XXX";

    
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    


    // Set up the Consumer and create a datastream from this source
    Properties properties = new Properties();
    properties.setProperty("bootstrap.servers", bootstrapServers);
    FlinkKafkaConsumer<String> flinkConsumer = new FlinkKafkaConsumer<>(
        "first_topic",                          // input topic
        new SimpleStringSchema(),           // serialization schema
        properties);                        // properties
     

    DataStream<String> readingStream = env.addSource(flinkConsumer);
    
    env.execute();
}
}

除了这段代码，下面是我的 pom.xml

  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.elayat.flink</groupId>
  <artifactId>FlinkApplication</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>FlinkApplication</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <dependencies>
  
     
    <!-- https://mvnrepository.com/artifact/com.google.code.gson/gson -->
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.6</version>
</dependency>
  
<!-- https://mvnrepository.com/artifact/log4j/log4j -->
<dependency>
    <groupId>log4j</groupId>
    <artifactId>log4j</artifactId>
    <version>1.2.17</version>
</dependency>

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.30</version>
    
</dependency>
  
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-java</artifactId>
            <version>1.7.2</version>
        </dependency>
        
<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-java -->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-streaming-java_2.12</artifactId>
    <version>1.7.2</version>
    <scope>provided</scope>
</dependency>

        
        <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-clients -->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-clients_2.12</artifactId>
    <version>1.7.2</version>
</dependency>
        
        <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-connector-kafka -->
<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-connector-kafka_2.12</artifactId>
    <version>1.7.2</version>
</dependency>
        
  
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    
       <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-shade-plugin</artifactId>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <transformers>
            <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
              <mainClass>com.elayat.flink.FlinkApplication.FlinkApplication</mainClass>
            </transformer>
          </transformers>
          <filters>
            <filter>
              <artifact>*:*</artifact>
              <excludes>
                <exclude>javax/xml/**</exclude>
                <exclude>org/apache/xerces/**</exclude>
                <exclude>org/w3c/**</exclude>
                <exclude>META-INF/*.SF</exclude>
                <exclude>META-INF/*.DSA</exclude>
                <exclude>META-INF/*.RSA</exclude>
              </excludes>
            </filter>
          </filters>
        </configuration>
      </plugin>
    </plugins>
  </build>
  
 
</project>

我已将此程序编译为 jar 文件并将其加载到 Flink 仪表板，但它给出了以下错误：

很抱歉，出了点问题。 服务端响应： org.apache.flink.client.program.ProgramInvocationException: 主方法报错。 [![在此处输入图像描述][1]][1]

当我检查 Flink 集群的日志时，它显示以下 WARN 和错误：

2021-05-18 14:31:25,167 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: Neither a 'Main-Class', nor a 'program-class' entry was found in the jar file.

2021-05-18 14:31:57,982 WARN org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Configuring the job submission via query parameters is deprecated. Please migrate to submitting a JSON request instead.

2021-05-18 14:31:58,003 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

2021-05-18 19:32:32,124 ERROR org.apache.flink.runtime.webmonitor.handlers.JarDeleteHandler  - Exception occurred in REST handler: File 8b2141cd-a854-4958-b192-efeb58db63e6_FlinkApplication-0.0.1-SNAPSHOT.jar does not exist in /tmp/flink-web-38332dc6-8876-4610-92a2-8ad4badc2460/flink-web-upload.

2021-05-18 19:33:07,093 WARN  org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Configuring the job submission via query parameters is deprecated. Please migrate to submitting a JSON request instead.

2021-05-18 19:33:07,116 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

2021-05-18 19:33:42,322 WARN  org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Configuring the job submission via query parameters is deprecated. Please migrate to submitting a JSON request instead.

2021-05-18 19:33:42,342 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

2021-05-18 19:34:03,232 WARN  org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Configuring the job submission via query parameters is deprecated. Please migrate to submitting a JSON request instead.

2021-05-18 19:34:03,252 ERROR org.apache.flink.runtime.webmonitor.handlers.JarRunHandler    - Exception occurred in REST handler: org.apache.flink.client.program.ProgramInvocationException: The main method caused an error.

这是我在 cmd 中尝试查看它是否可执行时 jar 文件显示的错误。

Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/functions/source/SourceFunction
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:650)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:632)
Caused by: java.lang.ClassNotFoundException:  
        org.apache.flink.streaming.api.functions.source.SourceFunction
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 7 more

2 个解决方案

解决方案1
 0  2021-05-18 23:56:30

在 jar 文件中未找到“主类”和“程序类”条目。

听起来您上传的 JAR 文件不是阴影文件，或者您在 XML 阴影插件定义中的mainClass标签配置不正确

您可以将 jar 文件重命名为 zip 文件，将其解压缩，然后查看清单文件以查看是否定义了主 class

您还可以执行java -jar /path/to/file.jar来判断 JAR 是否实际可执行

相关问题

如何使用 Maven 创建具有依赖关系的可执行 JAR？

解决方案2
 0  2021-05-19 14:50:30

问题已解决。 这是由于main方法中缺少Map Function。 因为仪表板不会在没有 map function 的情况下加载 jar 文件。 在我的程序中编写 Map Function 后，我已经能够在 Flink Dashboard 中加载和运行。 下面是我在程序中使用的 Map Function：

DataStream<String> readingStream = env.addSource(flinkConsumer);
    readingStream.rebalance().map(new RichMapFunction <String, String>(){

private static final long serialVersionUID = -2547861355L; //random number 
        
        public String map(String valueFromKafka) throws Exception{    
            System.out.println(valueFromKafka);
            return "Successfull";
        }
    });
    
    env.execute();
}
}

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 使用pdfbox.jar将pdf文件转换为文本文件时，出现以下错误。如何解决这些问题？ 在执行可运行的jar文件（java代码）时，我正在获取java.lang.reflect.InvocationTargetException 将wsdl转换为JAR文件时出现错误 使用ant构建jar文件时出错 Apache Flink - 外部 Jar 更改为war文件后，我已将我的应用程序创建为jar 现在我在应用程序中部署时出现404错误 tomcat 8.5.9 为什么会出现错误“加载类时找不到类。错误”？ Apache Flink 仪表板未显示指标 我正在尝试使用zip4j jar文件提取受密码保护的ZIP文件，但出现以下错误 在我的应用程序中使用库 github.barteksc.pdfviewer 加载 PDF 时出现错误
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM