• 如果您觉得本站非常有看点,那么赶紧使用Ctrl+D 收藏吧

Storm-Kafka:java.lang.NoClassDefFoundError:com/google/common/base/Strings

maven 来源:konquestor 6次浏览

我想使用storm-core 1.0.2和kafka-spout 1.0.2来运行风暴LocalCluster。Storm-Kafka:java.lang.NoClassDefFoundError:com/google/common/base/Strings

我的POM文件看起来像下面

<dependencies> 
    <!-- Storm Dependency --> 
    <dependency> 
     <groupId>org.apache.storm</groupId> 
     <artifactId>storm-core</artifactId> 
     <version>1.0.2</version> 
     <scope>provided</scope> 
    </dependency> 

    <!-- Storm Kafka Dependency --> 
    <dependency> 
     <groupId>org.apache.storm</groupId> 
     <artifactId>storm-kafka</artifactId> 
     <version>1.0.2</version> 
     <exclusions> 
      <exclusion> 
       <groupId>org.apache.zookeeper</groupId> 
       <artifactId>zookeeper</artifactId> 
      </exclusion> 
      <exclusion> 
       <groupId>log4j</groupId> 
       <artifactId>log4j</artifactId> 
      </exclusion> 
     </exclusions> 
    </dependency> 
..... 
</dependencies> 
<build> 
    <resources> 
     <resource> 
      <directory>src/main/resources</directory> 
     </resource> 
    </resources> 
    <plugins> 

     <plugin> 
      <artifactId>maven-assembly-plugin</artifactId> 
      <executions> 
       <execution> 
        <phase>package</phase> 
        <goals> 
         <goal>single</goal> 
        </goals> 
       </execution> 
      </executions> 
      <configuration> 
       <descriptorRefs> 
        <descriptorRef>jar-with-dependencies</descriptorRef> 
       </descriptorRefs> 
       <archive> 
        <manifest> 
         <mainClass>com.walmartlabs.midas.storm.Topology</mainClass> 
        </manifest> 
       </archive> 
      </configuration> 
     </plugin> 

     <plugin> 
      <groupId>org.codehaus.mojo</groupId> 
      <artifactId>exec-maven-plugin</artifactId> 
      <version>1.2.1</version> 
      <executions> 
       <execution> 
        <goals> 
         <goal>exec</goal> 
        </goals> 
       </execution> 
      </executions> 
      <configuration> 
       <executable>java</executable> 
       <includeProjectDependencies>true</includeProjectDependencies> 
       <includePluginDependencies>false</includePluginDependencies> 
       <classpathScope>compile</classpathScope> 
       <mainClass>${storm.topology}</mainClass> 
      </configuration> 
     </plugin> 
    </plugins> 
</build> 

我用下面的命令来运行本地风暴集群。

mvn package 
storm jar target/myproject-0.1-SNAPSHOT-jar-with-dependencies.jar com.company.project.storm.Topology 

我得到以下异常

Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/Strings 
    at org.apache.storm.kafka.KafkaSpout.declareOutputFields(KafkaSpout.java:184) 
    at org.apache.storm.topology.TopologyBuilder.getComponentCommon(TopologyBuilder.java:431) 
    at org.apache.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:135) 
    at com.walmartlabs.midas.storm.Topology.submitLocalTopology(Topology.java:50) 
    at com.walmartlabs.midas.storm.Topology.main(Topology.java:75) 
Caused by: java.lang.ClassNotFoundException: com.google.common.base.Strings 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357) 

在谷歌共同依赖的是,风暴卡夫卡神器。我已经将storm-kafka的范围设置为默认值,所以我相信它的所有依赖项都应该放在我的jar中。

如果我用只读一些文件的简单喷口替换KafkaSpout,一切工作正常。 我相信,虽然包装的jar我没有包装风暴卡夫卡神器的所有依赖。这里有什么不对?


===========解决方案如下:

https://github.com/apache/storm/tree/v1.0.2/external/storm-kafka#using-storm-kafka-with-different-versions-of-kafka

风暴1.0.2改变“kafka_ [scala_version]”到“提供”伪影的范围,这意味着,需要添加此作为依赖性。这是为了在scala版本和/或kafka版本上具有组合的灵活性。


版权声明:本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系管理员进行删除。
喜欢 (0)